SkimmerDB Docs
Guides Markdown

Importing files

Load CSV, NDJSON or Parquet into a namespace in one request, parsed straight into typed columns, every row visible together.

For data that already lives in files (exports, logs, tables from a warehouse or a lake), import it in one request instead of writing documents in batches. Every row becomes a document; the file is parsed straight into typed columns, and all rows become visible together, or none.

POST /v1/namespaces/{name}/import?format=csv|ndjson|parquet&id=<column>&text=<col,col> with the file as the body. Write scope; up to 2 GB and 1,000,000 rows a request.

B=https://api.skimmerdb.com/v1
curl -s "$B/namespaces/support/import?format=csv&id=ticket&text=subject,body" \
  -H "Authorization: Bearer $KEY" --data-binary @tickets.csv
import os, requests

with open("tickets.csv", "rb") as f:
    r = requests.post(
        "https://api.skimmerdb.com/v1/namespaces/support/import",
        params={"format": "csv", "id": "ticket", "text": "subject,body"},
        headers={"Authorization": f"Bearer {os.environ['SKIMMERDB_KEY']}"},
        data=f,
    )
print(r.json())
import { readFile } from "node:fs/promises";

const url = "https://api.skimmerdb.com/v1/namespaces/support/import?format=csv&id=ticket&text=subject,body";
const res = await fetch(url, {
  method: "POST",
  headers: { Authorization: `Bearer ${process.env.SKIMMERDB_KEY}` },
  body: await readFile("tickets.csv"),
});
console.log(await res.json());

The answer:

json
{"written": 52000, "deleted": 0, "version": 4, "stored_bytes": 18412203, "rows": 52000, "took_ms": 1840, "created": false}

An import to a name that doesn't exist makes the namespace (text only; "created": true).

Parameters

ParameterMeaning
formatcsv (the default), ndjson or parquet.
idThe column whose values are the documents' ids. Without it, ids are made from the import and the row number. A later row with the same id wins.
textThe columns that make each document's searchable text, as name: value lines (note: refund requested). None: rows are found by SQL and filters only.
schemaCSV only: a CREATE TABLE statement (URL-encoded) giving the columns and their types, for files without a header row.

Every column is metadata, whether or not it's in text: filters, SQL and search hits all see it.

Tip Pick a natural key for id (an order number, a ticket number) and re-importing the same file replaces rather than duplicates. See Data model.

Types

FormatHow columns are typed
CSVA header row is required (or schema=). Kinds are inferred: integers and true/false are int, other numbers float, YYYY-MM-DD a date, YYYY-MM-DD HH:MM:SS a time, anything else text.
NDJSONOne JSON object a line, its keys the columns, typed like meta (integers and booleans int, other numbers float, strings text).
ParquetKeeps its types. Lists and maps aren't supported yet.

date and time columns compare with strings like '2026-10-01' in SQL and work with DATE_TRUNC. See Analytics with SQL.

NULL versus empty

NDJSON and Parquet keep NULL separate from empty strings. In CSV, an empty field (quoted or not) means a missing value. Use NDJSON or Parquet when the difference between NULL and empty text matters.

NDJSON

One flat object a line:

events.ndjson
{"event_id": "e-1", "ts": 1760572937, "agent": "agent-1", "tool": "search", "ok": false, "message": "error E504 upstream timeout"}
{"event_id": "e-2", "ts": 1760572951, "agent": "agent-1", "tool": "sql_query", "ok": true, "message": "ok"}

Import it:

curl
curl -s "$B/namespaces/events/import?format=ndjson&id=event_id&text=message" \
  -H "Authorization: Bearer $KEY" --data-binary @events.ndjson

CSV without a header

Give the columns with schema=, a URL-encoded CREATE TABLE:

curl
SCHEMA=$(jq -rn --arg s "CREATE TABLE orders (order_id TEXT, placed_at TIMESTAMP, country TEXT, amount DOUBLE)" '$s|@uri')
curl -s "$B/namespaces/orders/import?format=csv&id=order_id&schema=$SCHEMA" \
  -H "Authorization: Bearer $KEY" --data-binary @orders-noheader.csv

Types: TEXT (or VARCHAR, STRING), DOUBLE (or FLOAT, REAL), DATE, TIMESTAMP (or DATETIME), and integer types (INT, BIGINT, ...).

Speed and limits

Measured on 8 vCPUs: CSV about 440,000 rows a second, Parquet about 500,000, NDJSON about 175,000. A request takes up to 2 GB and 1,000,000 rows; split bigger files into several imports. Each import is one atomic write: its rows become visible together, after which they compact with the rest of the namespace like any write.

An import is metered as a write of that many documents and the file's bytes. See Limits and plans.