# Your own bucket (S3, R2, B2, MinIO) > Import files from your bucket and deliver exports into it, on AWS S3 or any S3-compatible store such as Cloudflare R2, Backblaze B2, Wasabi or MinIO. Your keys are used for the request and never stored. Your data can stay in your bucket. SkimmerDB reads files from it and writes files into it with keys you pass in the request, on AWS S3 or on any S3-compatible service: - **[Import files](#import-files-from-your-bucket)** from a key, a prefix or a pattern: `POST /v1/namespaces/{name}/import/bucket`. - **[Deliver exports](#deliver-exports-to-your-bucket)** of a SQL answer as CSV, NDJSON or Parquet: `POST /v1/sql/exports` with a `destination`. - **[Sync a DuckLake table](/guides/ducklake)** whose data lives in the bucket: `POST /v1/namespaces/{name}/lake/sync`. All three take the same `s3` object: ```json "s3": { "endpoint": "https://.r2.cloudflarestorage.com", "region": "auto", "access_key_id": "...", "secret_access_key": "...", "session_token": "" } ``` Leave `endpoint` out for AWS S3 and give the bucket's `region`. Leave the keys out for a **public bucket** (such as [AWS Open Data](https://registry.opendata.aws/)): requests then go unsigned, for imports only. For anything else, `endpoint` is the service's address: scheme and host, with an optional port and no path. Requests are path-style (`https://endpoint/bucket/key`) and signed with AWS Signature Version 4. | Service | `endpoint` | `region` | |---|---|---| | AWS S3 | (leave out) | the bucket's, e.g. `us-east-1` | | Cloudflare R2 | `https://.r2.cloudflarestorage.com` | `auto` | | Backblaze B2 | `https://s3..backblazeb2.com` | e.g. `us-west-004` | | Wasabi | `https://s3..wasabisys.com` | e.g. `us-east-1` | | DigitalOcean Spaces | `https://.digitaloceanspaces.com` | e.g. `nyc3` | | Google Cloud Storage (XML API, HMAC keys) | `https://storage.googleapis.com` | `auto` | | MinIO and other self-hosted stores | `https://minio.example.com:9000` | your server's, often `us-east-1` | ## Import files from your bucket ```sh title="curl" curl -s https://api.skimmerdb.com/v1/namespaces/events/import/bucket \ -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \ -d '{ "path": "s3://events/2026/10/*.parquet", "id": "event_id", "text": ["message"], "s3": {"endpoint": "https://.r2.cloudflarestorage.com", "region": "auto", "access_key_id": "...", "secret_access_key": "..."} }' ``` ```python title="Python" import os from skimmerdb import SkimmerDB db = SkimmerDB() r2 = { "endpoint": "https://.r2.cloudflarestorage.com", "region": "auto", "access_key_id": os.environ["R2_ACCESS_KEY_ID"], "secret_access_key": os.environ["R2_SECRET_ACCESS_KEY"], } answers = db.namespace("events").import_bucket( "s3://events/2026/10/*.parquet", r2, id="event_id", text=["message"] ) print(sum(a.rows for a in answers), "rows") ``` ```ts title="TypeScript" import { SkimmerDB } from "skimmerdb"; const db = new SkimmerDB(); const r2 = { endpoint: "https://.r2.cloudflarestorage.com", region: "auto", access_key_id: process.env.R2_ACCESS_KEY_ID!, secret_access_key: process.env.R2_SECRET_ACCESS_KEY!, }; const answers = await db.namespace("events").importBucket("s3://events/2026/10/*.parquet", r2, { id: "event_id", text: ["message"], }); ``` `path` is one of these: - **One file:** `s3://bucket/exports/orders.csv`. - **Every file under a prefix:** `s3://bucket/exports/`. - **A pattern:** `s3://bucket/exports/*.parquet` matches the files under `exports/` whose names end in `.parquet`. One `*`, in the last part of the path. The format comes from each file's extension (`.csv`, `.ndjson`, `.jsonl` or `.json`, `.parquet`) unless you give `format`. `id`, `text` and `schema` work as they do for [an upload](/guides/import#parameters). The answer lists the files imported: ```json {"files": [{"key": "2026/10/01.parquet", "bytes": 48210331, "rows": 1000000, "written": 1000000}], "written": 1000000, "rows": 1000000, "bytes": 48210331, "next": "", "took_ms": 2140, "created": true} ``` - **In key order, in pages.** One request takes up to 100 files and 4 GB, and each file can be up to 2 GB. When more are left, `next` names the last key taken: pass it as `after` to go on. The Python and TypeScript helpers do that for you until `next` is empty. - **Ids.** With `id`, each row's value names its document. Without it, a row is `:` (from 0), so importing the same file again replaces its documents instead of adding them twice. - **Each file is its own import.** All of a file's rows become visible together. If a file fails (a bad row, a read error), the request stops. The error answer lists the files already in, `failed` (the key that failed) and `next` (pass it as `after` to retry from that file). Ids make a rerun safe: a document written again replaces itself. - **The next file downloads while the current one is parsed**, over connections kept open. - Metered as writes of the documents and the files' bytes, like an upload. ## Deliver exports to your bucket Add `destination` to an [export](/guides/export#exports). The file goes into your bucket by multipart upload (parts of 32 MB). No copy is kept on our side, so there's nothing to download and nothing expires. ```sh title="curl" curl -s https://api.skimmerdb.com/v1/sql/exports -H "Authorization: Bearer $KEY" \ -H "Content-Type: application/json" -d '{ "sql": "SELECT * FROM events WHERE day >= '\''2026-10-01'\''", "format": "parquet", "destination": { "path": "s3://analytics/exports/events-october.parquet", "s3": {"endpoint": "https://.r2.cloudflarestorage.com", "region": "auto", "access_key_id": "...", "secret_access_key": "..."} } }' ``` ```python title="Python" e = db.wait_export(db.export( "SELECT * FROM events WHERE day >= '2026-10-01'", format="parquet", destination="s3://analytics/exports/events-october.parquet", s3=r2, ).id) print(e.destination, e.bytes) ``` ```ts title="TypeScript" const e = await db.waitExport((await db.export( "SELECT * FROM events WHERE day >= '2026-10-01'", "parquet", { path: "s3://analytics/exports/events-october.parquet", s3: r2 }, )).id); ``` - A `path` ending in `/` gets a name made for it: `export-