# Embeddings

> Have SkimmerDB embed your documents' text and semantic searches, with your own OpenAI, Voyage, Gemini or Cohere key, or hosted at cost.

Vector search needs embeddings. You can make them yourself and send `vector` with each document
([Vector and hybrid search](/guides/vectors)), or let SkimmerDB make them: give a namespace an
embedding model, write documents with plain `text`, and search with `"semantic": "..."`.

There are two ways to pay for it:

| | Your own key | Hosted |
|---|---|---|
| `provider` | `openai`, `voyage`, `gemini` or `cohere` | `skimmerdb` |
| Models | any of the provider's embedding models | `voyage-3.5-lite` |
| Who bills the tokens | the provider bills you directly | SkimmerDB, at Voyage's price ($0.02 per million tokens), no margin |
| Setup | store the key once (dashboard or API) | none |
| Free plan | not counted against anything | 5 million tokens a month included |

Either way, the same copy of each document serves text search, vectors and SQL, and a vector
you send yourself always wins over one SkimmerDB would make.

## 1. Store your provider key

Skip this for hosted embedding. A key is checked with the provider when you store it (one tiny
embedding, billed by the provider), then kept encrypted. It's never shown again: listings show
only its first and last characters. Storing keys needs an admin key or the owner's login, as
managing API keys does. In the [dashboard](https://skimmerdb.com/dashboard): **Embedding**,
pick the provider, paste the key, **Save key**.

```sh title="curl"
B=https://api.skimmerdb.com/v1
curl -s -X PUT $B/embedding-keys/openai -H "Authorization: Bearer $KEY" \
  -d '{"key": "sk-proj-..."}'
# {"provider": "openai", "hint": "sk-…a1b2", "created_at": "...", "updated_at": "..."}
```

```python title="Python"
db.set_embedding_key("openai", "sk-proj-...")
```

```ts title="TypeScript"
await db.setEmbeddingKey("openai", "sk-proj-...");
```

`GET /v1/embedding-keys` lists the providers you have keys for; `DELETE
/v1/embedding-keys/{provider}` forgets one at once. One key per provider per tenant: storing
another replaces it.

## 2. Make a namespace with a model

`embed` names the provider, optionally the model, and optionally `dims`, the vector length to
ask the model for. Without `dims` you get the model's own length (up to 2,048).

```sh title="curl"
curl -s $B/namespaces -H "Authorization: Bearer $KEY" \
  -d '{"name": "kb", "embed": {"provider": "openai", "model": "text-embedding-3-small", "dims": 512}}'

# hosted: no key needed
curl -s $B/namespaces -H "Authorization: Bearer $KEY" \
  -d '{"name": "kb2", "embed": {"provider": "skimmerdb", "dims": 512}}'
```

```python title="Python"
kb = db.create_namespace("kb", embed={"provider": "openai", "model": "text-embedding-3-small", "dims": 512})
kb2 = db.create_namespace("kb2", embed={"provider": "skimmerdb", "dims": 512})
```

```ts title="TypeScript"
const kb = await db.createNamespace("kb", { embed: { provider: "openai", model: "text-embedding-3-small", dims: 512 } });
const kb2 = await db.createNamespace("kb2", { embed: { provider: "skimmerdb", dims: 512 } });
```

The model is checked when you set it (one tiny embedding): an unknown model, a missing key or a
length the model can't make is refused then, not at your first write.

| Provider | Default model | Lengths |
|---|---|---|
| `openai` | `text-embedding-3-small` | 1,536, or any shorter `dims` (`text-embedding-3-large`: 3,072, so ask for 2,048 or less) |
| `voyage` | `voyage-3.5-lite` | 1,024, or 256, 512, 2,048 |
| `gemini` | `gemini-embedding-001` | 3,072 (SkimmerDB asks for 2,048), or a shorter `dims` |
| `cohere` | `embed-v4.0` | 1,536, or 256, 512, 1,024 |
| `skimmerdb` | `voyage-3.5-lite` (the only one) | 1,024, or 256, 512, 2,048 |

## 3. Write text

Documents with `text` and no `vector` are embedded before they're written. A write of 10,000
documents is embedded in batches, a few requests at once, and is still all-or-nothing: if the
provider fails, nothing is written and the answer says why.

```sh title="curl"
curl -s $B/namespaces/kb/docs -H "Authorization: Bearer $KEY" -d '{"wait": true, "docs": [
  {"id": "refunds", "text": "To get your money back, open the order and choose Request refund.", "meta": {"topic": "billing"}},
  {"id": "password", "text": "Forgot your password? Use the reset link on the sign-in page.", "meta": {"topic": "auth"}}
]}'
```

```python title="Python"
kb.write([
    {"id": "refunds", "text": "To get your money back, open the order and choose Request refund.", "meta": {"topic": "billing"}},
    {"id": "password", "text": "Forgot your password? Use the reset link on the sign-in page.", "meta": {"topic": "auth"}},
], wait=True)
```

```ts title="TypeScript"
await kb.write([
  { id: "refunds", text: "To get your money back, open the order and choose Request refund.", meta: { topic: "billing" } },
  { id: "password", text: "Forgot your password? Use the reset link on the sign-in page.", meta: { topic: "auth" } },
], { wait: true });
```

- **Changing a document's text** (`PATCH .../docs` with `text`) embeds it again; a change to
  `meta` alone doesn't call the provider.
- **What's embedded:** the `text`, up to its first 6,000 characters for OpenAI and Gemini (their
  models take about 8,000 tokens) and 32,000 for Voyage, Cohere and hosted (the provider cuts the
  rest). For long documents, write each chunk as its own document (`docs/install.md#3`).
- **Your own vectors still work** in the same namespace: a document with `vector` keeps it.

## 4. Search by meaning

`semantic` is embedded with the namespace's model (as a query, for the providers that tell
documents and queries apart), then searched like a vector: nearest first, with `q` and
`filter` narrowing the candidates.

```sh title="curl"
curl -s $B/namespaces/kb/search -H "Authorization: Bearer $KEY" \
  -d '{"semantic": "how do I get reimbursed", "limit": 5}'

# hybrid: by meaning, among documents containing "refund" with topic billing
curl -s $B/namespaces/kb/search -H "Authorization: Bearer $KEY" \
  -d '{"semantic": "money back", "q": "refund", "filter": {"topic": "billing"}}'
```

```python title="Python"
hits = kb.search(semantic="how do I get reimbursed", limit=5).hits
hits = kb.search("refund", semantic="money back", filter={"topic": "billing"}).hits
```

```ts title="TypeScript"
const { hits } = await kb.search("", { semantic: "how do I get reimbursed", limit: 5 });
await kb.search("refund", { semantic: "money back", filter: { topic: "billing" } });
```

Answers are the same shape as a vector search: `hits` with `id`, `score`, `meta`, and the
query's cost. Over MCP, the `search` tool takes `semantic` too.

## Change or stop the model

`PATCH /v1/namespaces/{name}` sets, changes or removes a namespace's model. A namespace's vector
length is fixed, so a new model must make vectors of the same length (most current models take
`dims`, so this usually works: ask for the namespace's length). Documents already written keep
their vectors: write them again to embed them with the new model.

```sh title="curl"
curl -s -X PATCH $B/namespaces/kb -H "Authorization: Bearer $KEY" \
  -d '{"embed": {"provider": "voyage", "model": "voyage-3.5"}}'

# stop: text is written without vectors again
curl -s -X PATCH $B/namespaces/kb -H "Authorization: Bearer $KEY" -d '{"embed": null}'
```

```python title="Python"
kb.set_embedding("voyage", "voyage-3.5")
kb.set_embedding(None)
```

```ts title="TypeScript"
await kb.setEmbedding({ provider: "voyage", model: "voyage-3.5" });
await kb.setEmbedding(null);
```

> **Note** Mixing models in one namespace makes their vectors incomparable: a query embedded by
> the new model is far from documents embedded by the old one. Switch models on a namespace you
> rewrite, or make a new namespace and write to both until it has caught up.

Agents can do this themselves over MCP: `create_namespace` takes `embed_provider` and
`embed_model`, and `update_namespace` sets, changes or stops them. Storing provider keys is
deliberately not an MCP tool: paste keys in the dashboard, not into an agent's context.

## Errors

Nothing is written when embedding fails. See [Errors](/reference/errors) for the shape.

| Status | `code` | When | What to do |
|---|---|---|---|
| 422 | `embedding_key_missing` | the namespace's provider has no key in this tenant | store one, or send vectors |
| 422 | `embedding_key_invalid` | the provider refused the key | store a working key |
| 422 | `embedding_rejected` | the provider refused the request (an unknown model, say) | check the model |
| 409 | `embedding_dims` | the model can't make vectors of the namespace's length | another model, or a new namespace |
| 429 | `embedding_rate_limited` | the provider is rate-limiting your key | wait `retry_after` seconds, retry |
| 402 | `payment_required` | the free plan's hosted tokens are used up | your own key, or pro |
| 502 | `embedding_failed` | the provider failed | retry: writes are by id |

## Cost and metering

- **Your own key:** the provider bills you for the tokens; SkimmerDB doesn't meter or charge
  for them. Writes and queries cost what they always do.
- **Hosted:** tokens are metered per tenant (`embed_tokens` in `GET /v1/usage`) and billed at
  $0.02 per million on pro, Voyage's own price for `voyage-3.5-lite`. The free plan includes 5
  million tokens a month (a few thousand pages of text); past them hosted embedding answers 402
  while your own vectors and keys keep working. See [Limits and plans](/reference/limits).