Configure embeddings

Embeddings are local-first: the default provider is Ollama, a local model server — no API key, no per-token cost. The design behind the configuration — including the “return null, never block a write” degradation contract — is explained in Embeddings.

Quick start (Ollama, the default)

EMBEDDING_PROVIDER defaults to ollama, so setup is just getting an Ollama server running with the default model pulled:

# Option A — host install (https://ollama.com/download)
ollama pull nomic-embed-text

# Option B — opt-in Docker Compose profile
docker compose --profile ollama up -d
docker compose exec ollama ollama pull nomic-embed-text

Done. The server calls Ollama’s native /api/embed endpoint (plain HTTP, no SDK) at OLLAMA_URL (default http://localhost:11434) with nomic-embed-text (768 dims).

Pick a provider

.env

EMBEDDING_PROVIDER=ollama     # default; local Ollama server, no API key
# EMBEDDING_PROVIDER=openai   # opt-in; needs OPENAI_API_KEY
# EMBEDDING_PROVIDER=local    # deterministic hash vectors, no network
# EMBEDDING_PROVIDER=disabled

Provider	Needs	Recall behavior
ollama (default)	An Ollama server at OLLAMA_URL with the model pulled	Full semantic recall, fully local (nomic-embed-text, 768 dims, by default)
openai	OPENAI_API_KEY	Full semantic recall via the OpenAI API (text-embedding-3-small, 1536 dims, by default)
local	nothing	Deterministic vectors — stable similarity for tests/CI/offline dev, not real semantics
disabled	nothing	Writes proceed vector-less; semantic recall returns no vector hits

Notes:

• Any provider failure — Ollama unreachable, model not pulled, request timeout (30 s) — logs a warning and returns null; the memory workflow continues without a vector. Writes never fail because of embeddings. A 404 from Ollama logs a hint to run ollama pull <model>.
• With openai and no OPENAI_API_KEY, embedding generation silently disables itself — same runtime behavior as disabled.
• local stays useful for CI and cheap bulk imports: ingest with deterministic vectors, then upgrade to real embeddings with a regenerating reindex (see the agent-memory migration runbook). With the free Ollama default, importing with real embeddings directly costs nothing either.

Model, endpoint, and dimensions

Variable	Default	Purpose
EMBEDDING_MODEL	per provider: nomic-embed-text (ollama), text-embedding-3-small (openai)	Embedding model id — any id is accepted
OLLAMA_URL	http://localhost:11434	Ollama server base URL (native /api/embed)
VECTOR_DIMENSIONS	unset	Optional strict pin — see below

Empty-string values are read as unset, so compose-style ${VAR:-} defaults are safe.

Known models and their dimensions:

Model	Provider	Dimensions
nomic-embed-text	ollama	768
mxbai-embed-large	ollama	1024
all-minilm	ollama	384
bge-m3	ollama	1024
text-embedding-3-small	openai	1536
text-embedding-3-large	openai	3072
local-hash	local	1536

Arbitrary model ids are allowed — for a model not in this table, dimensions are taken from the actual vector the provider returns.

VECTOR_DIMENSIONS no longer defaults to 1536; it is an optional strict pin. When unset, both vector backends infer dimensionality from the first upserted vector. Set it when you want the server to fail fast if the pipeline ever produces vectors of a different size — the backends verify and throw an actionable error on mismatch.

Opt in to OpenAI

OpenAI stays fully supported:

.env

EMBEDDING_PROVIDER=openai
OPENAI_API_KEY=sk-...
# EMBEDDING_MODEL=text-embedding-3-small   # openai default; -3-large = 3072 dims

Switching an existing deployment between Ollama (768 dims) and OpenAI (1536 dims) is a dimension change — follow the model-change procedure below.

Change the model or provider (dimension change)

Switching to a model with a different dimensionality (e.g. from the 768-dim nomic-embed-text default to the 1536-dim text-embedding-3-small) invalidates every stored vector — the index shape changes. The canonical sequence:

1. Set the new provider/model env (EMBEDDING_PROVIDER, EMBEDDING_MODEL, and — if you pin — the new VECTOR_DIMENSIONS).

2. Restart the server.

3. Run an unscoped reindex that both recreates the index and regenerates embeddings:

   pnpm --filter mcp-server reindex -- --recreate --regenerate

   or via the admin MCP tool:

   {
     "name": "reindex_memories",
     "arguments": {
       "adminToken": "<MCP_ADMIN_TOKEN>",
       "recreate": true,
       "reuseExistingEmbeddings": false
     }
   }

This drops the old-dimension index, regenerates every embedding at the new dimensionality, and writes the new vectors back to Postgres. Recall is empty while the rebuild runs. recreate is only honored by an unscoped full pass — no userId, cursor, or memory cap.

Budget note: regeneration calls the provider once per memory — free with the local Ollama default, token cost with openai. For a large corpus, prefer the queued variant (queue_reindex_memories, same arguments) so the pass runs in the background — see Reindex embeddings.

GPU

On a host install, Ollama uses a GPU automatically when one is available. For the Docker Compose ollama service, GPU passthrough needs the standard Docker GPU configuration (e.g. the NVIDIA container toolkit plus a gpus reservation). CPU-only is fine for nomic-embed-text.

Repairing individual memories

If a memory’s content was edited while the provider was down, its vector is stale (embeddingStale). Repair a single memory without a full pass with the reembed_memory tool — or leave it to the next regenerating reindex.

Verify

• Store a memory, then recall it with a paraphrase — a semantic (not keyword) match confirms embeddings are live.
• If recall only ever matches on exact keywords, check the server logs: an unreachable provider or unpulled model logs a warning and returns null, so writes still succeed but land without a vector.