Knowledge Connectors

A knowledge connector pulls a bounded batch from one certified provider, normalizes it into upserts and tombstones, and applies that batch to the knowledge base in a single persist step. The cursor advances only after that persist succeeds.

The allowlisted providers are drive, confluence, slack, and github. The runtime registry rejects any other providerId (for example notion). The process default registry certifies those four after the knowledge-connector-contract/v1 fixture suite. An uncertified provider can be stored, but it is not advertised as knowledge.sync and cannot queue a run.

Providers and filters

Every connector config must include a workspace credentialId. Provider-specific filters are validated at fetch time.

Changing a filter set invalidates the stored cursor. Drive, Confluence, Slack, and GitHub each bind the cursor to a selectionKey derived from the current filter. A mismatch throws before fetch mutates documents.

Credentials

Sync resolves config.credentialId in the connector workspace with workspace scope. The run fails closed (no provider fetch) when:

• the credential is missing or has no secret
• the credential is not workspace-scoped or belongs to another workspace
• the credential type is not compatible with the provider (comparison is case-insensitive and strips non-alphanumeric characters)
• the stored secret is JSON and declares scope / scopes, and those grants omit a required provider scope

Drive treats https://www.googleapis.com/auth/drive as covering drive.readonly. If the secret is not JSON, or JSON without scopes, the scope check is skipped.

Providers may return updatedSecret (OAuth refresh). The service writes it with compare-and-set on that credential id. Concurrent connectors that share one credential serialize on workspaceId:credentialId so the second fetch sees the refreshed secret. A not_found update fails the run. A compare-and-set conflict does not fail the run.

Manual runs

Create a connector, then POST /api/v1/knowledge-connectors/{id}/runs with a required Idempotency-Key and an optional options object. The only accepted body key is options. Caller records, cursor, or defer fields return 400 validation_failed.

Default fetch budget (override only inside these bounds):

The same HTTP request queues the run and executes it. Agents can trigger the same path through the preview capability knowledge.sync (confirmation and idempotency required).

State and progress

Connector status is idle, syncing, succeeded, or failed. Run status is queued, running, succeeded, failed, or cancelled.

While a run holds the lease, the connector is syncing and lastError is cleared. Success sets connector succeeded, writes lastSyncedAt, stores the batch cursor, and records upsertedCount / tombstonedCount on the run. Failure sets connector failed and copies the error onto lastError and run.error.

Inspect progress with GET /api/v1/knowledge-connectors/{id} or GET /api/v1/knowledge-connectors/{id}/runs/{runId} (list: GET .../{id}/runs). Counts and cursors are the applied batch, not a live byte meter. A failed POST .../runs still persists run.status: failed and then returns an error envelope (not 201). Slack may persist a mid-channel cursor so the next run resumes. Drive, Confluence, and GitHub also persist opaque cursors after a successful apply.

Applied upserts become workspace documents keyed by (connectorId, externalId), linked to the connector's knowledge base, with indexingStatus reset to pending. Tombstones disable and trash matching documents. Built-in providers emit fullSnapshot: false; a true snapshot would also trash connector documents absent from that batch.

Cancellation, retries, and leases

POST /api/v1/knowledge-connectors/{id}/runs/{runId}/cancel (idempotency required) cancels a queued or running run. A second cancel, or cancel of a terminal run, returns 409. A cancelled run cannot be started.

Provider HTTP retries: up to three attempts, only for HTTP 429 / rate-limit errors. The delay uses retryAfter / retryAfterMs when present (capped at 30s), otherwise 1000 * 2^attempt ms. Abort cancels the wait.

Run-level retry: replay the same Idempotency-Key on POST .../runs with the same options. The stable run id is ksr_ plus the first 40 hex chars of sha256(workspaceId + NUL + connectorId + NUL + key). A failed or cancelled run with that key is re-queued (retryTerminal). A succeeded run is replayed without a second apply. The same key with different options is 409.

One connector holds one lease (default 5 minutes, heartbeat at one-third of that). A second in-flight execute on the same connector is 409 and leaves the queued run in place. An expired lease can be reclaimed; the stale running run is marked failed (Knowledge sync lease expired and was recovered by a newer run.). Losing the lease during fetch aborts the provider and fails the run.

Degradation

Authorization

All /api/v1/knowledge-connectors routes require a workspace API key. Missing/invalid token: 401. Personal or copilot key: 403 (A workspace API key is required.). Workspace id comes from the key, not from query workspaceId. A connector or knowledge base in another workspace is 404.

Next steps