---
title: Sharing knowledge
---

A collection is a namespace that only grants open. A grant can name a person signed in
to Loom, or an end user, group or agent of a service in front of Loom. A collection
inherits the grants of the collection it sits in. This page describes that model, why it
has this shape, and what it costs.

## The model

```mermaid
flowchart LR
    G["end-user token<br/>(signed per request)"] -->|"sub: user:u1<br/>agent: agent:a1"| R["request runs as user:u1"]
    R --> P["principals:<br/>user:u1, its groups, agent:a1, everyone"]
    P --> M["each principal's grants<br/>(cached grant maps)"]
    T["collection tree<br/>(loom.collections)"] --> D{"best role on the collection<br/>or an ancestor"}
    M --> D
    D -->|">= required"| A["admitted"]
    D -->|"otherwise"| X["refused"]
```

- **Collections.** A collection is a namespace with `kind = 'collection'` and a row in
  `loom.collections` naming its parent. `collection.create` makes one (the creator holds
  `admin`), `collection.move` puts it under another collection or detaches it,
  `collection.list` lists the ones the caller can read, each with the caller's role there
  and its title and description (the dashboard's Library shows them). With
  `include_document_counts=true`, each returned collection also carries its direct live
  document count, excluding child collections. Counts read the same current document
  metadata as `knowledge.list`, which can lag a commit awaiting publication. Without
  that option, `document_count` is null. Only namespaces authorized for the returned
  page are counted, in one grouped query.
  `collection.delete` takes one out of the directory and the tree (its children become
  top-level) once a purging erasure of it has taken what it holds off disk
  (`namespace.delete` refuses a collection). A
  deleted collection's name is never used again, and no grant reaches it any more. A
  document lives in one collection and follows its grants from the moment it is written;
  `knowledge.move` writes it into another, whose grants it follows from then on, and
  deletes it where it was ([moving documents](/docs/es/features/knowledge#moving-documents)).
- **Collections in the namespace lists.** `namespace.list_mine` gives a collection
  `kind = 'collection'` from its directory row; a `col-` name with no row takes
  `collection`, or `personal` for `col-me-…`, from its reserved prefix, so it still reads
  as a collection. Each `list_mine` row also carries the directory's `title`. A refusal
  on a collection says "cannot access the collection org/name", chosen by the prefix
  alone, so it reads the same whether or not the collection exists. The dashboard never
  shows a collection as a namespace: the Library lists them, the token form and Flow give
  them a Collections heading of their own, and admin Users tags each grant on one inline
  ("collection Research · writer"), each by its title, and another person's own collection
  as "personal collection".
- **A collection's name and its title.** The name is the address: grants, search scope,
  audit and erasure key on it, so it is unique in the org and never reused. The title is
  what people see; titles repeat. `collection.create` with `derive_name` takes a title
  and derives the name (`server/identity/domain/collection_names.py`): the title's slug
  (`col-q3-plans`), else the slug with a four-character suffix hashed from the request,
  so a retry walks the same names. For a derived create, within one parent a title names
  one collection (at the top level, one per creator): a create whose title one already
  has answers it with `created: false`, which makes it safe to retry and a folder upload
  safe to repeat. A create that names the collection keeps the title it is given, without
  that check. A title is what people read, not an identity: two titles can look alike
  (letters from different scripts do), and the address tells the collections apart; the
  dashboard shows it beside a title that looks like another. The title sits in
  `loom.namespaces.title`, beside the description: every writer of an existing directory
  row copies the row whole, so a build that does not know the column keeps it. A name
  already used answers `COLLECTION_NAME_TAKEN`, saying whether a deleted collection held
  it, with a free name to use instead.
- **Only grants open a collection**, and only one in the tree: a `col-` name that was
  deleted, or never created, admits no one but a platform admin. A collection is known by
  its name (`col-…`; the prefix is reserved: `namespace.register` and `namespace.restore`
  refuse it), so `check_authorized` hands it to `server/auth/sharing.py` before any other
  rule, with no read. A token's namespace claim, an org admin role and the code-namespace
  rules reach no collection; a scoped or delegated token reaches one no further than its
  own claim names it, and a delegated one no further than its creator's role there. A
  token with an `ns_write` claim writes no collection outside it (it holds `viewer` there
  at most), as on any namespace. A platform admin passes, for operator erasure and export.
- **Principals.** A request holds its subject, every group whose members include it, the
  agent its end-user token signs in, and `everyone`. A grant names any of them: a login
  handle or email, an `sa:` service account, `user:<id>`, `group:<id>`, `agent:<id>` or
  `everyone`. Groups, agents and `everyone` hold grants on collections only.
- **Inheritance.** A collection's role for a request is the best role any of its
  principals holds on the collection or on one of its ancestors, at most eight up. A child
  only adds to what its parents give. To make a child narrower, detach it: it then keeps
  its own grants only.
- **Personal collections.** Each end user has one, `col-me-<hash of org and user>`; a
  request names it `namespace: "@me"`, which only the server maps. Its name says whose it
  is: it admits that user (as admin) and no one else (an org admin's export of that user
  reads it, see below), takes no grant, sits in no tree, and needs no setup (its
  directory row appears with its first write). It holds the user's own documents (their
  files, a chat's attachments), chunked and embedded whatever their text holds, and the
  memories a chat writes for them. A message the user saves from a chat is not a
  document there; see [Saved messages](#saved-messages).

## Saved messages

A chat's saved message is a conversation turn flagged for its user:
`conversation.save_message` saves one, `conversation.unsave_message` withdraws it,
`conversation.list_saved_messages`, `conversation.saved_status` and
`conversation.search_saved_messages` read them
([Saving single messages](/docs/es/features/conversations-and-sessions#saving-single-messages)).
The same five ops serve both ways a chat runs in front of Loom:

- **A chat that records its conversations** in its own namespace writes the transcript
  with a namespace-scoped writer token and calls these ops with a token pinned to the
  user (`read_user_id`). Saving a turn the transcript holds flags it in place, so the
  message is stored and embedded once; the transcript row decides its owner, so two
  users never both save one turn, and compaction keeps a saved turn.
- **A chat that signs end-user tokens** calls them with `namespace: "@me"`. The other
  conversation ops refuse an end-user token (it is pinned to one user, and a thread has
  no user axis to bound), so no transcript lives in a personal collection: a save there
  stores the text the request sends as a saved-only row, and unsaving it removes the row.
  As under the pinned token above, a body naming another user is refused with 403.

A save that is not indexed answers `indexed: false` with a `reason` (listed on the
feature page). A client draws a conversation's saved state with one
`conversation.saved_status` call, and a user erasure takes the user's conversation rows,
saved ones included.

## Identity

A service in front of Loom (a chat app) signs a short-lived token for each request it
makes for one of its users, with a key registered for it as an end-user issuer
(`LOOM_JWT_END_USER_ISSUERS_JSON`: issuer → the orgs it may sign for, and its RS256 keys,
fetched from its `jwks_url` or listed by kid). The token names the user (`sub:
user:<id>`), one of those orgs, its own id (`jti`), and, when the request runs inside one
of its agents, that agent (`agent: agent:<id>`). `server.auth.jwt` verifies it into the
request's identity: subject `user:<id>`, the `viewer` role of an org member with no claim,
and reads pinned to that user's rows (`read_scope = own_user`). There is no long-lived
secret per customer, and no header can name a different user.

| An end-user token may name              | It may not                                                   |
| --------------------------------------- | ------------------------------------------------------------ |
| one user and one agent, in a listed org | name an org its issuer does not list, or none                |
|                                         | carry namespaces, roles or a read scope                      |
|                                         | lack a `jti`                                                 |
|                                         | live longer than `LOOM_JWT_END_USER_MAX_TTL_SECONDS` (5 min) |
|                                         | be signed by a key of another issuer                         |

An end-user key verifies only its own issuer's tokens, and only as an end user: it can
never sign a Loom-shaped token (another `sub`, roles, Loom's own issuer). The agent is a
signed claim because an agent's grants widen what the request reaches; the service
decides who may use which agent before it signs one in, so Loom keeps no copy of that.

An issuer's `orgs` are exact names or `prefix*` patterns, never `*`: one issuer serves the
orgs of its own deployment, and a stolen key reaches no other. A `jwks_url` is fetched by
every process at boot and every `LOOM_JWT_END_USER_JWKS_REFRESH_SECONDS` (5 min); a failed
fetch keeps the last keys, and until the first fetch lands the issuer's tokens are
answered 503, never admitted, and its fetch is retried every 2 s. A key is never both an
end user's and a federated one (`LOOM_JWT_PUBLIC_KEYS_JSON`), compared by key, not kid: a
federated key verifies Loom-shaped tokens. An issuer rotates by publishing the new key one
refresh before it signs with it.

Ids are lowercase (`server/core/principals.py`), so two spellings never name one user; a
SCIM member is the service's own lowercase user id, not an identity provider's. Of the
org-level ops (those that name no namespace to gate), an end user may call only
`collection.create`, `collection.list`, `namespace.list_mine`, `grants.pending_for_me` and
`mentor.guide` (the static agent guide).
An end-user token is served by the op endpoint and `knowledge.download`; MCP, `/v1/me/*`,
the admin routes and the task watch refuse it (403). A trusted caller
(`LOOM_TRUSTED_CALLERS`) acts for Loom subjects only and cannot assert a subject spelled
like a grant principal (`user:`, `group:`, `agent:`, `everyone`, in any case, bare or
after a `/`). One user is cut off by revoking their tokens (`loom revoke-token --subject
<org>/user:<id>`, the org named: `user:<id>` names one person per org, so a bare id
revokes no end user's token) and grants, or by erasing them. `/oauth/revoke` revokes
nothing for an end-user token: its `jti` is its issuer's to choose, and the revocation
list spans the deployment, so revoking it there could name another org's token or subject.

**Everyone** is a built-in principal every request acting in the org holds: a grant to
`everyone` on a collection opens it to the whole org (a chat app's "public"), with no
group to keep in step with the org's users. No token is ever `everyone`.

**An agent's files** go in a collection of its own (say `col-agent-<id>`) granted to
`agent:<id>`: whoever the service signs in with that agent reads them. An agent's search
and listing name its collections explicitly (`knowledge.search` and `knowledge.list` with
`namespaces`), each checked as a read of its own, so it never reaches another agent's
files; a chat's attachments go in the user's `@me` with the conversation as their
`session_id`.

**Groups** come from one place: SCIM `/scim/v2/Groups`, written with a token of the org's
admin. The group lands in that org; a member is a user id, held as `user:<id>`.
Membership is read by Loom on every request, never claimed by a token, so removing a
member takes effect within the consistency window below.

## Consistency

Every change to a grant, a group or the tree writes a row to `loom.acl_versions` for its
org and kind (`grants`, `groups`, `collections`; `grant_cache.changed`) before it returns.
Each process polls the watermark every `LOOM_ACL_WATERMARK_POLL_SECONDS` (1 s), one
sequential statement for every change count (`sum(changes)` per org and kind; a sum only
grows, whatever merges or clocks do), and drops what that kind of change makes stale when
its count moves: the org's cached grant maps, its groups, or its tree. Grant churn in a
tenant therefore re-reads only grant maps, one statement per subject that asks, never the
tree or the group table. The poll reads at most three rows per org once merged: 3,000 rows
in 2 ms at 1,000 orgs and 30,000 rows in 10 ms at 10,000 orgs (p50 of 9), per process per
interval. So a revoke, a removed member, a move or a detach is enforced in every process,
on that server or another, within about one poll interval, and at once in the process that
made it.

A process whose last poll is older than `LOOM_ACL_WATERMARK_MAX_STALE_SECONDS` (10 s)
serves no cached grant and reads per request; a failed read then falls back only to an
entry read at the last watermark it saw. The 10 s grant-cache TTL stays as a backstop
for a watermark row that was never written, and for grant changes made by a build that
predates the table during a rolling upgrade.

A collection's name, not a lookup, decides that it is one, so a namespace that already
used the `col-` prefix admits only a platform admin after the upgrade; the server logs the
active ones at boot.

## Read cost

`namespace` follows `org` in every knowledge sort key, so a search over the collections
a request may read is one statement per leg with `namespace IN {readable}`, pruned on the
primary key. Rows read are bounded by the readable rows plus one granule per readable
collection, whatever the tenant's size.

Measured on ClickHouse 26.9 with `loom.chunks`' own DDL (3,072-dimension vectors), 12
chunks per collection, the readable set scattered through the sort order (p50 of 5):

| Readable collections | Tenant of 12,000 collections: rows read | p50   | Tenant of 48,000 collections: rows read | p50    |
| -------------------- | --------------------------------------- | ----- | --------------------------------------- | ------ |
| 1                    | 3,352                                   | 11 ms | 3,352                                   | 11 ms  |
| 10                   | 16,948                                  | 20 ms | 16,948                                  | 20 ms  |
| 1,000                | 145,676 (all)                           | 38 ms | 442,877                                 | 54 ms  |
| 10,000               | 145,676 (all)                           | 78 ms | 577,676                                 | 110 ms |

`LOOM_KNOWLEDGE_SEARCH_MAX_NAMESPACES` (10,000) caps one search; the rest come back in
`namespaces_skipped`. Smaller granules read fewer rows for a handful of collections but
were 6–8 times slower at 1,000, so the default granule stays.

A lexical or hybrid search first reads the BM25 statistics (chunk count and mean length) of
the namespaces its process has not cached yet, in one statement; each process keeps them
for a few minutes. Cold, on a tenant of 10,000 collections and 320,000 chunks (p50 of 5):

| Readable collections | Rows read | Read    | Memory  | p50   |
| -------------------- | --------- | ------- | ------- | ----- |
| 1                    | 1,675     | 0.1 MB  | 5.8 MB  | 5 ms  |
| 10                   | 16,731    | 0.7 MB  | 5.6 MB  | 8 ms  |
| 1,000                | 319,866   | 19.1 MB | 26.8 MB | 12 ms |
| 10,000               | 320,000   | 19.1 MB | 41.9 MB | 35 ms |

A watermark move re-reads only what its kind made stale, in each process, once per value
and on its next use. On the 48,000-collection tenant (5,000 users, 200 groups of 25,
84,000 grant rows; p50 of 7):

| Change   | Re-read                                        | Rows read | p50   |
| -------- | ---------------------------------------------- | --------- | ----- |
| a grant  | one subject's grant map, per subject that asks | 84,000    | 12 ms |
| a group  | the org's groups                               | 200       | 2 ms  |
| the tree | the org's tree                                 | 48,000    | 8 ms  |

A grant map read scans the org's grants: `loom.namespace_grants` is ordered by namespace,
not by subject, and a skip index on the subject prunes nothing at this granule size.

## Erasure

Every document and chunk carries who wrote it and what it is: the agent, the end user,
the session it was written in (a chat, an agent run), the token's principal and the
content's class. Access never reads them; a write declares the agent, the session and a
downgraded class (`derived`, `external`, `imported`), the end user comes from the token,
and `knowledge.search` returns all of them on each hit and filters on the session and the
class. A rewrite of the same text keeps them; new text, or a new original, takes the
writer's.

A user erasure (`scope_type = user_id`) supersedes the documents and chunks that user
wrote in any namespace of the org (an end-user token, or a token minted with
`--read-user-id`, stamps its `user_id`), or, with `purge`, deletes every
version of them from disk, after superseding another writer's versions that one of theirs
replaced, so none reads as live again. It deletes their uploaded originals that no other
writer's live document shares (each upload by an end user is recorded in
`loom.blob_uploads`, because the document version that named it may have been replaced and
merged away since), revokes and then deletes every grant naming `user:<id>`, blanks their
id as the issuer of the grants they gave others (those grants stay), removes them from
every group, blanks their id as the creator of the collections they made (the collections
stay), and retires their personal collection's directory row (`namespace.delete` and
`collection.delete` refuse a personal collection: it goes only with its user's erasure).
The request ledger keeps the user's calls for cost with their `user:<id>` subject
scrubbed, and the user's newest-call row is deleted. Deleting a SCIM group revokes the
grants that named it, so a group made later under the same id starts with none.
Collections that hold other writers' documents stay: Loom deletes only what its writer
asked to delete. `collection.delete` purges what its collection held (a namespace erasure
that takes the rows off disk) before the collection leaves the tree. A user-scope
`erasure.export` returns the chunks and the conversation rows (transcript turns, saved
messages and threads) that user owns in the collections its caller may read, and in the
user's own personal collection. An org admin may erase a collection's rows without
reading them; a platform admin sees every collection. Each section's read leaves out what
the caller may not see before its limit, so a hidden row never takes a visible row's place
on the page.

## Namespace guides

A shared namespace is usable only when a newcomer knows its conventions: which key
prefixes mean what, what a tag marks, which rows to ignore. The rows do not say. Each
namespace therefore has a guide (`namespace.guide_*`, [namespaces and
sharing](/docs/es/features/namespaces-and-sharing#a-namespaces-guide)): Markdown text and a
one-line summary that its writers keep current and every reader is shown.

`loom.namespace_guides` holds one row per revision, ordered by `(org, namespace,
revision)`; the highest revision is the guide, and the older ones are its history. A
save is optimistic: it names the revision it was read at, and under a deployment-wide
lease on the namespace's guide (`server.core.leases.holding`) the server compares that
with the current revision and inserts the next one only on a match. Without the lease,
two replicas inside ClickHouse's visibility lag could both find revision N current and
both write N+1. A mismatch writes nothing and returns the current guide: the agent that
lost merges its edit into that text, which it does better than a text merge on the
server would.

A save's audit row records the revision it was made from and the text's length and
SHA-256, never the text, summary or note (`Operation.audit_subject`): `loom.audit_log`
outlives an erasure. The directory's search matches a guide's summary, the one part the
org sees; matching the text would let someone who may not read the guide rebuild it a
character at a time. An end user's personal collection (`col-me-…`) takes no guide,
since only their erasure removes it and that scope does not reach the table.

A guide is the namespace's, not its writer's: it describes how the namespace's data is
laid out, as a category description or a profile schema does, and carries no `user_id`.
So a namespace erasure deletes it, and a user erasure, which follows the `user_id` of
memory rows, leaves it, with the writer's actor on the revisions they wrote. The guide
is not a place for facts about anyone: the docs and the skill send those to memory,
where the user's erasure reaches them.

The guide is shown where an agent already looks: the session-start pack, the
`namespace.browse` directory (a public row's guide summary to everyone, a private code
row's, which is listed to people who may not read it, to its members only, and a
restricted row's to no one) and `namespace.get`.
One text serves agents and people, so there is no second copy to drift.

### Alternatives not taken for guides

- **A memory row at a reserved key.** The conflict resolver, extraction, dedup and
  search would all treat a README as a fact, and a memory write has no compare-and-set.
- **Separate guides for agents and people.** Both need the same facts, and two texts
  drift apart.
- **Sections with a revision each.** Two agents editing different sections at once
  would both land, at the cost of a section API; a whole-text revision keeps the API to
  one text, and a conflict costs one merge and one retry.
- **Merging concurrent edits on the server.** A three-way merge of prose either
  conflicts on most real edits or silently produces text neither writer wrote.

## Alternatives not taken

- **An owner column on documents.** Every read path would add a row predicate, one miss
  would leak, and the column would not prune (it is not in the sort key), so a user's
  search would still read the whole tenant.
- **A personal-documents table with `user_id` in the sort key.** It fixes the cost of
  a user's own documents by building a second document store: chunking, lexical
  statistics, embedding retries and erasure twice, and documents that can never join a
  collection.
- **Saved messages as documents in the personal collection**
  (`chat/<conversation>/<message>` through `knowledge.upsert`). It needs no op of its
  own, but it serves only a chat that signs end-user tokens, so a chat that records its
  conversations would need the conversation ops anyway: two ways to save one message. A
  document gets no role or sensitivity check, and a client finds a conversation's saved
  state by listing a URI prefix. A saved message is a conversation row in both setups.
- **One namespace per tenant with row-level ACLs.** Every read filters rows one by one and
  prunes nothing, and a move or a revoke rewrites every affected chunk row.
- **An ACL key in the sort order.** A request's readable set is a union over its
  principals and differs per user; no one sort order clusters every user's union.
- **Group membership in a token claim.** A claim is fixed until the token expires, so a
  removed member keeps access for the token's life.
- **Inheritance resolved at write time.** Moving a collection would rewrite the grants of
  every descendant; resolving at check time from a cached tree costs a walk of at most
  eight parents.

## Related

- [Security](/docs/es/architecture/security): tokens, the gate, restricted namespaces.
- [Identities and namespaces](/docs/es/concepts/identities-and-namespaces).
- [Tokens and grants](/docs/es/guides/tokens-and-grants).
