Reference architecture
This architecture is intentionally small. It describes the minimum separation of concerns for a system that accepts a user request, may invoke generative inference, and may cause a consequential side effect.
The responsibilities
- Edge gateway
- Authenticate, validate, set request identity, and apply cheap admission controls. It should not conceal a multi-minute job behind a single response.
- Identity and policy
- Resolve acting user, tenant, scopes, data policy, and spend authority. This is independent from model selection.
- Job record
- Persist the request’s business identity, state, idempotency key, reservation, and result references. It is the source of truth for work.
- Queue or workflow
- Defer and retry work without treating redelivery as a new business event. Use a workflow when the workflow itself is the durable unit.
- Coordinator
- Serialize transitions that must not race. A Durable Object is often suitable for a job, session, or scarce resource coordinator.
- Tool boundary
- Expose narrowly scoped operations, validate each invocation, and preserve the user and tenant context.
- Inference boundary
- Select models by capability, policy, price, latency, and evaluation evidence. The application still owns semantic routing.
- Artifacts and evidence
- Store large outputs outside relational state; record references, hashes, retention class, and provenance.
Minimum viable invariants
Before launch, ensure that:
- a retry cannot create a second charge or duplicate external side effect;
- every tool operation has an authenticated principal and scope;
- every terminal job is either settled or explicitly marked for reconciliation;
- a provider error never becomes the user-facing source of truth;
- a model, tool, or retrieval result can be traced to its request and policy;
- deletion and retention rules cover prompts, artifacts, logs, and embeddings.
Diagram sources
Public assets live under /public/field-guide. Each figure has Mermaid source and an exported SVG so the public site remains static and diagrams remain editable.
