76 specialized LLM agents compiled into a seven-wave DAG with hard barriers between waves.
Architecture walkthrough available on request
Litigation teams spend enormous associate hours on work product that's high-volume but structurally repetitive. The bottleneck isn't legal judgment — it's the mechanical work between judgments.
The pipeline runs as a seven-wave DAG rather than freely-communicating agents. Steps inside a wave execute concurrently; wave N starts only once every step in N−1 has finished, by success, skip, or failure. That barrier is what keeps 76 agents consistent — no agent ever reads half-built upstream state, and a failure is attributable to a step rather than to an emergent interaction.
Single-step waves at the start gate a 25-step fan-out in the middle. Those bottlenecks are the highest-leverage failures in the system: everything downstream inherits their output. It also means the widest wave’s concurrency is in-process — asyncio.gather inside one container — so throughput scales with provider concurrency limits rather than with infrastructure.
46 of the 76 agents run a bounded loop. A plan node has the model author its own queries and decide whether to search at all; a validate node then judges whether coverage is sufficient and can re-enter retrieval with new queries, capped so it terminates. The other 30 never touch the vector store — their evidence is upstream analysis output, which is correct for them.
Retrieval shipped first on Pinecone — managed index, semaphore-bounded async calls, and a per-vector metadata ceiling to work around. It now runs on pgvector co-located in the primary Postgres, with Bedrock Titan Embed V2 pinned to 1024 dimensions so the index never has to migrate. Co-location is the payoff: an agent joins semantic hits against relational facts without crossing a service boundary or reconciling two consistency models. The migration preserved the service interface so 32+ call sites were untouched — at the cost of a class still named after the thing it no longer is.
Prompt assembly sorts cache-stable sections first and attaches cache-control markers; variable sections follow uncached. Shared case context is prefetched and memoized per session rather than re-read per agent, and cooperative cancellation checks between retries and streaming events give sub-second abort. With dozens of agents in a chain sharing one case dossier, that ordering is what makes a full run affordable: roughly 60% less token spend than the first working version.