The Agent Factory: One Repo, Many Agents, No Meltdowns

Over the last couple of projects I’ve settled on a setup for running multiple AI agents against the same codebase at the same time. It proved solid enough that I want to write it down. I call it the agent factory.

This setup reflects my workflow as a nvim and terminal dweller. It wasn’t designed upfront — it evolved, project by project, into whatever fit the way I actually work: everything is a file I can grep, every moving part is a command I can run and inspect from a plain shell, and nothing forces me out of the terminal. No dashboards, no extra services to babysit — just containers, git, and a handful of scripts. If that’s how you work too, you’ll feel at home. And if it isn’t, the ideas still travel; the terminal just happens to be where mine live.

The recipe: container per agent, worktree per task — the host supplies the project, the shared state, and the skills; secrets ride in as env vars.

Let me unpack that, because each part earns its place.

The container

Each agent gets its own container — Podman in my case; any OCI runtime will do — and inside it runs one harness, whatever agent loop you use, pointed at exactly one task. The container is disposable. Everything that matters lives outside of it:

 HOST
 ------------------------------------------------------------------------------
 my-project/  (pwd)   .state/       skills/     scratch     secret source
 code, worktrees, .env  shared state  capabilities  host /tmp  (.env or curated)
          |                 |              |             |                 |
          | mounted         | mounted      | mounted     | mounted         | env vars
          | -> /project     | -> .state    | -> /skills  | -> /tmp         | -> process env
          v                 v              v             v                 v
 +------------------------------------------------------------------------------+
 |                           container "harness-a"                              |
 |                                                                              |
 |   +--------------------------------------------------------------------+     |
 |   |                     agent harness  (the loop)                      |     |
 |   |                                                                    |     |
 |   |   work:     /project/.worktrees/task-a   <- its worktree           |     |
 |   |   memory:   shared state in .state     <- survives restarts        |     |
 |   |   skills:   /skills                      <- from the host          |     |
 |   |   scratch:   /tmp                        <- exists to be wiped     |     |
 |   |   credentials: env only                 <- never written to disk   |     |
 |   +--------------------------------------------------------------------+     |
 |                                                                              |
 +------------------------------------------------------------------------------+

Three rules:

  • The mounts bring in everything: pwd — code, worktrees, .env — plus .state, the shared harness state, the host-living skills, and /tmp scratch.
  • The worktree convention does the isolating: whatever an agent works on, it works on in its own worktree — never the main checkout.
  • Credentials come in through the environment only. Nothing secret is ever a file the agent can commit.

Concretely, one agent is one podman run:

# one agent, one container, one task
podman run --rm -it \
  -v "$PWD:/project" \
  -v "$PWD/.state:/project/.state" \
  -v "$HOME/skills:/skills" \
  -v "/tmp/agent-factory/$TASK:/tmp" \
  --env-file .env \
  my-harness:latest

pwd and .state are the durable mounts, ~/skills gives every agent the same capabilities, /tmp is scratch, and secrets ride in through the environment. The harness’s first move inside is always the same: cd /project/.worktrees/task-a.

How secrets get in — the gitignored .env, or curated env vars, OPSEC-style — is a topic of its own; it lives in the appendix.

Mounts: everything lives on the host

Each container gets four volume mounts, and together they are all the persistence there is:

  • pwd — the project directory. Code, the .worktrees/ folder, the gitignored .env: everything the agent works on.
  • .state — the harness state. The directory every agent shares: what they’re doing, what they’ve learned, what they’re blocked on.
  • skills — capabilities mounted in. A directory of host-living skills, mounted into every container: update a skill once on the host, and every agent picks it up on its next start. No image rebuilds, no copying skills between containers — that’s the whole integration story.
  • scratch — throwaway space. A directory under the host’s /tmp, mounted at /tmp in the container: build junk, experiment output, everything that shouldn’t outlive the task.

The project, state, and skills all live on the host and survive the container; scratch is the opposite — it exists to be wiped. No separate metadata volumes, no per-container volume sets, no volume lifecycle to babysit.

 every container mounts the same project -- and the same .state

    container A           container B           container C
   +------------+       +------------+       +------------+
   |  harness A |       |  harness B |       |  harness C |
   +------+-----+       +------+-----+       +------+-----+
          |                    |                    |
          v                    v                    v
 +----------------------------------------------------------------------+
 |  my-project/  (pwd)                                                  |
 |  one directory -- the only durable thing                             |
 |                                                                      |
 |    src/  docs/ ...         <- main checkout, stays clean             |
 |    .worktrees/             <- gitignored, one worktree per task      |
 |      task-a/               <- harness A works here                   |
 |      task-b/               <- harness B works here                   |
 |      task-c/               <- harness C works here                   |
 |    .state/                 <- mounted, shared harness state          |
 |    .env                    <- gitignored, tier-1 secrets             |
 +----------------------------------------------------------------------+

 (skills and scratch live outside the project, on the host: ~/skills -> /skills, /tmp/agent-factory/<task> -> /tmp)

Those mounts do several jobs at once:

  1. Persistence for free. The project directory lives on the host; the container is disposable. Kill it, start a new one, mount the same things again — the harness finds the project and the shared state exactly where it left them.
  2. State you can actually work with. Files and directories can be grepped, diffed, backed up together with the project, and deleted when the task is done. A Podman volume is an opaque blob by comparison.
  3. One .gitignore as the only access list. Everything that isn’t source is gitignored: .worktrees/, .state/, .env. The ignore rules draw the line between project, machine state, and secrets.
  4. Skills integrate for free. Skills live on the host and are mounted into every container: add or update one, and the next container start picks it up — no image rebuild, no per-agent copying.

Why the scratch mount matters more than it looks: without it, throwaway files end up in one of two wrong places. In the project directory, where they pollute git status, confuse every agent in a worktree, and occasionally get committed by one that felt helpful. Or inside the container’s own filesystem, where they die with the container and fill up the overlay layer while it runs. A bind mount from the host’s /tmp fixes both: you can watch what an agent is scribbling from a normal shell, wipe it with rm -rf without touching anything durable, and the project directory stays exactly as clean as git status claims.

And notice where isolation comes from — not from volume topology, but from the convention: whatever you do, work on a git worktree. Each harness lives in its own worktree and leaves the main checkout alone. What they share is exactly what’s mounted to be shared: .state. Code travels through git, runtime state through that one directory.

That’s the whole persistence story: project, state, and skills directories that already exist on the host, plus a scratch space that’s supposed to be disposable — mounted into containers that don’t need to survive.

The catch: all of them at the same time

Here’s the problem this whole setup solves. Multiple agents, one project, running at the same time. Two harnesses editing the same working directory is chaos — uncommitted files get stomped, branches get tangled, and you spend your evening doing archaeology on who broke what.

The answer: git worktrees.

A worktree is a full extra checkout of your repo at a different path, on its own branch, sharing the same object database. Every harness gets one. They live in a .worktrees/ folder inside the project — gitignored, because they’re disposable machine state, not source.

One command per agent:

git worktree add .worktrees/task-a -b task-a

And once the MR is merged, one command retires it:

git worktree remove .worktrees/task-a && git branch -d task-a
                      my-project/                    <- main checkout, stays clean
                      /        \
                     /          \
              .worktrees/         src/  docs/ ...   <- .worktrees is gitignored
              /   |   |   \
             /    |    |    \
        task-a  task-b  task-c  task-d               <- one git worktree per task,
           |      |      |      |                       own branch, own files
       +------+ +------+ +------+ +------+
       |  A   | |  B   | |  C   | |  D   |              <- one container per worktree,
       +------+ +------+ +------+ +------+                 one agent harness inside

Now “at the same time” is boring in the best way: each agent commits to its own branch in its own directory, and the main checkout stays pristine. Merging becomes an explicit, ordered decision — not an accident.

Gotchas I hit so you don’t have to

  1. .env does not exist in new worktrees. Gitignored files live only in the checkout that created them. Your harness bootstrap must copy or symlink secrets (and dependencies) into each worktree, or agents will fail mysteriously at startup.
  2. Dead worktrees accumulate. If a container dies without teardown, stale worktrees and branches pile up. Run git worktree prune as part of your cleanup step.
  3. Dependencies don’t carry over either. A fresh worktree has no node_modules, no venv. For Python, uv has mostly retired this gotcha for me — creating and tearing down a venv per worktree is so cheap you stop thinking about it:
cd .worktrees/task-a && uv venv && uv sync

For anything slower to install, keep a dependency cache in the project root, gitignored like everything else that isn’t source.
4. CI becomes the traffic jam. Every agent that finishes opens an MR, and every MR triggers a pipeline. In a resource-limited CI environment, a handful of agents working concurrently can saturate the queue: pipelines back up, agents sit waiting for their own verification feedback, and iteration speed collapses right when you thought you were scaling. Throttle accordingly — fewer concurrent agents than you have pipeline slots, and let the harnesses run the fast checks themselves (lint, unit tests in that uv venv) before anything reaches CI.

The honest part: what’s not nice

The setup itself? Great. You can really hog away — spin up four agents, each grinding on its own task, and get an unreasonable amount of work done per hour of your attention.

What’s actually exhausting is the other side: managing several harnesses that need feedback or revision. Every agent eventually gets stuck, does something questionable, or finishes and wants review. Multiply by four and you’re context-switching all day. That’s not a tooling problem, it’s a work-in-progress problem. Things that helped:

  • Cap concurrent harnesses at two or three. Your review bandwidth is the constraint, not compute — and in a constrained CI environment, so is the pipeline queue (gotcha #4). More agents just means a longer queue waiting for you.
  • Batch your feedback. Don’t react to each harness the moment it finishes. Give every agent one place to post when it’s blocked, and review everything in one sitting. The context-switching is what drains you, not the reviewing.
  • Force a structured handoff. Every session ends with a pull request plus a short report: what it did, what it’s blocked on, what it needs from you. You review artifacts, not live agent state.
  • Merge serially, rebase the rest. Merge one worktree’s PR, then have the others rebase onto main before continuing — otherwise you resolve the same integration conflicts N times.

The next iteration, when I get to it, is a thin supervisor layer: a queue plus a triage inbox, so the factory runs itself between my review sessions instead of every harness polling me directly.

The takeaway

The agent factory isn’t really about containers or worktrees — those are plumbing. The actual insight is this: agents scale cheaply, review doesn’t. Isolate the work so the agents can run in parallel, keep the state in plain files so nothing gets lost, keep the secrets in the environment so nothing leaks — and then treat your own attention as the scarce resource it is.

The factory scales just fine. The bottleneck is you. Batch accordingly.

Appendix: secrets, two levels of OPSEC

The hook said secrets ride in as env vars — here’s the full treatment.

OPSEC — operational security — is the military discipline of making sure sensitive information reaches only those who need it. I run secrets at one of two levels:

 TIER 1: .env file                       TIER 2: curated env vars
 "comfortable"                           "need-to-know"

 .env  (gitignored, never committed)     secret store / password manager
      |                                       |
      |  plain KEY=value lines                |  hand-pick exactly the keys
      |  copied into each worktree            |  this harness needs
      |  by the bootstrap step                |  injected at container start
      v                                       v
 +----------------+                      +----------------+
 |    harness     |                      |    harness     |
 +----------------+                      +----------------+
 sees every key in the file               sees only what it was given

Tier 1 is a .env file that is gitignored, full stop. Tier 2 is need-to-know: no secret file reaches the container at all — instead you curate the specific environment variables each container needs and pass them at startup. The harness working on docs doesn’t get the production database password. The harness working on the database migration doesn’t get your API tokens for everything else.

In podman terms, the two tiers are one flag apart:

# tier 1 -- comfortable: the gitignored file
podman run --env-file .env ...

# tier 2 -- need-to-know: only what this container needs
podman run -e MISTRAL_API_KEY -e GITHUB_TOKEN ...

With -e VAR and no value, the variable is passed through from your shell — the secret never has to exist in a file at all. That’s the whole point of tier 2.

One caveat I owe you, because it’s the sharp edge of this design: the shared state is where secret isolation goes to die. Curating env vars per container isolates secrets at injection time — but .state is shared by every agent, and a harness that logs its environment, dumps a transcript, or caches an API response into the state directory has just punched a hole through that isolation. Everything one agent stores there, every other agent can read. So the one OPSEC rule that actually keeps secrets isolated: secrets live in env vars and never, ever enter state. If you don’t trust the harness on that discipline, treat every container as sharing one secret pool — because functionally, it does.