rch — Remote Compilation Helper

curl -fsSL "https://raw.githubusercontent.com/Dicklesworthstone/remote_compilation_helper/main/install.sh?$(date +%s)" | bash -s -- --easy-mode

TL;DR

Problem: Many concurrent AI agents can saturate local CPU and make your workstation unusable.

Solution: RCH runs as a Claude Code PreToolUse hook, classifies build-like commands in milliseconds, executes them on remote workers, and returns artifacts/output as if they ran locally.

Design constraint: RCH is fail-open. If remote execution is not safe/possible, commands run locally.

What RCH Intercepts

RCH currently recognizes and can offload:

Nix builds only route to workers that advertise a nix capability (a usable nix binary plus a populated /nix/store); on a fleet with no such worker they fall back to local execution (or are refused under RCH_REQUIRE_REMOTE=1, exactly as with Bun/Node). Nix outputs stay in the worker's /nix/store behind a result symlink, so these run as streaming, exit-status-only commands (no artifacts are copied back — the flake source is synced out, the build runs, the result stays remote).

Go builds support an explicit, project-relative file output, for example go build -o bin/app ./cmd/app or go build -o 'products/app [dev]*?' main.go. The output is returned to that exact path, including when CARGO_TARGET_DIR is set. Common build flags such as -p, -tags, -trimpath, and -race are supported, as are linker stripping flags (-ldflags '-s -w') and -X variable definitions. Builds require the durable Unix execution path and a local Go installation. Both endpoints must report empty effective GOFLAGS and the dispatcher's native GOOS/GOARCH. The caller's selected Go version, CGO configuration, experiments, and architecture tuning settings are captured before upload and must match the worker exactly before compilation starts. Ordinary builds with CGO enabled remain supported when those settings match.

The worker compiles into a fresh private file outside the source root, preserving wildcard go:embed inputs. Only after successful compilation produces a regular file does it create output parents and an adjacent stage for atomic replacement of the previous worker file. Retrieval then stages and validates that file before replacing the local output. A missing download cannot be satisfied by an old local binary, and durable recovery resumes collection without rerunning Go. Directory outputs, symlinks or symlinked output parents, cross compilation, native-library modes such as c-shared/c-archive, and low-level flags that can create extra files are outside this output contract. Implicit forms such as go build, go build ., and go build ./... use Go's own package query to prove that exactly one main package is selected and bind its executable name before execution.

Compile-only tests also return an exact binary: for example, go test -c -o bin/package.test ./package. Both endpoints verify that the same single package has test sources; packages without tests and selections spanning multiple packages stay local. Compilation never runs the test binary. The binary uses the same fresh-file publication and collection-only recovery as go build, and its caches and temporary files stay outside the selected source tree. Package discovery for implicit builds and compile-only tests requires Go 1.18 or newer with workspace mode disabled. Test profiling, fuzz corpora, implicit test-binary names, and output-producing test execution (-coverprofile, -cpuprofile, -trace, -fuzz, -o without -c, and related -test.* forms) stay local. Strict remote mode refuses unsupported forms instead of executing them locally.

RCH explicitly does not intercept local-mutating or interactive patterns (examples):

• Package management: cargo install, cargo clean, bun install, bun add, bun remove
• Bun runners/dev: bun run, bun build, bun dev, bun x / bunx
• Nix interactive/mutating: bare nix develop / nix shell, nix run, nix repl, nix profile, nix flake update, nix store gc, nix-env, nix-shell
• Watch/background/piped/redirected commands where deterministic offload is unsafe

Yarn Berry / Plug'n'Play preparation

Linux workers can prepare Yarn Berry projects using the standard project-local PnP layout. Set enableGlobalCache: false in .yarnrc.yml, keep the default cache, unplugged, virtual and install-state paths, and declare TypeScript as a dependency of the workspace that runs the check. The worker's Yarn version must match an authored packageManager: "yarn@VERSION" declaration. Berry configuration probes require Linux descendant supervision, including when Berry uses the node-modules linker; Yarn Classic keeps its existing preparation path.

rch exec -- tsc --noEmit --incremental false --composite false

This PnP launch accepts literal bare tsc arguments with explicit non-emitting, non-incremental, non-composite settings. Disabling both incremental and composite output prevents inherited tsconfig.json settings from writing build-info files, including when the compiler exits with an error. It uses Yarn's dependency binary resolver, so a package script or global tsc cannot replace the prepared compiler. Shell wrappers, npx, Bun, watch/build modes, and emitting overrides are outside this PnP launch contract.

Qualified workspace checks run from the workspace root or a selected member's package directory. Member-local Yarn configuration and environment-file overrides that would change resolution from the prepared root are refused. Live portal: / link: dependencies and pnpIgnorePatterns are also outside this initial contract: dependency resolution must stay within the prepared package graph.

Preparation runs an immutable install. Source-qualified clean-overlay runs bind workspace preparation to the selected workspace root, record exact generated loaders, local cache/unplugged files and installation state, and rehash them before skipping a later install. Ordinary preparation reruns Yarn's cached immutable install. Committed zero-install loaders and cache files remain source; .yarn/releases, plugins and patches are never admitted as generated dependency outputs. Global or custom PnP output paths require a separate ownership contract and are refused.

Why It Works Well

• Transparent hook behavior: agents see normal command semantics.
• 5-tier classification pipeline optimized for very fast non-compilation rejection.
• Daemon-owned worker state and slot accounting.
• Cache-aware worker selection and project affinity.
• Queue + cancellation primitives for overloaded scenarios.
• Deterministic reliability subsystems for convergence, pressure handling, and remediation.
• Unified status surface with posture and remediation hints.

Current Architecture

Agent Shell / Claude Code
        |
        v
PreToolUse Hook -> rch (classifier + hook protocol)
        |
        v
      rchd (daemon)
      - worker selection
      - queueing and cancellation metadata
      - health, alerts, telemetry, history
      - reliability subsystems (convergence, pressure, triage)
        |
        v
Remote workers (rch-wkr)
      - execute build/test commands
      - manage worker cache
      - report capabilities/health/telemetry

Workspace crates:

• rch/: Hook + primary CLI
• rchd/: Local daemon + scheduling/reliability APIs
• rch-wkr/: Worker execution/caching agent
• rch-common/: Shared protocol/types/patterns/UI foundations
• rch-telemetry/: Telemetry collection/storage integration

Reliability Model (Operational)

RCH now includes a deterministic reliability stack for multi-repo and multi-worker stability:

• Path-dependency closure planning: builds can include required repository closure rather than a single root.
• Canonical topology enforcement: worker/project roots are normalized around /data/projects and /dp conventions.
• Repo convergence service: tracks worker drift vs required repos and can repair drift.
• Disk pressure resilience: pressure scoring, admission control, safe reclaim with active-build protection.
• Process triage/remediation: bounded TERM/KILL escalation with audit trail.
• Cancellation orchestration: deterministic cancellation metadata and worker health integration.
• Unified posture/reporting: status output includes posture, convergence state, pressure, and actionable remediation hints.

Installation

Recommended: Installer

curl -fsSL "https://raw.githubusercontent.com/Dicklesworthstone/remote_compilation_helper/main/install.sh?$(date +%s)" | bash -s -- --easy-mode

The installer also refreshes the rch agent skill in ~/.claude/skills/rch and ~/.codex/skills/rch. It leaves both directories untouched when jsm reports that it manages an installed rch skill, or when you pass --skip-skill (or set RCH_SKIP_SKILL=1, which suits unattended updaters). Skipping the skill never affects the binaries, the hook or the daemon.

From Source

git clone https://github.com/Dicklesworthstone/remote_compilation_helper.git
cd remote_compilation_helper
cargo build --release
cp target/release/rch ~/.local/bin/
cp target/release/rchd ~/.local/bin/

Source Build Note

All dependencies — including the FrankenTUI (ftui-*), rich_rust, and TOON (tru) crates — resolve from crates.io, so a clean git clone && cargo build builds on any machine with no special directory layout or pre-cloned dependency tree required.

rsync Requirement (macOS note)

RCH moves source and artifacts with rsync over SSH and works best with rsync 3.2 or newer (--info=progress2 progress, zstd compression, --append-verify resumes). On Linux the distro rsync qualifies.

On macOS, the stock /usr/bin/rsync is openrsync (macOS 15+) or Apple's rsync 2.6.9 (older releases); neither accepts rsync 3.x flags. RCH handles this automatically: it probes rsync --version, prefers a Homebrew/MacPorts rsync (/opt/homebrew/bin/rsync, /usr/local/bin/rsync, /opt/local/bin/rsync) even when it is not first on PATH, and otherwise drives the stock binary with a compatible flag set (--progress --stats -vv, zlib instead of zstd, and the zero-build-output detector fails open). For full-speed transfers:

brew install rsync

rch doctor reports which binary and flavour RCH resolved. To pin one explicitly, set [transfer] rsync_bin = "/path/to/rsync" or export RCH_RSYNC_BIN.

First-Time Setup

Fastest Path

rch init

rch init can guide:

1. Worker discovery from SSH config/aliases
2. Worker probing and selection
3. rch-wkr deployment
4. Toolchain synchronization
5. Daemon startup
6. Hook installation
7. Validation build

Manual Path

# 1) configure workers
mkdir -p ~/.config/rch
cat > ~/.config/rch/workers.toml << 'TOML'
[[workers]]
id = "css"
host = "203.0.113.20"
user = "ubuntu"
identity_file = "~/.ssh/id_rsa"
total_slots = 32
priority = 100
TOML

# 2) start daemon
rch daemon start

# 3) verify workers
rch workers probe --all

# 4) install hook
rch hook install

# 5) check posture
rch check
rch status --workers --jobs

Command Surface

Global flags:

-v, --verbose
-q, --quiet
-j, --json
-F, --format json|toon
--color auto|always|never
--no-color
--schema
--help-json
--robot-triage

Core Operations

rch daemon start|stop|restart|status|logs|reload
rch workers list|capabilities|probe|benchmark|compare|drain|enable|disable
rch workers init|discover|setup|deploy-binary|sync-toolchain
rch status [--workers] [--jobs]
rch check
rch queue [--watch|--follow]
rch cancel <id> | --all
rch gc [--root DIR]... [--workers <id>...]      # PREVIEW stale rch runtime dirs
rch gc --apply [--root DIR]...                  # ...and actually collect them
rch cache warm|clean|status [--workers <id>...] # remote source/target caches
rch rabs gc plan|run|history [--cas-root DIR] [--mode normal|emergency]
rch rabs worker|doctor|inventory
rch rabs worker reconcile <WORKER>

On macOS, daemon autostarts defer to a registered com.rch.daemon launchd service without restarting its running process. The managed daemon waits if another daemon still owns its socket. A private daemon bypasses this service discovery when given both a nondefault socket and an explicit workers configuration; a custom socket alone cannot be delegated to the shared service.

Fleet GC reports progress on stderr as workers finish. Each worker has a 15-minute budget shared across connections, scanned roots, and collection batches; set rch gc --worker-timeout=120 for a shorter budget. A timeout makes the command exit nonzero while preserving completed workers' results. JSON/TOON output contains one final response with those results and the error.

On timeout, GC terminates its foreground SSH process and allows two additional seconds to reap it; cleanup failures are reported. This does not confirm whether a remote collection batch finished: an interrupted --apply batch is reported as having an unknown outcome, alongside confirmed earlier batches. The default invocation remains a preview.

rch cache status also scans workers concurrently, with one 15-minute deadline and a 10 MiB output budget per worker across all roots. Set rch cache status --worker-timeout=120 to shorten the deadline. Completed workers and roots remain in the report if another scan fails; a timeout exits nonzero, with one final JSON/TOON error response retaining those results.

Windows cache status and GC inspect the same C:/rch base as the tar transport, plus the worker's resolved Cargo cache root. Every Windows directory remains kept until native process/handle ownership checks are available; --apply reports an explicit refusal. See Windows cache inventory for additional roots and incomplete-scan behavior.

Capability refreshes skip duplicate requests for a worker while its probe is running. Workers reuse verified Rustup inventory across requests and invalidate entries when installed toolchains or component metadata change. Inventory runs in bounded batches; unfinished or failed entries appear in probe_warnings, and their component capabilities remain unavailable until verified. Disk, load, and project-topology observations are collected again for each request.

Hook + Agent Integration

rch hook install|uninstall|status|test
rch shim install|status|uninstall      # cargo shim: offload builds started by scripts/Makefiles, not just hooked tool calls
rch agents list|status|install-hook|uninstall-hook
rch diagnose "cargo build --release"
rch admit "cargo build --release"      # read-only preflight: offload / local / queue / defer verdict
rch admit --job --require-tool clang -- ./run_shards.sh   # same, for a job-mode admission
rch why miss|refusal                   # RABS: explain a cache-key miss diff or an index refusal code
rch exec -- cargo build --release
rch --robot-triage --json
rch capabilities --json
rch robot-docs guide

RABS offers an experimental, opt-in worker-local dependency cache through cargo rabs --rabs-remote check. See the setup and measured validation boundaries before enabling it. Correct cache serving has passed real Cargo/SSH tests. Four native Linux scenarios also passed real compiler execution, replay, delivered binaries, and persistent worker/SDK reuse over authenticated loopback TLS. Practical acceleration remains under qualification; the measured whole-project warm run was slower than ordinary RCH.

A configured shared canonical coordinator is also available through cargo rabs --rabs-canonical-remote PROFILE.json -- build --workspace. This explicit route uploads captured inputs over pinned mutual TLS, runs Cargo at the coordinator, shares eligible dependency compilations across clients and returns verified build artifacts to a fresh private destination. Reusable profiles support Rustup toolchain selection and a separate output root. See remote Cargo profiles and command support for setup, supported commands and the current qualification limits.

Named Cargo JSON for installer consumers

An installer selecting explicit binary targets can request caller-visible Cargo JSON through the canonical Cargo shim:

RCH_CARGO_JSON_STDOUT=1 RCH_REQUIRE_REMOTE=1 cargo build --locked --release \
  --bin my-tool --bin my-alias --message-format=json,json-render-diagnostics

This opt-in requires the durable POSIX worker route and refuses local fallback, arbitrary jobs, machine envelopes, and unsupported target selections. Selected binary records reach stdout after artifact publication and acknowledged worker release. Their executable, filenames, manifest, and source paths refer to the caller tree. Selected output files are copied into an invocation-private retained directory beneath the caller's actual target root and checked against the exact publication fingerprints. A later build cannot replace these copies by publishing into the shared target paths. Each record carries an rch extension retaining the original worker record, exact retained stdout receipt, wrapper/build identity, and executable BLAKE3 publication fingerprint. Unselected dependency records keep their worker metadata and do not assert local delivery of intermediates. Diagnostics stay on stderr. Compiler failure stdout comes from the same invocation's immutable nonzero completion receipt; a successful compile with failed artifact delivery advertises no caller-local success records. The ordinary hook and rch --json exec envelope contracts are unchanged.

The delivery JSON, its private file paths, and their fingerprints are persisted before source retirement and worker release. After the original wrapper has exited, recover that same invocation without compiling again:

rch jobs recover ORIGINAL_WRAPPER_ID --cargo-json

This explicit output mode requires an invocation originally admitted with RCH_CARGO_JSON_STDOUT=1; it cannot upgrade an ordinary job. It returns verified retained stdout only after the original recovery and acknowledgement complete, and uses the original exit status. Repeating it returns the first delivery's exact JSON bytes and private paths, including after later builds or a failed stdout write. A live unacknowledged wrapper or unknown owner identity refuses detached recovery. --json machine envelopes conflict with --cargo-json. The complete compiler stderr is collected from the same supervisor log with the original completion status and retained before retirement. Its path, length, BLAKE3 and compiler status appear in the selected record's receipt. Recovery verifies those bytes and writes them to stderr without line or UTF-8 conversion. Failed compiler stdout and stderr use the exact nonzero completion; no successful Cargo record is advertised when artifact delivery failed. Older completed journals lacking retained diagnostics refuse this explicit output mode rather than fabricate diagnostic parity.

Use the exit status together with the record binding. This route does not provide native Windows delivery, an installer transaction, a source-content receipt, or performance qualification. scripts/e2e_cargo_json_stdout.py checks the actual shim, retained producer bytes, both real native binaries, cached delivery, completed same-id stdout recovery, a broken stdout pipe, and a genuine Cargo target-selection failure against an externally admitted source and artifact root. No test result is implied by this description. Wrapper-loss, pending recovery, cancellation, full installers, native Windows, and same-invocation live-incumbent/A/A performance acceptance remain open.

Job Mode (non-compilation workloads)

rch exec --job admits an arbitrary NON-compilation workload (sharded tests, fuzzing, benchmarks, mutation testing) onto the same remote rails. It bypasses ONLY the compilation classifier — the PreToolUse hook never sets it, so auto-delegation of ordinary commands is impossible.

rch exec --job -- ./run_shards.sh
# Declared result directories sync back on ANY exit code (including failures):
rch exec --job --result-dir fuzz/corpus --result-dir crashes -- ./fuzz_target.sh

Semantics: the remote exit status surfaces verbatim; no toolchain/worker-env rerun heuristics apply. A declared --result-dir that is missing or only partially transferable fails loudly (RCH-E309, exit 102) regardless of the job's own exit status. Paths must be repository-relative; conflicts with --clean-overlay / --source-content-receipt are refused.

Native Apple commands can use job mode too: direct xcodebuild and xcrun invocations require a worker declaring os = "darwin". This preserves the OS fence: ordinary jobs cannot consume that worker, and Linux workers cannot take an Xcode job. Invoke the tool directly; RCH does not inspect arbitrary scripts or infer a Mac requirement from text merely mentioning Xcode.

RCH_REQUIRE_REMOTE=1 RCH_BUILD_SLOTS=1 rch exec --job -- \
  xcodebuild -project ios/App.xcodeproj -scheme App -jobs 1 build

This is host routing, not proof that Xcode, signing credentials, or a Simulator are ready. Native test and Simulator safety checks still apply. Both rch and rchd need the native-command routing support; keep explicit strict-remote mode when qualifying a build so an older daemon cannot silently run it locally.

Requiring a verified tool

A compilation or job that needs a tool the fleet does not uniformly have can demand a worker that has verified it:

rch exec --job --require-tool clang -- ./native_fuzz.sh
rch exec --require-tool cargo-xwin --require-tool llvm-ar -- \
  cargo xwin build --locked --release --target x86_64-pc-windows-msvc --bin my-cli

The gate runs before any slot is reserved, and it is evidence-based: the worker must have run the operator-declared probe (see Worker Config Example) successfully. A project can make the requirement a default for all of its jobs:

[jobs]
required_tools = ["clang"]

Project defaults ADD to --require-tool in job mode. These [jobs] defaults apply only to jobs; explicit CLI requirements also apply to compilations while preserving their classifier, Rust target and toolchain checks. Any required tool makes execution strict remote, including when configuration forces local or disables RCH. Both client and daemon must support the verified-tools endpoint; an older daemon refuses the request instead of silently ignoring requirements.

A successful named probe establishes cached tool presence. It does not bind the executable bytes, compiler version or SDK inventory used by a particular build.

Exact Node and Bun versions. --require-tool (alias --require-toolchain) also accepts node@VERSION and bun@VERSION, matched against the version each worker reports in its capabilities (rch workers capabilities --refresh):

rch exec --require-toolchain [email protected] -- bun test
rch exec --job --require-toolchain [email protected] -- npm test

[jobs]
required_tools = ["[email protected]"]   # job mode only, like named tools

The version must be a complete MAJOR.MINOR.PATCH; an optional prerelease or build suffix must match exactly, and the v that node --version prints is ignored. Ranges, partial versions (22, 22.11) and aliases (lts) are refused, and a worker that reports no version, or another one, is not admitted. Like named tools, the check happens before reservation, makes the run strict remote, and only filters: it never reorders workers. It compares the version the worker's capability probe found. It does not install runtimes, and it does not override a version manager (nvm, fnm, a login profile) that picks a different binary when the job runs.

There is no rust@VERSION: a worker's default rustc says nothing about the compiler a job selects. Pin Rust exactly with the toolchain instead, for example channel = "1.90.0" in rust-toolchain.toml (or RUSTUP_TOOLCHAIN, or cargo +1.90.0). RCH sends that toolchain to the worker, only selects a worker on which rustup run <toolchain> rustc --version and cargo --version succeed, and runs the build under that toolchain.

rch admit [--job] [--require-tool NAME]... -- <command> preflights the same decision read-only — it syncs nothing and reserves nothing. Without --job the preflight consults the compilation classifier, so a fuzz script answers local; with it the classifier is bypassed exactly as exec --job does, and the answer is offload. The classification facts are still reported, so a compilation accidentally passed to --job still shows its family. This static preflight reports requirements and classification; it does not verify current worker readiness or establish native release qualification.

A required name that no worker has verified admits no worker at all, including when the name is a typo. That is deliberate: silently dropping the requirement would route the job to a machine that cannot run it, and job mode returns the remote exit status verbatim — so the resulting failure would be indistinguishable from the job's own. Selection reports capability_missing:tool:<name>:probe_failed when a worker declared the tool but its probe failed, and ...:not_declared when nothing declared it.

Same-identity job recovery

rch jobs                                      # list durable local job leases
rch jobs attach <wrapper-id> --timeout-secs 300 # observe the original job
rch jobs cancel <wrapper-id>                   # cancel the identity-matched job
rch jobs recover <wrapper-id> --timeout-secs 300

For an unfinished detached retrieval, the current [transfer] configuration can extend the original recipe's max_transfer_time_ms. Recovery persists a larger positive budget under its exclusive journal lock; smaller values never shorten the retained budget. Recovery extensions are bounded to 24 hours. It keeps the original worker, build, output policy, verification settings and ownership evidence, and never replays compilation. Set --timeout-secs large enough for the entire recovery operation as well; that observation deadline is separate from the transfer budget.

[transfer]
max_transfer_time_ms = 900000
retrieval_staging_dir = "/mnt/controller-storage/rch-retrieval"
retrieval_min_free_bytes = 67108864

retrieval_staging_dir is an absolute durable controller staging base, separate from the worker's remote_base, the checkout and TMPDIR. Unset uses the job-lease directory's retrieval/ subtree. Before downloading, Unix retrieval checks the selected uncompressed file size plus retrieval_min_free_bytes against space available to the controller user. Windows-worker tar retrieval also stages its bounded archive there and budgets both the archive and extracted outputs while they coexist. This preflight cannot reserve space against concurrent writers; a later storage failure remains retryable. Failure retains the outputs and ownership and does not acknowledge completion. Internal .rch-clean-overlay-base.tar source archives are excluded even when recovering an older broad Make artifact recipe.

Recovery can redirect an old recipe's staging before any phase has published or recorded a pending publication. It leaves the old stage intact and collects the same worker outputs into the new location. Once publication begins, a location change is refused to preserve its pending and published evidence; extend its timeout or provide space on its existing filesystem instead. Terminal acknowledgment still requires verified delivery and source release.

rch --json jobs reports valid leases alongside journal_errors. An unreadable or malformed journal, or an entry observed as nonregular, makes complete false and the command exits 1, while valid jobs remain visible in the same JSON document. Each valid row also reports owner_presence as live, absent or unknown; a false legacy wrapper_alive value alone does not establish that an owner is absent. The listing does not remove, repair or treat the unreadable entry as an absent owner. An initially missing journal directory is an empty, complete listing. The directory is opened once and journal names are read relative to its file descriptor. Journal opens reject symlinks and do not wait for a FIFO writer; the opened descriptor must represent a regular file before any bytes are read. The directory root itself must be a directory rather than a symlink. Initial attach/recover loads and recovery-owner reloads use the same regular-file, nofollow, nonblocking reader. A changed parent directory is refused; a FIFO cannot block these journal reads while waiting for a writer. Detached recovery retains the opened directory and persistent lock through journal publication and terminal reporting. Its clones share that ownership; reads and atomic writes use the held directory. A replaced directory or lock refuses acknowledgement. Failed recovery publications retain their exclusive temporary bytes for diagnosis; recovery does not remove them. Ordinary running wrappers keep their existing journal writer. complete describes this scan, not an immutable ownership snapshot: entries and regular-file contents can still change during observation. If a journal path contains non-UTF-8 bytes, its error has a printable path and an exact Unix path_bytes array. A job command whose output cannot be written or flushed exits with an error rather than reporting successful delivery.

Completed/recovered reports include an admission reference from one committed terminal observation: exact decimal build ID, wrapper, worker, journal path and caller fingerprint, matching the ordinary execution reference. Unadmitted terminal jobs have admission: null. Unknown exit, incomplete identity or unacknowledged state cannot become a completed report. This reports journal identity/custody; it does not replay the original execution envelope or attest compiler, SDK, slots or artifact contents.

These commands never replay the original command. Recovery requires retained identity and completion/retrieval evidence; ambiguous or missing evidence is reported as an error. For a live wrapper stalled during artifact retrieval, recover_requested means the request was recorded, not that outputs have arrived: use jobs attach to observe completion. The wrapper cancels and reaps the old transfer before retrying that retrieval phase. An absent wrapper can recover outstanding outputs from its retained journal without starting another build. Use a rebuilt daemon with the identity-aware job routes; older daemons are refused.

On POSIX workers, source ownership is saved before synchronization starts and remains active through output retrieval, including after the SSH lock holder disappears. Another job cannot overwrite an overlapping source root while that ownership is unresolved; disjoint source trees can still run concurrently. Uploads, verification, execution, and collection check the same ownership token. Operations within one token are serialized. An uncertain preparation failure ends that preparation; its token is drained and cancelled before a new attempt, so an older upload cannot arrive after verification and change compiler inputs. Recovery of an interrupted preparation drains surviving transfers and cancels that token, so a delayed upload cannot mutate a tree after ownership is released. Once execution may have started, recovery requires its exact completion record or worker activity evidence that the original execution is gone; a missing receipt alone does not prove completion. If a worker loses its ownership registry or the local recovery recipe predates this protocol, recovery refuses to infer ownership from the current source bytes. Worker cache cleanup also respects retained source grants, including jobs whose outputs are quiet but still await recovery.

rch jobs recover also reconciles an acknowledged terminal cancellation when its retained recipe still owes worker settlement. A local state=finished and terminal_acknowledged=true are insufficient by themselves: recovery verifies the exact journal, failed result, resource retirement, worker records and drained activity before recording source_cancellation_settled=true. This operation does not reopen source trees or rerun the command. A retained .cancelling marker may remain after success to fence delayed requests for the old token; its presence alone does not block a new token. Missing or conflicting evidence remains an error rather than permission to remove records.

Recovery records the source registry namespace as part of its durable recipe. For older journals it first verifies exact worker evidence and checkpoints the current or legacy binding before cancellation or cleanup. This keeps recovery on the original registry and lock hierarchy instead of creating a contradictory receipt in a newer namespace. Ambiguous or incomplete evidence remains refused; cleanup also checks for competing lexical and physical owners.

New POSIX source-lock sessions send heartbeats every 10 seconds and expire after 90 seconds without a complete heartbeat, subject to worker process scheduling and termination. Expiry retains durable ownership and leaves separately running compiler/rsync activity to the normal recovery protocol. See the source namespace and terminal cancellation runbook for diagnostics, legacy registry handling, and the native-worker acceptance still required for #111.

Config + Diagnostics

rch config show|get|set|reset|init|validate|lint|doctor|edit|diff|export
rch doctor [--fix] [--dry-run]
rch doctor --reliability [--check-schemas] [--scope <scope>] [--strict|--lenient] [--json]
rch doctor --reliability --watch [--watch-interval N] [--transitions-only]
rch doctor --runbook RCH-Rnnn | --runbook-list
rch error explain <RCH-Ennn|RCH-Innn|RCH-Rnnn> | list [--category <name>]
rch self-test [--worker <id>|--all]
rch self-test status
rch self-test history --limit 10

Fleet + Release + UX

rch update [--check|--rollback|--fleet]
rch fleet deploy|rollback|status|verify|drain|history
rch speedscore <worker>|--all [--history]
rch dashboard   # alias: rch tui
rch web
rch schema export|list
rch completions generate|install|uninstall|status

Agent Discovery Surface

For AI agents and automation, start with:

rch --robot-triage --json       # quick_ref + recommended commands + health probes
rch capabilities --json         # commands, aliases, env vars, exit codes, output formats
rch robot-docs guide            # in-tool operating guide, no README lookup needed
rch --help-json workers/list    # machine-readable help for nested command paths

--json uses the standard API envelope for command output; --format toon emits the same data as TOON. The legacy rch --capabilities flag remains a raw JSON shortcut for lightweight discovery.

Configuration

Primary files:

• User config: ~/.config/rch/config.toml
• Worker list: ~/.config/rch/workers.toml
• Project override: .rch/config.toml
• Optional project excludes: .rchignore

Precedence (highest first):

1. CLI flags
2. Environment variables
3. Profile defaults
4. .env / .rch.env
5. Project config
6. User config
7. Built-in defaults

Canonical Project Root

By default rch expects projects to live under /data/projects (with the /dp alias symlink). Both roots are configurable, so repos under ~/code, /workspace, etc. work without relocating anything:

[path_topology]
canonical_root = "/home/me/code"   # default: /data/projects
alias_root = "/home/me/code"       # default: /dp (set equal to canonical_root if you have no alias)

Or via the CLI / environment (env wins over TOML):

rch config set path_topology.canonical_root /home/me/code
rch config set path_topology.alias_root /home/me/code
# or
export RCH_CANONICAL_PROJECT_ROOT=/home/me/code
export RCH_ALIAS_PROJECT_ROOT=/home/me/code

Empty strings are treated as unset (defaults apply). See docs/guides/configuration.md ([path_topology]) for details.

Loaded or High-Latency Workers

Increase bounded control-operation deadlines together when healthy workers need more time for SSH setup, filesystem scans or lock acquisition:

[timeouts]
scale = 2.5                  # default 1.0; finite values from 1.0 through 10.0

RCH_TIMING_SCALE=2.5 rch exec -- cargo check overrides the client policy for one invocation. The daemon reads its own user configuration and environment at startup; restart rchd after changing its policy. A project override or an environment variable on rch exec does not reconfigure an already running daemon. Inspect the effective client value with rch config get timeouts.scale or rch config show --sources.

The scale extends slow-path control deadlines and bounded retry windows. The shared timing policy caps each scaled control budget at one hour, including at scale 1.0. Build and test execution time limits, signal-escalation grace, polling cadence, disk thresholds, ownership checks and the proof required to release a worker remain separate. Invalid, non-finite or out-of-range values are configuration errors. See the configuration guide for scope and precedence.

Minimal Example

[general]
enabled = true
force_local = false
force_remote = false
socket_path = "~/.cache/rch/rch.sock"
log_level = "info"

[compilation]
confidence_threshold = 0.85
min_local_time_ms = 2000
remote_speedup_threshold = 1.2
build_slots = 4
test_slots = 8
check_slots = 2
# Optional additional build-disk headroom for each remote job (GiB).
# Include expected output growth and a safety margin. Zero disables it.
# disk_headroom_gib = 80
build_timeout_sec = 300
test_timeout_sec = 1800
bun_timeout_sec = 600
external_timeout_enabled = true

[transfer]
compression_level = 3
# Staging base for the transfer paths that do NOT mirror the client's layout:
# `--clean-overlay` roots, Windows workers, and `rch cache warm`. An ordinary
# `rch exec` mirrors the project under the worker's `[path_topology]
# canonical_root` instead, with the pooled Cargo target store inside that
# mirror — see `[remediation.pooled_target] store_base` below to place those
# stores on another filesystem.
remote_base = "/data/tmp/rch"
# Optional durable controller staging base (absolute, independent of TMPDIR).
# retrieval_staging_dir = "/mnt/controller-storage/rch-retrieval"
# Free-space reserve in addition to the selected uncompressed outputs.
# retrieval_min_free_bytes = 67108864
# Optional per-attempt source-sync cap. When unset, the default is payload-aware:
# 30 seconds plus one second per MiB, capped at one hour.
# sync_timeout_ms = 120000
# Abort a source sync after this many seconds of NO rsync output (dead channel /
# wedged rsync); a progressing transfer is never affected. 0 disables.
# source_sync_silence_timeout_secs = 120
# Explicit rsync binary. Unset lets RCH prefer a modern (3.x) rsync over the
# stock macOS openrsync automatically; RCH_RSYNC_BIN overrides this.
# rsync_bin = "/opt/homebrew/bin/rsync"
# Opt in to one supervised SSH connection throughout a Unix dispatch, shared
# by preflight, source locks, rsync, verification, execution, and retrieval.
# The foreground master is reaped when the dispatch and its cleanup finish;
# it does not leave a background ControlPersist process between invocations.
# Unset or zero keeps the existing one-shot exec paths. Windows workers and
# automatic recovery do not use this dispatch connection. Enable it for
# high-latency workers: without it every SSH step pays a fresh handshake.
# ssh_control_persist_secs = 60
adaptive_compression = true
verify_artifacts = false
max_transfer_mb = 2048

[remediation.pooled_target]
# Where pooled Cargo target stores are PLACED on the worker. Unset (the
# default) keeps them inside the project mirror, i.e. under the canonical
# root, on whatever filesystem holds it. Set an absolute path to move every
# pooled store to <store_base>/<project_id>/.rch-target-<worker>-pool-<key>,
# which is how a worker with a small root disk keeps multi-GB warm pools on a
# larger volume. `rch gc`, `rch cache status`, and the daemon sweep scan this
# root in addition to `remote_base` (with `reaper_max_cache_gb` applying per
# scan root). Ignored for Windows workers.
# store_base = "/bigdisk/rch-pools"
#
# Where the reaper SCANS for stale target dirs (not a placement setting).
# remote_base = "/data/projects"
#
# EXTRA roots `rch gc` scans, on top of the ones it derives itself
# (`remote_base`, `store_base`, and the worker temp base resolved exactly as
# the code that creates the dirs resolves it: $TMPDIR -> /data/tmp -> /tmp).
# Use this for a runtime root rch cannot know about, e.g. an operator build
# root on a mounted volume. Each entry must be absolute, `..`-free and free of
# shell metacharacters -- it is embedded in a remote command. `rch gc --root`
# adds one for a single run.
# gc_extra_roots = ["/mnt/big/rch"]
#
# Idle days before `rch gc` may collect a durable per-worker Cargo cache
# (`rch-cargo-cache-*`). 0 disables. Collection ALSO requires zero open file
# descriptors under the dir and no live process rooted at it; a gate that
# cannot be evaluated counts as "in use".
# gc_cargo_cache_idle_days = 14

[selection]
strategy = "balanced"
# Reserve this much free disk, then budget space for each concurrent slot (GiB).
min_free_gb = 10.0
disk_gb_per_slot = 10.0
# Critical when both this ratio and the absolute cap below are reached.
# disk_critical_free_ratio = 0.05
# Default cap in GiB; use "uncapped" for an unconditional critical ratio.
# disk_critical_ratio_cap_gb = 25.0

[self_healing]
hook_starts_daemon = true
daemon_installs_hooks = true

[alerts]
enabled = true
suppress_duplicates_secs = 300

When the Unix launcher enforces a compilation deadline, RCH preserves exit 137 and reports that the configured deadline expired. It does not retry on a larger worker or fall back locally. Machine-mode rch exec reports outcome: "deadline_exceeded". This classification requires evidence from the launcher; exit 137 or elapsed time alone does not establish a timeout.

For projects with large build outputs, set compilation.disk_headroom_gib in .rch/config.toml. A value of 80 requires 80 GiB of additional space on the worker's reported build filesystem. Selection requires a successful disk probe within 90 seconds and subtracts budgets already held by active builds in this daemon. Budgets survive daemon restart and remain held until the owning build completes. Releasing a budget does not prove its output files freed any space: the next budgeted admission requires a disk probe started after that completion. A probe already in flight cannot reuse the earlier free-space reading. Retries, queue recovery, and rch diagnose use the same requirement. Smaller CPU-slot estimates and cache affinity cannot bypass it. Older daemons reject the distinct budgeted selection endpoint rather than ignoring the requirement.

This is admission accounting for the worker's reported canonical/alias build roots, not a filesystem quota or a measurement of arbitrary custom target mounts. Other dispatchers, undeclared jobs, and external writes are outside its accounting. The full budget stays reserved even after a disk sample reflects some of the build's output, so admission deliberately errs toward leaving extra space. The default is 0, retaining ordinary disk-pressure admission.

Without a declaration, the daemon still learns each project's footprint. For every remote build that had a worker to itself (no other build from this daemon overlapped there), it records how far the worker's free build-disk space fell between the admission probe and the lowest probe seen while the build ran. Footprints are kept per project and command class (cargo test is learned separately from cargo check), for 30 days, in history.footprints.json beside the build history. When some candidate worker has room for the largest recent footprint plus 10% (at least 5 GiB), after declared budgets and the remaining growth of builds already running there, selection only considers those workers. If none has room, selection is unchanged. Learned footprints are evidence, not a budget, so they never refuse a build. Other dispatchers' builds can inflate a measurement and cache cleanup can shrink one; declare disk_headroom_gib when you need a hard requirement.

Unix artifact downloads estimate the files matched by the retrieval filters before transferring them. Their default total retry budget grows with that size (30 seconds plus one second per MiB, adjusted for a slower bwlimit_kbps, up to one hour). An explicit max_transfer_time_ms keeps its existing hard ceiling and skips estimation. Rsync also stops after 30 seconds without network I/O. If estimation fails, RCH logs the failure and retains the configured retry budget. A literal cargo build --bin NAME or --example NAME retrieves only the named payloads, not the pooled target's other cached outputs. Anything the narrowing cannot parse exactly (unknown flags, shell expansion, other target kinds, response files, custom target specs) keeps the broad per-profile filters, which can include other cached build outputs in the same profile.

Windows workers return artifacts through tar in both quiet and interactive execution. The transfer honors the requested output patterns and cache exclusions, including custom Cargo target directories and explicit compiler outputs outside target/. RCH inventories regular files, downloads only selected paths, and validates the complete archive before writing outputs locally. Remote tar errors, missing selected files, and unexpected archive members fail artifact retrieval; successful downloads include file counts and a manifest for output validation. Inventory and transfer use bounded deadlines, and cancellation reaps the transport before recovery can begin. This transport requires GNU find from the worker's Git for Windows installation and native tar.

Artifact staging retains the original caller directory as the source-protection reference. Recovery restores that reference for each output phase, so an empty staging directory cannot make source files eligible for a broad output pattern.

Built-in worker selection defaults to balanced, which blends speed, load, health, and cache affinity. Use priority only when you want explicit worker-priority control, and fair_fastest when you want extra load spreading.

Worker capacity is capped at the configured total_slots and reduced to floor((free_disk_gb - min_free_gb) / disk_gb_per_slot) as disk fills, with a minimum of zero. Unknown disk telemetry keeps the configured ceiling. Existing builds retain their reservations when capacity drops; new work waits for space. rch workers list reports the effective total when the daemon is reachable. Change the budget with rch config set selection.disk_gb_per_slot 10 and restart the daemon to apply it.

Separately, on build filesystems of at least 50 GiB, rchd marks a worker Critical when free space falls to 10 GiB, or when both the free ratio is at or below selection.disk_critical_free_ratio (default 0.05) and free space is at or below selection.disk_critical_ratio_cap_gb (default 25.0 GiB). These defaults also apply when the dispatcher has no config.toml. A 900 GiB disk with 30 GiB free retains disk-budgeted capacity despite its low free percentage; the same disk with 8 GiB free stays Critical. Existing small-filesystem and /tmp rules still apply. A noncritical classification alone does not admit a build: fresh storage evidence, active reservations, the authorized build budget and safety buffers must still fit.

Explicit caps such as 15.0 or 30.0 and custom critical ratios remain honored. Set disk_critical_ratio_cap_gb = "uncapped" to enforce the ratio independently of absolute free space. A finite cap must be at least 10 GiB; invalid numeric caps fall back to the stricter uncapped rule with a daemon warning. Inspect or change the settings with rch config get selection.disk_critical_ratio_cap_gb --sources and rch config set selection.disk_critical_ratio_cap_gb 25. The get and set commands also accept selection.disk_critical_free_ratio. Both keys take effect when rchd restarts, and its startup log reports the effective disk policy. The cap does not change per-slot allowances, protected free-space floors, or authorized build-storage budgets.

Balanced selection also favors free disk before a worker reaches the admission floor. selection.weights.disk (default 0.2, range 0 to 1) adds disk headroom credit after the other score adjustments. Equal ample headroom preserves the existing worker order. Credit rises from zero at min_free_gb to full credit at 25% free disk, with at least one slot's disk budget above the floor on small disks. Unknown measurements receive full credit; the admission gate still handles missing telemetry. Affinity pins and explicit worker requests retain their existing behavior. Set the weight to zero to disable disk ranking independently of disk admission and slot limits, then restart the daemon to apply the change.

rch workers list shows daemon-reported free disk and pressure. rch workers probe shows fresh disk measurements alongside latency and labels any cached daemon pressure separately. JSON and TOON include numeric free GiB and free ratios; unavailable measurements remain null (displayed as unknown in text).

Worker Config Example

[[workers]]
id = "css"
host = "203.0.113.20"
user = "ubuntu"
identity_file = "~/.ssh/id_rsa"
total_slots = 32
priority = 100
tags = ["fast", "ssd"]

# Verified named tools. Each entry is a FIXED argv the worker runs as the
# configured user; a zero exit marks the tool present, anything else marks it
# absent. Unlike `tags`, which are an unverified naming convention, these are
# evidence — which is what lets `--require-tool` gate worker selection.
tools = [
  { name = "clang",  command = ["clang", "--version"] },
  { name = "ld.lld", command = ["/usr/bin/ld.lld", "--version"] },
]

Names must be ASCII letters, digits, -, _, . or +: a name carrying a space or = could not survive the probe's fact format and would be read back as a different name, so it is refused when the config loads rather than becoming a capability lie. rch workers capabilities --refresh reports the results under Named tools as verified: and failed: — a declared probe that fails is a different operational state from one that was never declared, and only the first points at a broken worker.

There is deliberately no general remote-shell probe API. A caller that could ask a worker to run an arbitrary command "to check for a tool" would be a remote execution primitive wearing a capability-probe hat; declaring the argv in operator config keeps the set of probe commands finite and reviewable.

Worker Execution Policy (Linux workers)

A worker can lower the priority of the builds RCH runs on it, or confine them to a systemd slice, independently of rchd's own service settings. The policy is set on the worker, in exactly one of these files:

• /etc/rch/worker-execution.conf, owned by root, or
• ~/.config/rch/worker-execution.conf of the SSH user, owned by that user or by root.

version=1
nice=10
systemd_slice=jobs.slice
systemd_manager=user

version=1 is required, plus at least one of nice (-20 to 19) and systemd_slice (an existing, active *.slice unit). systemd_manager (user, the default, or system) names the manager that owns the slice and is only valid together with systemd_slice. Every setting in the file is enforced before the build starts. The launch is refused, never run unconfined, when the file is unsafe (not a regular single-link ASCII file of at most 8 KiB, writable by group or others, wrongly owned), has unknown or duplicate keys, or exists in both places, and when enforcement fails (no busctl, systemctl or timeout, an inactive slice, a failed priority change, a cgroup placement it cannot confirm, or a non-Linux worker). The launcher writes the cause to its stderr as RCH_WORKER_POLICY_REFUSED:<reason>, and what it applied as RCH_WORKER_EXECUTION_POLICY:nice=…,slice=…,manager=…,scope=….

With a slice, RCH attaches the build's existing process-group leader to a transient scope in that slice, so the watchdog, cancellation and recovery keep working, and cleanup covers the whole cgroup. Without a policy file, builds run as before.

The policy covers the tracked build payload only. Source sync, pre-launch package bootstrap and untracked execution are not confined, so it does not on its own make a shared developer machine safe to admit as a worker (#104).

Output Modes

RCH auto-selects output mode by context:

• hook: strict JSON for hook protocol
• machine: explicit machine output (--json, --format)
• interactive: rich terminal rendering
• colored: ANSI-only when forced without TTY
• plain: text fallback

Environment controls:

• RCH_JSON=1, RCH_HOOK_MODE=1
• NO_COLOR=1, FORCE_COLOR=1, FORCE_COLOR=0
• RCH_OUTPUT_FORMAT=json|toon, TOON_DEFAULT_FORMAT

JSON responses use a stable envelope (api_version, timestamp, success, data, error).

Placement Controls

Worker placement, strict-remote, queue, wait-timeout, visibility, and target-dir behavior are first-class, canonical controls — not folklore. The authoritative list is discoverable at runtime (rch capabilities --json), and the resolved plan for any command is shown by rch diagnose <command> --json under data.placement (and in the human Placement Controls section):

The resolved plan reports requested_worker, requested_profile, effective_worker, strict_remote_policy, queue_policy, visibility_mode, wait_timeout_ms, target_dir_policy, the requested-worker admissibility outcome, and a diagnostics list. Any control value that cannot be applied as written (an unrecognized value or a superseded alias) surfaces a diagnostic rather than being silently ignored.

The simulated selection also lists every worker the daemon considered, under data.worker_selection.inventory (human output: Inventory: under Worker Selection (simulated)). Each entry is selected, eligible (passed every gate, ranked lower), fallback_candidate or excluded, with the selector's reason and reason codes, for example available slots 2 < 4 (slots.insufficient), not in the requested worker set (selection.not_requested), or a project exclusion such as project.live_checkout_fenced. A daemon older than the client reports no inventory after a successful selection.

Monitoring and Observability

RCH exposes observability through daemon APIs and metrics:

• daemon health/readiness endpoints
• Prometheus metrics collection
• opt-in OTLP export of daemon request-duration metrics
• telemetry-backed worker SpeedScore history
• queue/build history, active alerts, cancellation metadata in status APIs
• an opt-in tailnet status API ([api] bind = "tailscale"): the daemon's full /status JSON over TCP, bearer-token gated, bound only to loopback or Tailscale addresses — so agents on other machines can ask a dispatcher "why are you local-only right now?" without ssh
• the fleet dashboard under dashboard/ (encrypted static console + an agent endpoint GET /api/fleet?view=problems|diagnose|help); rch web opens it

Enable OTLP metrics in the daemon's environment with RCH_OTEL_ENABLED=1 and RCH_OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317 (OTLP/gRPC). The endpoint falls back to OTEL_EXPORTER_OTLP_ENDPOINT. OTEL_SERVICE_NAME defaults to rchd; RCH_OTEL_EXPORT_INTERVAL_SECS defaults to 30. rch_request_duration_seconds{entrypoint="rchd_api"} records finite Unix-socket request handling after parsing, including errors and cancellation. Event streams and the separate HTTP API are excluded. The same observations reach Prometheus, including when OTLP is disabled. Shutdown drains request tasks before a best-effort final export. The banner reports exporter configuration, not collector connectivity.

The same opt-in settings enable direct OTLP verdict export for single-shot rch doctor --reliability, including under --quiet. Its service name defaults to rch. Each report emits one rch_doctor_verdict_total observation, labeled with the actual verdict and scope (combined scopes use other), and flushes before returning its original exit code. Reports whose daemon_unreachable flag is true also increment rch_doctor_daemon_unreachable_total once, even when several diagnostics report the missing daemon. The asynchronous daemon_status and repo_convergence probes also emit rch_doctor_probe_duration_seconds{probe,result} once on completion, RPC error, or timeout. Durations measure each probe's execution, not time spent waiting to join other probes. Skipped probes emit nothing; panic/cancellation and helper/ownership probe durations are not yet exported. With --fix, each remediation step also emits rch_doctor_fix_steps_total and rch_doctor_fix_duration_seconds, labeled by its actual outcome: applied, already_satisfied, would_apply, manual, or failed. Dry-run previews remain would_apply; they are never counted as applied changes. Durations measure step execution, excluding the initial shared configuration load. These CLI metrics are not forwarded to the daemon's Prometheus endpoint. Hook/watch telemetry, remaining doctor event metrics, and trace/log export remain unfinished.

Quick checks:

rch status --workers --jobs
rch speedscore --all
rch doctor --json

On Linux boxes configured with [general] role = "dispatcher", status and doctor also report unmanaged local Cargo/rustc processes. The warning remains visible while those processes run, including when the daemon is unavailable. Check rch shim status, PATH order, and absolute-path toolchain Cargo invocations. Processes descended from rch or carrying rch's internal local-fallback marker (RCH_CARGO_WRAPPER_BYPASS=1 with RCH_CARGO_WRAPPER_ORIGIN=rch) are excluded; a bare caller-set RCH_CARGO_WRAPPER_BYPASS=1 is not excluded.

An alarm event is logged once per observed episode and clears after a successful scan finds no unmanaged compiler processes. Status and doctor share a locked one-byte latch at $XDG_CACHE_HOME/rch/local-build-alarm (normally ~/.cache/rch/local-build-alarm). Scan failures preserve the previous episode; latch failures are reported without hiding current builds. Reliability doctor includes this check in the triage scope and its default all scope.

Testing and Validation

Workspace checks:

cargo fmt --check
cargo check --workspace --all-targets
cargo clippy --workspace --all-targets -- -D warnings
cargo test --workspace

Reliability and E2E suites are provided under:

• tests/
• tests/e2e/
• rch-common/tests/ (contract/reliability/perf suites)

If you are running CPU-intensive validation manually and want explicit offload:

rch exec -- cargo check --workspace --all-targets
rch exec -- cargo test --workspace
rch exec -- cargo clippy --workspace --all-targets -- -D warnings

When local fallback is not acceptable, set RCH_REQUIRE_REMOTE=1 on the rch exec process and keep the build command as direct argv:

RCH_REQUIRE_REMOTE=1 rch exec -- cargo test --workspace
RCH_REQUIRE_REMOTE=1 rch exec -- cargo clippy --workspace --all-targets -- -D warnings

rch exec --require-remote -- <cmd> asks the same for one call without touching the environment. Because it is always the caller's own request (the cargo shim exports RCH_REQUIRE_REMOTE=1 into every agent), it also outranks config that would run the command locally (force_local, enabled = false, the execution allowlist), as --worker does. A non-compilation command is refused with RCH-E301 instead of running locally; to run one on a worker, use rch exec --job -- <cmd>.

Parallel dependency uploads

Ordinary Cargo builds on Unix workers can upload up to eight independent dependency roots concurrently while holding their durable source grant. Every upload keeps checksum comparison and its existing transfer timeout, progress reporting, and receiver safety checks. Source verification, compilation, and retirement wait for all admitted uploads to finish. A failed upload stops new admission, drains every upload already admitted, and enters the existing durable preparation-cancellation protocol before another build can reuse those sources.

Overlapping source roots keep their original order. A worker destination with a symlinked ancestor or an ambiguous physical directory identity uses the exclusive upload path. Clean-overlay archive materialization and Windows transfers keep their existing ordering. A positive bwlimit_kbps also keeps uploads sequential so concurrency cannot multiply the configured bandwidth cap.

Exact source-content receipts

Proof-oriented callers can require a single-worker, fail-closed source transfer and retain the exact regular-file bytes admitted on that worker:

RCH_REQUIRE_REMOTE=1 rch exec --source-content-receipt -- \
  cargo test --locked --workspace

Receipt mode resolves the active Cargo path-dependency closure, stages its roots beneath the configured worker build base, transfers with checksum comparison, and reopens every selected file before and after the command. Preflight initializes the isolated staging topology and skips canonical worker repository convergence. The emitted rch.source_content_receipt.v1 JSON binds the worker and build IDs, exact command digest and exit code, transfer filter policy, per-root file paths, lengths, executable bits, SHA-256 digests, and content roots. Any unproved dependency closure, transfer delta, worker verification error, local lasting change, retry, or local fallback refuses the invocation instead of emitting a receipt. Receipt mode requires the Unix rsync transport.

While holding the source grant, receipt verification uses rsync to check the selected namespace and metadata, then independently verifies every worker file against the captured SHA-256 manifest and recaptures the local inputs. This avoids a duplicate full-file rsync checksum pass during verification. Uploads still use checksum comparison, and verification without a source grant retains the checksum barrier. Equal size and timestamps never establish byte identity.

Clean-overlay builds can request the same byte-level proof:

rch exec --base HEAD --clean-overlay --overlay-path src/lib.rs \
  --source-content-receipt -- cargo test --locked

The manifest covers the complete selected Git archive plus the exact files transferred by the explicit overlay filters. Unselected working-tree edits never become receipt inputs. --no-overlay proves the committed tree and its pinned submodules; each --dependency-base PATH=REV adds its own selected root. The filter policy records the commit, tree, invocation directory, pinned submodules, overlay paths, and fingerprint separately from the rsync rules, because those rules select only the live overlay. Every selected file is checked on the worker before and after execution, including committed files that are absent or dirty in the local checkout. Receipt mode preserves the clean-overlay source/target pair's stable paths and warm compiler cache. An additional read-only namespace check rejects extra worker files, symlinks, or directories, including in base-only runs. Only RCH-owned runtime paths and the exact root archive transport file are exempt; selected source inputs that overlap those exclusions refuse before upload. Missing selected paths also refuse. Commands that leave source changes or outputs outside the owned runtime paths therefore cannot produce a successful source-content receipt.

On Linux with target reuse enabled, supported Cargo commands use an owned source/target pair. The source layout preserves sibling dependencies, selected manifests, and enclosing workspace metadata under a stable source-content-pair-<key>/projects/ container. Every invocation materializes fresh source files there while holding the pair and its complete source grant. Uploads use checksums without deleting entries or running cache sweeps; the read-only rsync barrier still rejects surplus selected files. Cargo's target and build directories are bound to the granted pool outside that container.

An unchanged file can reuse its previous compiler-visible timestamp only when the previous attempt completed its source-content verification. An interrupted or unverified attempt forces fresh timestamps beyond the pool's artifacts on the next invocation. After artifact delivery, the source container moves to its recorded .retained-<attempt> sibling using Linux no-replace rename. The job journal retains both paths before acquisition, so recovery can reconcile an interrupted move without replacing an existing retained tree. Python 3 and the Linux rename capability are checked before source ownership is acquired.

This owned route validates Cargo manifests and configuration against the captured transfer inventory. Relative and absolute --manifest-path selectors map into that inventory. File-based --config, custom target-specification files, directory-changing wrappers, and caller overrides of managed output paths or Cargo home currently refuse before execution. Other Unix platforms, invocations with RCH_DISABLE_TARGET_REUSE=1, and external Cargo drivers such as Nextest, Zigbuild, and xwin retain invocation-unique staging paths. The owned route currently covers the built-in build, test, check, documentation, benchmark, and verified packaging families, plus Clippy.

The receipt is emitted after the remote command and its post-command source barrier, but before ordinary artifact retrieval completes. Artifact-grade callers must therefore retain both the receipt and the later successful retrieval/terminal transcript; the receipt alone is not proof that build artifacts reached the caller. A recursive local mutation watcher is still needed when transient edit-and-restore (ABA) detection is part of the caller's source-stability contract.

Clean Git overlays for shared working trees

rch exec can build an immutable committed tree plus an explicit, repeatable set of local paths without transferring unrelated working-tree changes:

RCH_REQUIRE_REMOTE=1 rch exec \
  --base HEAD \
  --clean-overlay \
  --overlay-path src/lib.rs \
  --overlay-path tests/focused.rs \
  -- cargo test --test focused

RCH_REQUIRE_REMOTE=1 rch exec \
  --base HEAD \
  --clean-overlay \
  --no-overlay \
  -- cargo check --workspace --all-targets

The client resolves --base to a commit object, streams that Git archive into a fresh isolated worker path, and then uploads only the literal repository- relative --overlay-path selections. Modified and untracked files are supported, together with selected deletions of tracked regular files and directories containing only regular files. A directory selection also includes tracked files and subdirectories removed within that directory. Deleted entries are omitted from the composed archive and source receipt; no deleting rsync is used, and cached source bases remain intact. Their absence is rechecked before execution, so a deleted input that reappears invalidates the admitted selection.

For a rename, select both the old and new paths, or select their containing directory. Selecting only the destination keeps the unselected original from the base. For example, after renaming src/old.rs to src/new.rs:

rch exec --base HEAD --clean-overlay \
  --overlay-path src/old.rs --overlay-path src/new.rs \
  -- cargo check

Missing paths that were not present in the base, absolute/traversing paths, Git metadata, non-ASCII/backslash/control-character paths, case-only or otherwise ambiguous filesystem spellings, overlay symlinks, and Git archive export-ignore/export-subst attributes fail closed. A selected directory may become empty through admitted deletions; newly selected empty directories remain unsupported. Exactly one of one-or-more --overlay-path options or --no-overlay is required. Clean-overlay mode also implies remote-only execution even when RCH_REQUIRE_REMOTE was omitted, so a worker or transfer failure never falls back to the ambient local tree. Explicit clean-overlay execution also admits the read-only cargo fmt --check diagnostic, which the ordinary interception classifier intentionally leaves local.

On Unix workers, clean-overlay commands can start below the Git root. RCH materializes the entire selected repository and executes from the corresponding selected subdirectory. Cargo discovers the nearest selected ancestor manifest and toolchain; dirty or untracked nearer files do not change that choice. Relative command arguments, --manifest-path, --dependency-base, target destinations, and job result directories retain their caller-directory basis. --overlay-path always remains relative to the Git root:

cd crates/app/src
rch exec --base HEAD --clean-overlay --overlay-path crates/app/src/lib.rs \
  -- cargo test --manifest-path ../Cargo.toml

The invocation directory must exist in the selected base or overlays, with an unambiguous spelling and no selected file or symlink ancestor. Trailing-space directory names are refused. The receipt names the invocation directory, and recovery keeps command/output locations separate from the full source roots. Named Cargo JSON outputs use the full selected source root to map ancestor manifests back to the caller, including after detached recovery. Source leases and retirement retain the complete selected roots. Disk-growth accounting still covers the full primary source root.

Initialized Git submodules are materialized recursively at the exact gitlink commits in the selected base. Their dirty working files and current checkout revisions do not affect the selected bytes. Each submodule must have the pinned commit locally; RCH never fetches or updates a checkout during admission. Missing pins, uninitialized or symlinked submodule roots, archive-transform attributes or NUL-containing attribute files in any selected repository, and overlays inside or containing a submodule fail closed. All clean-overlay compilers, including direct rustc, validate selected symlinks before upload: targets must exist inside the selected source inventory without escapes or cycles. The source fingerprint and execution receipt bind every submodule path, commit, and Git tree. Admission allows at most 1,024 submodules and 32 nesting levels, including queued descendants. Composed archives preserve executable permissions and symlinks and refuse case-aliased destination paths.

Admission reads pinned symlink targets, Cargo metadata, and archive attributes in Git object batches per selected repository. Symlink resolution still uses the complete selected inventory, so a linked Cargo configuration is read from its pinned target even when that target has a different filename. Every batch response must match the requested object ID, blob type, and exact byte framing; missing objects, malformed responses, and bounded-resource failures refuse admission. This reduces Git process startup work without consulting dirty checkout contents or introducing a timestamp-based source cache.

On Unix workers, pooled Cargo builds use a stable source path paired with a versioned target cache. Jobs sharing that pair wait until the preceding job has finished execution, artifact retrieval, and source retirement. Different pairs remain independent. Source timestamps are refreshed beyond cached artifacts so changed archive or overlay contents cannot appear older than a previous build; registry and Git dependency caches remain reusable. These paired builds also retain immutable Git archives in a private directory inside the owned target pool. Each selected repository has a separate slot, so primary and sibling dependency archives can stay warm together. Before reuse, the worker checks the bounded cache header against the caller's requested archive. A different identity is a miss without reading the old payload. A matching archive is copied into the fresh source tree while its length and SHA-256 are verified in one pass against the archive constructed by the caller; only a verified private copy can proceed to extraction. A verified hit skips the base archive upload; local archive creation and source-content verification still run. The cache admits at most 64 slots and 1 GiB including pending publications, and preserves the configured worker free-space reserve. Capacity limits or unavailable Python/layout support use the ordinary upload path. Partial or corrupt cache files never count as hits. Cache files are excluded from artifact retrieval, including priority includes and recovery. Workers without durable paired source/target ownership continue to use ordinary archive uploads. Clean-overlay execution also pins Cargo's build.build-dir to its assigned target directory, overriding ambient and command-line Cargo configuration. Explicit --build-dir options are refused. This placement applies to isolated builds too, so disabling target reuse cannot inherit shared intermediates. Managed placement currently supports built-in Cargo build commands and Clippy. Other compiling Cargo subcommands, including Nextest and Zigbuild, are refused in clean-overlay mode.

An interrupted pair retains its ownership marker and refuses further reuse, even if its SSH lock connection disappears. Inspect the prior job before recovering that pair, or use RCH_DISABLE_TARGET_REUSE=1 for a fresh isolated build. Non-Cargo Windows clean-overlay jobs also use isolated targets. The ownership marker protects against process interruption; it is not a power-loss durability guarantee.

Clean-overlay materializes the primary Git repository and explicitly bound sibling repositories. In-repository Cargo workspace members are present in the archive. Before Cargo starts, selected manifests, configuration, symlinks, and command-line path overrides are checked against the selected Git base and overlays. Escaping paths are refused with RCH-E413; retained sibling directories on the worker cannot supply those dependencies. To include a sibling Git repository, bind its revision explicitly:

rch exec --base HEAD --clean-overlay --no-overlay \
  --dependency-base ../dep=HEAD -- cargo test

Repeat --dependency-base PATH=REV for every external root, including transitive dependencies. Each revision resolves to a commit before worker selection; dirty files in those repositories are excluded. The selected repositories must be siblings of the primary Git root. They are staged under one owned container, preserving ../dep references, and the execution receipt names each commit, Git tree hash, and remote path. All roots share the same execution lease and are retired together. Cross-repository symlinks and non-sibling layouts are currently refused.

Manifest semantics follow the selected entrypoint's workspace members and local dependency closure, including dev, build, target-specific, optional, inherited, patched and replaced path dependencies. Unreferenced Cargo manifests remain transferred fixture bytes; naming the same fixture as a member, dependency or entrypoint activates the same containment and source checks. Workspace excludes are literal directory prefixes; explicit members override them, while excluded default members stay excluded. Exclusions never hide actual path dependencies. Legacy Cargo dependency-table and package-table aliases retain these checks. Workspace pointers in selected ancestor packages activate their captured root too; absolute, absent, aliased or non-workspace targets refuse before Cargo. Selected manifest-file symlinks are refused because resolving their bytes must not change Cargo's relative source base; unreferenced aliases remain data. All inventory paths/symlinks and selected Cargo configuration retain their containment checks. File-based --config, configuration includes, and changes to Cargo's working directory through -C or wrapper --chdir are refused. On the worker, selected ancestor configuration within the primary Git root is allowed; Cargo-home configuration and configuration above that root are refused before Cargo starts, even if harmless. Cargo clean-overlay jobs require a Unix worker for this check. These checks do not sandbox arbitrary file reads by build scripts. The client re-fingerprints overlays after upload and refuses execution if their contents changed during admission or transfer.

Do not batch several Cargo commands behind a shell wrapper such as rch exec -- bash -lc "cargo test ... && cargo test ...". Shell-wrapped commands are classified as non-compilation for hook safety. Under RCH_REQUIRE_REMOTE=1, RCH refuses that local fallback before executing the shell and reports RCH-E301; without the env var, ordinary non-compilation commands may still run locally. For several focused checks, run separate direct RCH_REQUIRE_REMOTE=1 rch exec -- cargo ... invocations.

Security Model

• Transport uses SSH.
• Worker commands are constrained to classified execution paths.
• Sensitive field masking and structured error taxonomy are built in.
• rch update enforces the release's SHA-256 checksums and verifies the archive's .minisig against a public key pinned in the binary (minisign key 69B3955C8D2E62A8, which signs every release since 2.1.0; 2.0.x and earlier used 1BBD79B28BF718D0); a bad signature aborts the update. A Sigstore bundle is also verified when the release ships one and cosign is installed. --skip-verify bypasses signature checks only. install.sh still verifies checksums only.
• Hook path remains fail-open to avoid deadlocks/stalls.

Operational recommendations:

1. Use workers you control.
2. Use dedicated SSH keys for worker access.
3. Keep workers patched and isolated.
4. Enable telemetry/alerting for production-like use.

Limitations

• Designed around SSH-based Linux worker environments.
• Tooling assumptions are strongest for Rust and selected build/test commands.
• Remote performance gains depend on network + worker capacity + project shape.
• The fleet dashboard (dashboard/) needs Node and a static host (Vercel/Pages); its live path needs a Vercel Blob store and rchd's [api] listener on a tailnet.

FAQ

Does RCH block my command if the daemon/workers fail?

No. It fails open and allows local execution.

Can I force local or force remote per project?

Yes, via .rch/config.toml (general.force_local / general.force_remote).

Is queue/cancel supported?

Yes. Use rch queue and rch cancel.

Can I inspect why a command is or is not intercepted?

Yes. Use rch diagnose "<command>".

About Contributions

Please don't take this the wrong way, but I do not accept outside contributions for my projects. You can still open issues and PRs for discussion/proof-of-fix, but I review and re-implement changes independently.

License

MIT License with an OpenAI/Anthropic rider — this is not ordinary OSI MIT: no rights are granted to the restricted parties named in the rider. See LICENSE for the exact, controlling terms (SPDX: LicenseRef-MIT-OpenAI-Anthropic-Rider).