RECORD SHA‑256 CHAINED · TRANSPORT SHM SEQLOCK · LATENCY <1 ms
When a machine does something consequential, a bag file tells you what it did. Little tells you what it was asked to do, which safety layer overrode that, who moved a parameter — or whether the record has been edited since. OmniLoop writes that record from inside the loop, and it is tamper-evident. Python, C++ and Rust.
Reward shaping is the iteration loop, and every hypothesis costs a full run. Drag a reward term’s weight and watch return respond over the next two hundred iterations — three hypotheses in one eight-hour run instead of three runs. When it goes NaN at hour fourteen, the loop freezes with its memory intact and the black box still holds the frames that explain it.
The same tripwires follow it onto the machine, and exactly one thing changes meaning: halt. A controller that stops emitting torque does not pause — it trips a watchdog and the arm drops. So a halt hands the loop to your fallback controller and keeps commanding at rate, inside limits the core enforces against every write.
A hand-tuned Rust seqlock for telemetry and a length-prefixed ring buffer for commands. Point clouds, LiDAR matrices and symbolic state cross process boundaries with microsecond transport overhead — no sockets in the hot path.
Don’t restart. Reward-term weights, PID gains, curriculum thresholds and boolean interlocks change inside an active tick, bound straight to the object that owns them — the reward manager, the optimizer, the controller. The loop never notices, except that it behaves better.
Freezing a training run is free; freezing a loop that is holding an arm up is a
fall. So the loop’s author declares how it may be stopped. Under
handoff the barrier never blocks — it reports
run_safety, and your fallback controller drives at full rate while the
state stays frozen for inspection. A deadman covers the operator who halts and
walks away. The policy is set by the process holding the actuators and by nothing
on the wire.
A safety layer that silently corrects a command and leaves no trail is the gap nobody fills. When limit enforcement clamps a write, the journal takes a record of its own: asked for 250 Nm, applied 100, because above_max, at the request of agent-7. Tripwires carry the same treatment — arm one against a requirement id and every trip it produces names what it was defending, without a mapping table living outside the file.
Append-only .omni journals capture every frame. Scrub back to the exception,
inspect the exact state that tripped it, then fork a fresh worker hot-patched with
that frame's variables.
Every record advances a SHA-256 chain covering the record header as well as the
payload, checkpointed as the run goes and again when a segment is sealed. Alter a
value, drop a record, re-date one, re-order two — omniloop verify says so,
including when the payload checksum has been recomputed to match. Optional
Ed25519 signing closes the gap the chain alone cannot: nothing in a chain is secret,
so a wholesale rewrite can produce a consistent one; it cannot produce the signature.
Writing signatures needs a key. Checking them needs neither a key nor a
build flag — an auditor should never have to trust how you compiled it.
An MCP server exposes the same control plane to
Claude, Cursor or any MCP host: the exception, the file, the line and the
±5 lines of source behind a frozen loop — then test the fix live and get back a
config diff. Read-only by default, and write access carries a real envelope:
a hard [min, max] installed in the running loop’s own core, which
reports back what it is enforcing.
The SDK instruments your control loop and publishes state into shared memory. The server rebroadcasts it to the console — and forwards your mutations straight back into the command ring.
TRANSPORT SHM SEQLOCK · LATENCY <1 ms · DIRECTION BIDIRECTIONAL
from omniloop import Loop
# No adapter, no framework. `cfg` is your own config object;
# these fields become live, bounded, recorded controls.
loop = Loop(
source=cfg,
tunable=["kp", "kd", "max_torque"],
limits={"max_torque": (-100.0, 100.0)}, # clamps are journaled
journal="run.omni",
)
loop.watch_all_nonfinite(True) # freeze on the first NaN
loop.watch("joint_torque", "greater_than", 100.0,
requirement="REQ-SAFE-014") # the trip names what it defends
while running:
with loop.tick(): # applies edits, evaluates tripwires
state = robot.read()
robot.command(controller.step(state, cfg))
loop.log(torque=state.torque)
// 1 kHz controller. Press Halt in the dashboard and the arm HOLDS —
// a loop that stops commanding does not pause, it falls.
omniloop::Config cfg;
cfg.halt_policy = omniloop::HaltPolicy::Handoff;
cfg.halt_timeout = std::chrono::seconds(5); // deadman
omniloop::Loop loop(cfg);
loop.tunable("kp", kp_, 0.0, 500.0); // live slider, zero-copy
loop.limit("kp", 0.0, 400.0); // hard envelope, in-process
loop.readout_array("joint_torque", torque_);
loop.watch_array_above("joint_torque", 100.0);
loop.deadline(std::chrono::microseconds(1000));
while (running_)
loop.step([&]{ controller_.update(dt); }, // body
[&]{ damping_.hold(dt); }); // fallback
from omniloop import load_events, JournalPlayer, verify_chain
# Before reading a trace somebody sent you, check it is the one they wrote.
result = verify_chain("run.omni")
if not result["ok"]:
raise SystemExit(f"tampered or damaged: {result['break_at']}")
# Mutations, freezes, watchpoint trips and guardrail downgrades
for e in load_events("run.omni"):
print(e["tick"], e["kind"], e["event"])
# Walk the run. `read_next_data_record` skips chain checkpoints.
player = JournalPlayer("run.omni")
while rec := player.read_next_data_record():
if rec["kind"] == "telemetry":
apply_state(rec["payload"])
# The robot froze at 03:14. The journal is the only witness.
$ omniloop why run.omni --tick 4350
TICK KIND DETAIL
-------- ---------- --------------------------------------------
4348 mutation p_gain: 1.2 -> 3.5 (by local)
4349 control resume
4350 control freeze_on_exception ValueError: velocity limit
# Is this the file that was written, or a file that was edited?
$ omniloop verify run.omni
declared chained : True
records walked : 8734
checkpoints : 137
[ok] chain intact — no record was altered, removed, re-timed, or re-ordered.
# Did last night's rerun actually reproduce it?
$ omniloop diff run.omni replayed.omni
[fail] first divergence at tick 4350
# Is the channel live, or just holding a dead frame?
$ omniloop status
Shared Memory IPC: [ok] live — publishing at ~1004 Hz
Telemetry Server: [ok] online (1 dashboard connected)
# The agent has the loop. Nobody is at the dashboard.
$ omniloop mcp --allow-control
→ omniloop_set_watchpoint(name="reward_mean", condition="non_finite")
→ omniloop_wait_for_halt() # blocks. no polling a 1 kHz loop.
halted: true cause: watchpoint tick: 8412
watch_trip: reward_mean non_finite = nan
exception:
file: train.py line: 214
code_lines: | 212 advantages = returns - values
| 213 advantages = (advantages - advantages.mean())
| 214 / advantages.std() <-- std() == 0
→ omniloop_set_variable(name="p_gain", value=1.4)
correlation_id: 5169c608… observed: 0.0001 applied: true
# A clamped write is never reported as a success:
→ omniloop_set_variable(name="p_gain", value=9.0)
observed: 0.005 applied: false
note: the loop is NOT running the value that was asked for.
# And the envelope is armed in the loop's own core, not just at the relay:
→ omniloop_set_bounds(name="p_gain", min=0.0, max=2.0)
enforced_by: [relay, target] target_confirmed: true
// Handles, not &mut — so a tick can be open while the body reads state,
// and a dangling binding is unrepresentable.
let mut lp = Config::new()
.halt_policy(HaltPolicy::Handoff)
.halt_timeout(Duration::from_secs(5))
.open()?;
let kp = lp.tunable("kp", 120.0, 0.0..=500.0)?;
let err = lp.readout("tracking_error", 0.0)?;
lp.limit("kp", 0.0..=400.0)?;
lp.deadline(Duration::from_micros(1000))?;
while running {
lp.step(
|| controller.update(dt), // body
|| damping.hold(dt), // fallback while halted
);
}
This is a live PID loop running in your browser at 60 Hz. Drag a fader and the mutation lands mid-tick — exactly how OmniLoop patches a real robot's RAM. Halt it. Nothing dies.
OmniLoop is free and open-source software dual-licensed under MIT or Apache-2.0. Get started in under 5 minutes.
.omni recording, replay & omniloop verifyIntegrating with custom ROS 2 nodes, Isaac Sim, MuJoCo or bare-metal actuators? Send your configuration — a robotics engineer answers, not a bot.