A running record of the skill-learning work on a Unitree G1 in MuJoCo — newest first.
Times are wall-clock per run; results are deterministic evals measured against the simulator state, not the
policy’s own claim.
Jul 3–5 2026 · The finale: ticket → brain → robot
The loop closes. An operator ticket in plain words goes to the trained retriever over the
1,012-SOP corpus (R@1 0.584 reproduced locally on this machine), the faithful planner turns the retrieved
SOP into a skill chain, and the same robot executes it in one continuous simulation — with every step
verified against measured physics and the verdict reported honestly, including the step that failed.
G6 · one takeall criteria pass
G7 · ticket → robotSOP retrieved rank 1
g6 one-take
The full loco-manipulation chain in a single unbroken simulation: walk to the table → pick
(force-gated latch) → carry 2.8 m with a 180° turn → place at the panel → RL press
34.5 mm, PUMPS=0 → return to rest. Box never dropped, robot never fell, every criterion measured
in-run. The take also surfaced a real bug (a stale carry force corrupting the place) — which is why gates
are run as takes, not stitched from clips.
one take · all pass walk→pick→carry→place→press
gate 7
First ticket-to-robot run: a plain-language ticket → the trained 1,012-SOP retriever
(semantic MiniLM interleaved with lexical TF-IDF) → faithful plan → the robot walks up and executes the
RL press. Non-embodied SOP steps (wait, read, notify) are captioned on screen rather than mimed.
R@1 0.584 reproduced ticket → walk + press
finale
An authored workstation SOP (SOP-1013, written for the demo and disclosed as such) is retrieved at
rank 1 (score 0.897) over the full corpus and its five-action plan runs in one take: pick, carry,
place, press 33.8 mm all PASS — and the lever fails honestly: best of 4 stance variants
drove it 0.883→0.443 rad against a <0.25 success bar. The verdict card says
PARTIALLY RESOLVED, on camera, because that is what happened.
4/5 steps pass · lever fail verdict: partially resolved
Problem solved: the thesis loop exists end to end — ticket → retrieval → plan →
verified execution → honest verdict, on one robot in one simulation. The remaining gap is named and queued:
the lever’s ~50% reliability traces to a per-episode arm bias the policy cannot observe; the obs fix +
retrain is next. A system that reports its own failed step is the whole point.
Jul 2–3 2026 · The action-envelope wall, press-once, and the lever’s first throw
A day of root causes. The biggest: the policy’s action envelope (±0.5 rad) was smaller
than the reach itself — from the rest pose the arm physically could not command the button’s contact pose,
which had silently capped every far-start attempt. Widening it (0.8 press / 1.2 lever) unblocked both skills.
The rest of the day was closing the gap between “the metric says pressed” and “a human watching
agrees” — the operator kept catching behaviors the numbers missed.
walk → press oncePUMPS=0 measured
lever · BC→PPO51° throw · ~50% det.
press-once
Three operator-caught failures, three fixes: the policy pump-cycled the button
(per-step depth income made re-pressing profitable → success now terminates the episode); then it
air-swung after the press (motion was free → post-press stillness cost + a dip-tolerant hold counter);
and the walk→press seam twitched scene objects (the handoff reset touched world state → full-pose
snapshot/restore, settle rendered on camera). The pump count is now measured inside the demo: PUMPS=0.
press once · hold · stop one-take walk→press
lever
Redesigned to a breaker-style pull-down (rests latched up; gravity assists) after the
up-throw read as fake. Pure PPO never completed the 0.63-rad arc (12 iterations: reward parking, seat-income
farming, exploration noise flinging the latch). The unlock: BC warm-start from scripted IK arc-follow demos,
then PPO — the same recipe that cracked the original press — plus an entropy crush to consolidate.
First full deterministic throw: 51° into the success band. Honest state: ~50% reliable and the motion is
jerky; root cause identified (the arm’s per-episode bias isn’t observable to the policy) and queued.
BC → PPO → ent-crush 1/2 det. seeds · WIP
Problem solved (mostly): the press is a finished, verified skill — walk up, press once,
done. The lever crossed from “never completes” to “completes half the time, ugly” — real, but not
shippable; reliability + smoothness are next. The deeper lesson repeated all day: watch the video, not the
metric — every reward hack was caught by human eyes first.
Jun 26–Jul 2 2026 · One robot, honest RL — the reach becomes learned
Three arcs in one week. First: one unified robot — the walking AMO humanoid with the
Robotiq gripper grafted onto its right wrist — replaces the two separate rigs, and every skill moves onto it.
Second: the press gets honest — smooth, collision-real, same-side. Third, the headline: the reach itself
becomes RL. Until now the arm was IK-servoed onto the target at episode start and RL only learned the final
push — a thin slice. A success-gated curriculum now trains the whole reach-and-press from the rest pose.
pick · IK + latchheld lift
press · low-passjerk 0.83→0.001
RL reach+press19 cm → 31 mm
unified
One embodiment for everything: gripper grafted on the walking humanoid (collision masks, AMO
reads re-addressed), pick moved from a floating pedestal to the real workstation table. The pick is a
controller — DLS-IK reach + force-gated latch. Fixing the close (clamp a fixed target instead of chasing
the block) cut the block’s slide from 13 cm to 2 cm and stopped it phasing into the tabletop.
controller, not RL held lift · jitter 0.12 mm
press
The anti-jerk reward penalty got reward-hacked — the policy flailed fast and the
depth metric called the accidental bumps “presses.” Fix at the control level instead: a low-pass on the arm
command makes flailing physically impossible (jerk 0.83→0.001, press held 165+ steps). Then a real find: the
blue button was a cross-body reach — with true arm↔torso collision enabled the arm simply can’t
get there. Switched to the same-side yellow button; the collision stays on as a hard guarantee. Base recoil
rock damped 10.4→6.6 cm with a base-stability term.
2.5M / run · ~30 min 25.9 mm held · upright
curriculum
The overnight result. Arm starts interpolated between the contact pose (trivial) and the
rest pose (full reach); the start distance grows every time recent success clears the gate. Two fixes made
it converge: the reach reward must target the button’s cap face, not its body origin (a hidden 7.7 cm
offset that flattened the gradient), and an anti-parking band — the first run learned to reach, then
hovered 5 cm short farming proximity. After both: the deterministic policy reaches ~19 cm from the true
rest pose and presses 28–31 mm, every curriculum level, no IK seed.
10.2M · overnight, 64-core CPU reach+press, all levels
Problem solved: the “RL” in the stack is no longer a thin slice on top of IK — the
reach is learned. It also answered the infrastructure question with data: MuJoCo on CPU can train this;
the Isaac Lab move is a choice, not an escape. In flight: a hold + stance-robustness retrain (so the press
survives wherever the walk lands, no reset at handoff) and the same curriculum ported to the lever.
Jun 22–24 2026 · Grasp rebuild — Robotiq 2F-85
The earlier grasp attempts failed on embodiment, not method: a rigid hand with no actuated
fingers, and a reward that paid for proximity. We rebuilt from the gripper up — a real Robotiq 2F-85
parallel gripper on a fixed base, and a reward gated on a true two-sided grip plus a sustained lift. First we
proved the tool was reachable and a clean grasp pose existed; then we trained.
gr-v0flings it
gr-v1holds 16/25
gr-v260% held
gr-v0
Pure RL from scratch, 1.5M steps, no demo and no curriculum. Learned to reach and grip on its own — then flung the tool 22 cm up ballistically, contact flickering. The reward paid for height, so it maximised height.
1.5M · ~33 min grasp 90% / hold 0%
gr-v1
Anti-yank reward redesign (resume gr-v0): cap the lift reward at 10 cm, make a steady hold pay more than height, pay out only after the grip is continuous, penalise ballistic tool velocity. The fling disappeared — it lifts to ~8 cm and holds 16 of the 25 steps a verified success needs.
1.5M · ~25 min grasp 100% / hold 16/25
gr-v2
Same reward, just +2.5M more training. The grip now holds the tool aloft for the full half-second in 60% of episodes — a real, measured grasp on the rebuilt gripper.
2.5M · ~40 min grasp 100% / hold 60%
Retracted: these gr-v0–v2 numbers were on a BROKEN sim — the arm drove the gripper 7.5 cm through the solid pedestal and the block spawned mid-air, so the “grasp” was a phase-through-and-jam exploit. gr-v2 (60% hold) scores 0% on the fixed sim. We fixed the contact + tool settle and rebaselined: gr-v4 learns a real 95% grip (no phasing) but won’t lift yet; a softened grip-stability reward + lift run (gr-v5) is training. See the research log for the video.
Jun 2026 · Button press
Pressing an e-stop button on the machine. The long-blamed “the base falls over” turned out to be
already fixed — the real blocker was geometric: at the spawn point the hand maxed out ~0.48 m short of the
button.
bp-v0–v2
Made the button reachable (spawn + reach bias), then reshaped the reward to reward press depth, not a threshold graze. Reward tuning alone stalled: hover → 1 cm → hover again.
2M each · ~22 min deterministic hover
bp-v5
Switched to a protruding mushroom e-stop and a closed-loop IK demo to seed behaviour cloning, then PPO. The base recoil was the real problem; with that handled the press held.
5.3 cm press held 79 steps · 10/10
Problem solved: stopped debugging a phantom (“balance”) and fixed the actual blockers — reachability, then base recoil. Full NL → SOP → walk + press demo runs verified.
Earlier 2026 · First grasp & the verifier — “policies lie”
The first grasp work (rigid hand) is where the core lesson came from: a policy’s reported success
is not the truth. Scaling the same reward 25× on cloud compute made it better at the score and worse at the
job.
v5.5
Best honest skill of that era — a real grasp-and-hold. Claimed 60% grasped; the verifier confirmed 55% as a sustained hold.
claim 60% → verified 55%
v5.6
10M steps on the cloud. Grasped 45% (a record) but refused to lift — it farmed the grasp and circled its hand. The policy claimed 30% success; the verifier confirmed 0%.
claim 30% → verified 0%
verifier
We stopped patching the reward and built a verifying executor: every skill carries machine-checked pre/postconditions, and a step counts only if the physics agrees it happened. It also exposed that an early “success” logged momentary lifts — fixed with a no-early-stop eval window.
measured vs sim state
Problem solved: a way to tell a real skill from a reward hack — the thing the whole project is built on. It also told the uncomfortable truth that our own best grasp only held ~1 in 3 times, which is what drove the gripper rebuild above.
Foundation · Feasibility & SOP study
The earliest result, and the reason the planner can trust the body: a calibrated balance-margin
threshold that predicts whether the real AMO-controlled G1 will stay up.
predictor
Across a payload × reach grid (n=24 rollouts of the real controller), a single balance-margin threshold of −0.066 m separated every configuration that fell from every one that stood.
100% separation n = 24
planner
A typed SOP planner reasons over that calibrated contract: a step it predicts infeasible is re-planned (more conservative reach) or aborted before the robot ever moves — closed loop on feasibility, not on a language model’s opinion.
re-plan / abort
Problem solved: grounding — a checkable, physical contract between the planner and the body, instead of asking a model “did that work?”
All MuJoCo simulation, single operator, proof-of-concept · not yet hardware ·
the SOP planner is not yet wired into the skill executor ·
← back to the work
