The full project, one page: what the system does, the measured numbers behind every claim, the videos of every skill, and the failures — reported the same way as the wins. All simulation (MuJoCo), all numbers measured by us.
An operator types a plant-floor incident in their own words. A fine-tuned 22.7M-parameter bi-encoder retrieves the governing Standard Operating Procedure from a corpus of 1,012 SOPs; a deterministic faithful planner turns the SOP into a skill chain (one skill per written step); and a simulated Unitree G1 humanoid executes that chain in one continuous MuJoCo simulation — walk, pick, carry, place, press, lever — with a verifier measuring every step against simulator state, not the policy’s own claim. A step the physics doesn’t confirm is reported as failed. The finale below is exactly that: the system retrieves the right SOP at rank 1 (score 0.897), runs the five-action plan in one take, and honestly reports the steps that failed — verdict: “partially resolved.”
The benchmark was rebuilt to be hard: 1,012 SOPs across 46 equipment families with within-family hard negatives, and 2,338 incidents written in operator voice that deliberately avoid the SOP’s title words. On the held-out test (269 incidents, full index), the fine-tuned MiniLM bi-encoder nearly doubles keyword search at R@1. Two negative results shaped the system more than the headline: a 10×-larger from-scratch bert-base retriever lost to TF-IDF, and an off-the-shelf cross-encoder reranker hurt the strong retriever — it is excluded.
† bert-base row measured on the smaller 418-SOP validation corpus (where TF-IDF scores 0.396 R@1); all other rows are the full 1,012-SOP / 269-incident held-out test, reproduced locally. Reranker row from the original training-repo evaluation (0.572 base → 0.442 reranked).

The architecture thesis was forced on us by evidence (see §04): where the task is geometric (walk to a pose, servo a gripper), a controller wins; where contact dynamics dominate (press a spring-loaded button, drive a latched lever), RL earns its complexity. The verifier doesn’t care which produced the motion — it measures outcomes.
RL grasping failed outright and became a controller. Across ~10 reward/configuration variants, deterministic verified success was 0. The training metrics said 30–60% “lift” — the videos showed the box being squeezed out of a crushing pinch and popping upward ballistically, with the height check firing mid-air. Three compounding causes: a contact-fidelity wall in the simulator’s pinch resolution, a dithering stochastic grip that never achieves force closure, and a reward that cannot tell held from launched. The fix was to stop using RL: the IK + force-gated-latch controller held 11/12 lifts on its first day. RL went where it earns its complexity — the press and the lever.
Every one of these was caught by a human watching the rendered rollout — none by a metric. Watch the video, not the metric.
The complete paper — system, retrieval benchmark, the RL post-mortem, the reward-hack catalog, limitations, and everything this page summarizes — lives in the repo. All numbers in it are measured; failures are first-class results.
Read the paper ↗ Training log →