Dexterity Card Protocol

Scope

robots: Cobot Magic; seeds: 8 (3 original + 5 new); demos: 20

Question

On Cobot Magic at 20 demos the fair-protocol transfer gain was +3.7% [-4.0%, +11.4%] with 3 seeds, an interval that includes zero. Is it above zero with more seeds?

Hypothesis

H1: the pooled Cobot Magic transfer gain at 20 demos is above zero. H0: it is zero or negative.

Design

Five more seeds (3 to 7, eight in all), each with its own pretraining and demo draw, same learning rates, same 3000 fine-tune steps, same 30 test episodes, zero-noise decoding. Nothing else changes: no new metric, budget, learning rate or decode.

Metric

Open-loop translation error on the test split; gain = 1 - sum(error_transfer) / sum(error_scratch). 95% interval: a t-interval combining between-seed variance with an episode-bootstrap variance.

Decision rule

• 1. GAIN: the interval's lower bound is above 0.
• 2. LOSS: the upper bound is below 0.
• 3. NO MATERIAL GAIN: the upper bound is below +5% (margin fixed before the data).
• 4. Otherwise INCONCLUSIVE.

Stopping rules

Single look at all eight seeds; no interim analysis; no seeds added afterwards; every run enters, including odd ones; infrastructure failures are re-run with the same seed and plan.

Stated expectation

An INCONCLUSIVE outcome with a tighter interval was judged a likely result and accepted in advance.

Verdict

GAIN (rule 1)

Result

+4.2% [+1.0%, +7.4%] at 20 demos with eight seeds. The three original seeds give +3.7% [-4.0%, +11.4%]. Two of the eight seeds are individually negative. The fair macro (+8.7%) was not recomputed and keeps the three-seed Cobot Magic cell.

Deviations

• The registration says the analysis refuses a test-set mismatch. The script does not actually compare episode ids; this was checked separately by hand (all 16 primary records share the same 30 test episodes) and passed. The analysis itself was not changed.

Limits

Open-loop error, not task success; grippers, not hands; the 30 test episodes bound the interval at about ±2.7 percentage points however many seeds run; the new pretrains ran on a later code image than seeds 0 to 2, so bit-for-bit equality with them is not claimed.

Results reported in

XE-0.1 pre-registered follow-ups, results 2026-10-06

Result records

tr:fair:cobot:20:s8 in release.json

Registration

2026-10-05 08:46:52 UTC, commit 8748bb9 in RC’s internal repository; full text available on request (sha256 86f34bbd30869c42…).

Caveat. The scratch arm is still on a loss plateau at the registered 3000 steps; at 10000 steps it clears the plateau and the gain is +5.8% [+3.3, +8.3].

Scope

robots: Cobot Magic, Galbot G1, R1 Lite; seeds: 3; demos: 20

Question

On stack_baskets the fair-protocol macro gain at 20 demos is +8.7% [+5.8%, +11.5%]. That is one task. Does a fair-protocol gain above zero also appear on a different task, with different scenes and garments?

Hypothesis

H1: on fold_clothes, the fair macro transfer gain at 20 demos over Cobot Magic, Galbot G1 and R1 Lite is above zero. H0: it is zero or negative. Nothing else is tested.

Design

Same design as stack_baskets: 20 demos, seeds 0 to 2, both arms on identical demos and the identical 30 test episodes, learning rate chosen per arm on a validation split, zero-noise decoding. The learning-rate grids were widened to the final stack_baskets grid, which keeps every stack_baskets pick interior. Known differences from stack_baskets (shorter pretraining, two source robots, end-effector-only action ids) were fixed in advance and are not adjusted for.

Metric

Open-loop translation error on the test split; macro gain over the three robots; same 95% interval as above. The author had looked at no fold_clothes result value when this was written.

Caveat

The scratch arm is still on a loss plateau at the registered 3000 steps; at 10000 steps it clears the plateau and the gain is +5.8% [+3.3, +8.3].

Decision rule

• Applied only if the headline guards pass (chosen learning rate not on a grid edge, no diverged run, arms not identical).
• 1. REPLICATES: the interval's lower bound is above 0.
• 2. DOES NOT REPLICATE (loss): the upper bound is below 0.
• 3. DOES NOT REPLICATE (no material gain): the upper bound is below +5%.
• 4. Otherwise INCONCLUSIVE; if the guard is still refused after the extension budget, INCONCLUSIVE with no headline.

Extension rule

If a chosen learning rate sits on an edge of its grid, run the next learning rate on that side for all three seeds, decided from validation records only. At most two extension rounds per robot and arm.

Verdict

REPLICATES (rule 1)

Result

Fair macro gain at 20 demos +11.6% [+9.7%, +13.4%]. Per robot: Cobot Magic +12.6% [+9.9%, +15.2%], Galbot G1 +20.2% [+16.7%, +23.7%], R1 Lite +1.9% [-1.7%, +5.5%] (interval includes zero). The point estimate lies just above the stack_baskets interval; magnitude is descriptive only. The per-robot pattern differs from stack_baskets, where R1 Lite had the largest gain.

Caveat

The scratch arm is still on a loss plateau at the registered 3000 steps; at 10000 steps it clears the plateau and the gain is +5.8% [+3.3, +8.3].

Deviations

• The extension rule fired once: Cobot Magic's scratch pick sat at the lower edge of its grid (1e-4), so 3e-5 was run for all three seeds. The pick stayed at 1e-4, now interior, so no second round was needed. The extension was decided from validation records only, before any fold_clothes test number was opened.
• The registration planned secondary single-sample and mean-of-8 results and a fixed-learning-rate comparison with the legacy protocol. These are not reported: under those decodes every scratch model was still on a loss plateau and its numbers are artefacts. The primary zero-noise decode is not affected by this.

Limits

Open-loop error, not task success; grippers; 30 test episodes per robot; three seeds. Galbot G1 is the only robot not shared with stack_baskets, and it has the largest gain.

Results reported in

XE-0.1 pre-registered follow-ups, second pass 2026-10-07, with the fold decode diagnosis

Result records

tr:fair:macro:20:fold in release.json

Registration

2026-10-05 08:52:01 UTC, commit 24b777c in RC’s internal repository; full text available on request (sha256 0d6f38639e630d8e…).

Scope

robots: Cobot Magic, R1 Lite, RMC-AIDA-L; seeds: 3; demos: 5 / 20 / 50 (R1 Lite 47)

Question

Under the fair protocol, how many of its own demos does the scratch model need to match what transfer reaches with 20, and does the gain shrink as the target robot gets more demos? Also: the old budgets confound demos with training steps (2k, 3k and 5k steps at 5, 20 and 50 demos).

Hypothesis

H1 (primary): per robot, scratch needs more than 20 demos to match transfer at 20 (n* > 20). H2: the gain shrinks with more demos (gain at the top budget < gain at 20). H3 (step control): the shrinkage from 20 to 50 demos is not just the extra steps: at a fixed 3k steps the gain at 50 is also below the gain at 20.

Design

Three robots (Cobot Magic, R1 Lite, RMC-AIDA-L), seeds 0 to 2, transfer vs scratch on identical nested demo lists, 5 / 20 / 50 demos (R1 Lite 47: all of its non-validation episodes, the same list for every seed), plus a fixed-3k-step control at 20 and 50 demos for Cobot Magic and RMC-AIDA-L. Four cells at 20 demos and Cobot Magic at 5 demos were reused from earlier sweeps and had been seen; the new cells had not. Learning-rate grids were centred on the nearest earlier pick. Part of the control design was cut for cost and can only be run under a new registration.

Metric

Open-loop translation error on the 30 fixed test episodes, zero-noise decoding. n* is where the scratch curve (isotonic, log-linear between budgets, no extrapolation) reaches the error transfer has at 20 demos.

Decision rule

1. H1 SUPPORTED if the n* interval's lower bound is above 20; CONTRADICTED if its upper bound is at most 20; otherwise INCONCLUSIVE. A censored n* ('more than the top budget') supports H1 only if the lower bound is above 20.
2. H2 SUPPORTED if the interval of gain(top) - gain(20) is entirely below 0; CONTRADICTED if entirely above 0; otherwise INCONCLUSIVE.
3. H3 SUPPORTED if the interval of gain(50 @ 3k) - gain(20 @ 3k) is entirely below 0; CONTRADICTED if entirely above; otherwise INCONCLUSIVE.
4. A verdict on a robot whose cells carry a headline-blocking flag is labelled 'no headline'.

Extension rule

If a new cell's validation pick sits on a grid edge, add the next learning rate outward for that arm (all three seeds), at most two per cell and arm, decided from validation records only.

Stopping rules

Single look: test numbers of the new cells are opened once, after every planned job and every triggered extension is done. No run is stopped, re-run or added because of its result.

Stated expectation

With three seeds the between-seed component dominates, intervals were expected to be wide and several verdicts INCONCLUSIVE. This was accepted in advance.

Verdict

H1 SUPPORTED for R1 Lite and RMC-AIDA-L, INCONCLUSIVE for Cobot Magic. H2 SUPPORTED for R1 Lite only, INCONCLUSIVE for Cobot Magic and RMC-AIDA-L. H3 INCONCLUSIVE.

Result

Scratch demos needed to match transfer at 20: more than 47 for R1 Lite (95% CI lower bound 40.2), 32.3 [22.6, 46.1] for RMC-AIDA-L, 26.1 [13.7, 49.7] for Cobot Magic (not resolved). Fair macro gain +19.7% [+10.8%, +28.6%] at 5 demos, +8.7% [+5.8%, +11.5%] at 20 and +6.4% [+4.5%, +8.2%] at 50 (47 for R1 Lite). The decline from 20 to 50 demos is clear on R1 Lite only, and the fixed-step control could not separate more demos from more steps. No pooled summary is given, because R1 Lite's n* is censored. The 5 to 20 segment still mixes demos with steps (2k to 3k).

Deviations

• The extension rule fired once: six of the new cells' fourteen (cell, arm) learning-rate picks sat on a grid edge. One extension round was run (validation records only); afterwards every pick was interior and unchanged, so no second round was needed.
• Some jobs were re-run after infrastructure failures (GPU loss), as the registration allows, on the registered code image.
• The registration gave no tool for building extension plans. A small script was added that regenerates each plan with the extra learning rate and every existing job byte-identical; the extension plans are stored beside the originals instead of replacing them.
• The analysis code gained two flag-only changes after registration (a simulation label and a noise-passthrough check). The single report was re-run with the code at the registered commit and every number was identical.
• Worst-case compute caps exceeded the pre-declared extension allowance by under 2%; actual spend stayed within it.

Limits

One task, three robots, parallel grippers, open-loop metric; three seeds; R1 Lite at 47 uses the same demo list for every seed, so its between-seed variance there is understated; the log-linear interpolation of the scratch curve between 20 and 50 demos is a modelling assumption.

Results reported in

XE-0.1 pre-registered follow-ups, second pass 2026-10-07, section 3

Result records

tr:fair:macro:5, tr:fair:macro:50, tr:fair:cobot:demos_to_match, tr:fair:r1lite:demos_to_match, tr:fair:rmc:demos_to_match in release.json

Registration

2026-10-05 08:56:34 UTC, commit f38d3c8 in RC’s internal repository; full text available on request (sha256 3e3aedcd4fa5c6e8…).

Caveat. The 3000-step number stands as registered and now travels with this 10000-step value. Learning rates were not re-tuned for the longer budgets, so this is not the best possible 10000-step comparison.

Scope

robots: Cobot Magic, Galbot G1, R1 Lite; seeds: 3; demos: 20; fine-tune steps: 3000 (reference), 6000, 10000

Question

On fold_clothes the fair macro gain at 20 demos is +11.6% [+9.7%, +13.4%] after the registered 3000 fine-tune steps, but at that budget every scratch model is still on a loss plateau, so part of the gain may be faster optimisation. Does the gain survive when both arms get a longer fine-tune, long enough for scratch to clear the plateau?

Hypothesis

H-rep: the gain is mostly representation; at 10000 steps it stays above zero and is compatible with the 3000-step value. H-opt: the gain is mostly faster optimisation; at 10000 steps its interval contains zero.

Design

Same three robots (Cobot Magic, Galbot G1, R1 Lite), 20 demos, seeds 0 to 2, and the same demo lists, validation and test episodes, recipe and decodes as the 3000-step study; the transfer arm warm-starts from the same pretrain. Only the fine-tune budget changes: 10000 steps (primary) and 6000 (secondary), with the learning-rate schedule spanning the full budget. Each arm keeps the learning rate validation chose at 3000 steps; no learning-rate grid is run at the longer budgets.

Metric

Fair macro gain on open-loop translation error, test split, zero-noise decoding, with the same 95% t-interval (between-seed plus episode-bootstrap variance). The analysis refuses to run unless it reproduces the 3000-step value exactly. A run has cleared the loss plateau when its final end-effector flow loss is below 0.1.

Caveat

The 3000-step number stands as registered and now travels with this 10000-step value. Learning rates were not re-tuned for the longer budgets, so this is not the best possible 10000-step comparison.

Decision rule

• Applied only if the guards pass (no diverged run, arms not identical); otherwise INCONCLUSIVE with no headline. G3k is the 3000-step gain.
• 1. MOSTLY OPTIMISATION: the 10000-step interval's lower bound is at or below 0.
• 2. REPRESENTATION: lower bound above 0 and upper bound at or above G3k.
• 3. PARTIAL: lower bound above 0 and upper bound below G3k: still above zero, but clearly smaller than at 3000 steps.

Stopping rules

Single look at test, after all 72 planned jobs; training logs are read before that only to monitor infrastructure. No run is stopped, dropped, re-run or added because of its result.

Verdict

PARTIAL (rule 3)

Result

+5.8% [+3.3%, +8.3%] at 10000 steps, against +11.6% [+9.7%, +13.4%] at 3000 (reproduced exactly); paired change -5.7 pp [-9.1, -2.4]. At 6000 steps (secondary): +6.7% [+4.9%, +8.6%]. With the longer budgets every scratch run clears the loss plateau (0 of 9 at 3000 steps, 9 of 9 at 6000 and at 10000). Both arms improve or hold with more steps and scratch improves more, so the gain shrinks because scratch catches up, not because transfer gets worse. The gain stays above zero, but part of the 3000-step gain was faster optimisation.

Caveat

The 3000-step number stands as registered and now travels with this 10000-step value. Learning rates were not re-tuned for the longer budgets, so this is not the best possible 10000-step comparison.

Deviations

• Several runs lost their GPU mid-job (a host configuration issue, since fixed). Every relaunch re-ran the same committed plan with the same seeds and skipped finished jobs; no test number was read before all 72 jobs were done.
• The registration allowed a later code build only for relaunches that had orphaned evaluations; the automatic recovery used it for every relaunch, so all Cobot Magic jobs ran on it. Its training and evaluation code paths are the same as the registered build (checked file by file).
• One Galbot G1 transfer fine-tune finished but its evaluation was lost with the GPU; it was retrained from the same plan entry in one clean run and that model was evaluated. Its training-loss log is from the discarded first run.
• A known intermittent trainer crash restarted 10 fine-tunes from step 0 with the same config and seed; the evaluated model is always the final, complete run, and the plateau log was read from that run.
• Compute ran above the registered worst-case estimate because of the GPU-loss relaunches.

Limits

Learning rates not re-tuned at 6000 or 10000 steps; the 10000-step interval has few effective degrees of freedom (three seeds); per-robot values are not monotone in the budget, so read the trend at the macro level only; one task, three robots, grippers, open-loop metric, 30 test episodes per robot.

Results reported in

XE-0.1 fold_clothes loss plateau test, results 2026-10-10

Result records

tr:fair:macro:20:fold:10k, tr:fair:macro:20:fold in release.json

Registration

2026-10-08 16:02:57 UTC, commit cf052e4 in RC’s internal repository; full text available on request (sha256 0a471508b2d22074…).

Caveat. The comparison with XE-0.1 spans two code versions, and the drift of the transfer arm was not measured; a same-code re-run of XE-0.1 would remove the confound and was ruled out of scope on cost. The XE-0.1 gains on this site stay as registered and are labelled XE-0.1.

Scope

robots: Cobot Magic, R1 Lite, RMC-AIDA-L; seeds: 3; demos: 20

Question

XE-0.1 trains its end-effector twist targets in raw units (metres, radians) while its other inputs are normalised. Does normalising the twist targets per robot change the fair transfer gain at 20 demos on stack_baskets, where XE-0.1 gives +8.7% [+5.8%, +11.5%]? The model with this one change is XE-0.2.

Hypothesis

No directional hypothesis. Normalisation may help scratch more than transfer, which would shrink the gap, or help both arms; the absolute errors of both arms are reported, not only the gain.

Design

Identical to XE-0.1 unless listed: the same three held-out robots (Cobot Magic, R1 Lite, RMC-AIDA-L), 20 demos, seeds 0 to 2, demo lists, validation and test episodes. Each robot's twist statistics are fitted on the run's own training split only. Pretrains are re-run with the change; the learning rate is chosen per robot and arm on validation from one grid for all robots, with a pre-declared extension rule. A drift check per robot re-runs one XE-0.1 scratch job on the new code: if it falls outside XE-0.1's seed range, the comparison with XE-0.1 is reported as confounded by code drift, whatever the verdict.

Metric

Primary: the change in fair macro gain, XE-0.2 minus XE-0.1, on open-loop translation error, test split, zero-noise decoding; its 95% interval treats the two macros as independent, which is conservative. Co-reported: the XE-0.2 gain with its own interval, and each arm's error reduction from normalisation.

Caveat

The comparison with XE-0.1 spans two code versions, and the drift of the transfer arm was not measured; a same-code re-run of XE-0.1 would remove the confound and was ruled out of scope on cost. The XE-0.1 gains on this site stay as registered and are labelled XE-0.1.

Decision rule

• Applied only if the XE-0.2 headline guards pass (no grid edge after the extension budget, no diverged or identical arms); otherwise INCONCLUSIVE with no headline.
• 1. NORMALISATION SHRINKS THE TRANSFER GAIN: the interval's upper bound is below 0.
• 2. NORMALISATION ENLARGES THE TRANSFER GAIN: the lower bound is above 0.
• 3. GAIN UNCHANGED: the whole interval lies within ±5 percentage points.
• 4. Otherwise INCONCLUSIVE.
• Adoption (separate): normalisation becomes the default if, for both arms, the lower bound of the error reduction is above -2%.

Stopping rules

Single look at test after all planned jobs; before that only training logs, completion markers and validation records are read. No run is stopped, dropped, re-run or added because of its result. XE-0.1's numbers are not replaced: XE-0.2 is reported next to them.

Verdict

NORMALISATION SHRINKS THE TRANSFER GAIN (rule 1); comparison confounded by code drift (registered label)

Result

Change from XE-0.1: -6.5 pp [-10.3, -2.6]. XE-0.2 fair macro gain: +2.2% [-0.9%, +5.2%], so the transfer gain is not demonstrated under XE-0.2. XE-0.1 reproduced at +8.7% [+5.8%, +11.5%]. Normalisation lowered the error of both arms, transfer by 15.5% [13.1%, 17.8%] and scratch by 21.0% [19.0%, 23.1%]; it lowered scratch more, which is why the gain shrinks. Adoption rule: adopt. Drift check: one of three robots (RMC-AIDA-L) fell 0.24 mm outside XE-0.1's seed range, so the comparison is reported as confounded by code drift, as registered.

Caveat

The comparison with XE-0.1 spans two code versions, and the drift of the transfer arm was not measured; a same-code re-run of XE-0.1 would remove the confound and was ruled out of scope on cost. The XE-0.1 gains on this site stay as registered and are labelled XE-0.1.

Deviations

• Amendment 1 (listed below) dropped one launch-gate condition after the CPU smoke result, before any GPU job; disclosed as post hoc.
• Several runs lost their GPU mid-job; the interrupted jobs had no outputs and re-ran from their plan entries on the same registered build. A known intermittent trainer crash was retried once for seven jobs.
• One recovery run started on a build without the model change; it crashed while reading its configuration, before any training step, and wrote no result. It still breached the registration's rule on recovery builds.
• Compute ran slightly above the expected total because of the relaunches, within the registered worst case.

Limits

One task, three robots, grippers, open-loop metric, three seeds, 30 test episodes per robot; only per-robot mean / standard-deviation scaling of the targets was tested; nothing here says how normalisation affects the second task, which would need its own registration.

Results reported in

XE-0.2 end-effector normalisation, results 2026-10-10

Result records

tr:xe02:macro:20, tr:fair:macro:20 in release.json

Registration

2026-10-08 16:50:02 UTC, commit ef0aff0 in RC’s internal repository; full text available on request (sha256 5a3d6c338971c2dc…).

Later commits

• 2026-10-08 17:00:02 UTC, commit 1d1493b, amendment: Amendment 1, appended after the CPU launch-gate result and before any GPU job: one launch-gate condition (a residual-noise slope measured in each model's own loss units, so it cannot compare the two normalisations) was dropped, disclosed as post hoc. The question, primary outcome, analysis, decision rule and adoption rule are unchanged. (sha256 after it 7a0c2788ae1b38c0…)
• 2026-10-10 00:31:15 UTC, commit f9846a7, results appendix: Results section appended below the registered text after the single analysis, as the registration requires. (sha256 after it 960e0aaf5411a59f…)

Scope

robots: Cobot Magic, RMC-AIDA-L; seeds: 3; demos: 20

Question

At a fixed set of robots and tasks, does about four times more pretraining data help? And at a fixed pretraining budget, is data from the target task recorded on another robot better than data from other tasks, or than a 50/50 mix?

Hypothesis

Three primary contrasts, each tested at 95% without multiplicity correction: Q, 1673 vs 420 pretraining episodes from the same 7 robots and 42 tasks; T1, 100 same-task episodes vs 100 episodes of 10 other tasks; T2, a 50/50 mix vs the other tasks alone.

Design

Targets Cobot Magic and RMC-AIDA-L on stack_baskets, 20 demos, 3000 fine-tune steps, seeds 0 to 2, the demo lists and splits of the earlier sweeps; every pretraining pool excludes both target robots. The 420-episode arm reuses an earlier study's arm whose numbers had already been read (disclosed); everything else was unseen. Learning rate chosen per arm and target on validation.

Metric

Open-loop translation error on the test split, zero-noise decoding; a contrast A vs B is 1 - error(A) / error(B), paired by seed and test episode, averaged over the two targets, with the same 95% t-interval as above.

Decision rule

• 0. REFUSED while a chosen learning rate sits on a grid edge or a chosen cell is diverged or identical.
• 1. A BETTER: the interval's lower bound is above 0.
• 2. A WORSE: the upper bound is below 0.
• 3. NO MATERIAL DIFFERENCE: the whole interval lies within ±4%.
• 4. Otherwise INCONCLUSIVE.

Stopping rules

Single look per contrast after all its runs and the one pre-declared extension round. Every run enters; a bad result is never a reason to re-run.

Verdict

Q: NO MATERIAL DIFFERENCE (rule 3). T1, T2: INCONCLUSIVE (rule 4). T3 (secondary): INCONCLUSIVE.

Result

Q, about 4× more pretraining data (1673 vs 420 episodes, same 7 robots and 42 tasks): +0.8% [−1.5%, +3.0%], inside the ±4% margin: no material difference at this scale, with pretraining compute held fixed; the interval does not rule out a gain below 4%. T1, same task from another robot vs other tasks: −3.0% [−7.4%, +1.3%]. T2, 50/50 mix vs other tasks: −2.4% [−5.7%, +0.8%]. Same-task or mixed pretraining is not shown to beat other-task pretraining: both point estimates are negative and both intervals include 0; neither is "no material difference", because both intervals reach beyond −4%. T3 (secondary), mix vs same task: +0.6% [−3.5%, +4.7%]. Every condition still beats scratch (descriptive): 420 episodes +5.4% [+2.1%, +8.7%], 1673 episodes +6.1% [+3.8%, +8.4%], same task +3.9% [+1.8%, +5.9%], other tasks +6.7% [+3.0%, +10.5%], mix +4.4% [+0.7%, +8.1%]. No multiplicity correction across the three primaries, as registered.

Deviations

• Rule 0 first refused every contrast: on five target-and-arm cells the learning rate chosen on validation sat on the lowest grid point. The one registered extension round (the next learning rate, 1e-4, for exactly those cells, decided from validation records only) then ran. No cell picked 1e-4, every pick stayed where it was, and the final gains and intervals are the pre-extension ones, now with a verdict.
• The registered analysis command computes the contrasts together with the refusal, so the test numbers were seen before the extension round. Which cells were extended was fixed mechanically from validation, so seeing them could not change it.
• For one extended cell (the 420-episode arm on Cobot Magic) the 1e-4 runs already existed from an earlier study and were read rather than re-run; the analysis code was changed to read them, with a test.
• Several runs lost their GPU or hit a known intermittent trainer crash and were relaunched from the same plan with the same seed; three pretrains resumed from their last checkpoint and two fine-tunes restarted from their warm start. A resumed run is not bit-identical to an uninterrupted one; the registration treats these as infrastructure re-runs, so they are listed, not excluded. One fine-tune whose evaluation was lost was retrained from the same plan entry and evaluated once.

Limits

The same-task pool is capped at 100 episodes (only one non-target robot has the task); one task; two target robots; three seeds. Q holds pretraining at 10000 steps, so it asks about more data at fixed compute (each episode is seen about 4× less often), not more data trained to convergence. The quantity contrast (Q) and the task contrasts (T1 to T3) use different budgets and robot sets and are not compared with each other. T uses one source robot and 100 pretraining episodes per arm.

Results reported in

XE-0.1 data-mix study, results 2026-10-10

Result records

tr:datamix:q:20, tr:datamix:t1:20, tr:datamix:t2:20, tr:datamix:t3:20, tr:datamix:q420:scratch:20, tr:datamix:q1680:scratch:20, tr:datamix:same:scratch:20, tr:datamix:cross:scratch:20, tr:datamix:mix:scratch:20 in release.json

Registration

2026-10-08 16:20:55 UTC, commit 3734cf2 in RC’s internal repository; full text available on request (sha256 0243ed6c67b75787…).

Later commits

• 2026-10-08 16:22:32 UTC, commit 3dee581, edit: Two minutes after registration, before any compute: three internal job-queue numbers in the cost table were renumbered because the registered numbers were already taken. No design, metric or rule changed. (sha256 after it 9b37ece1fc5f034a…)

Scope

robots: Cobot Magic, R1 Lite, RMC-AIDA-L; seeds: 3; demos: 20

Question

How does a publicly available policy, fine-tuned under the identical protocol, compare with XE-0.1's transfer and scratch arms on the same cells? Results are published as "Baseline (RC-run)": RC ran them, they are not submissions by their authors and are never ranked.

Hypothesis

H1 (primary): Octo-Base is worse than XE-0.1 transfer on all three robots. H2: Octo-Base is better than XE-0.1 scratch. H3: Octo-Small vs Octo-Base, descriptive only. Expectations stated in advance so they can be wrong.

Design

Held-out Cobot Magic, R1 Lite and RMC-AIDA-L on stack_baskets, 20 demos, seeds 0 to 2, 3000 fine-tune steps; demo lists, validation and test episodes copied from XE-0.1. Fine-tuned from the authors' MIT-licensed checkpoints with the authors' recipe for a new action space; learning rate chosen per model and robot on validation, with a pre-declared extension rule. XE-0.1's arms are existing records, not re-run.

Metric

Open-loop translation error on the test split, scored by the XE-0.1 evaluator; gain = 1 - error(Octo) / error(XE-0.1 arm), per robot, with the same 95% t-interval as above.

Decision rule

1. Per robot: SUPPORTED if the interval lies entirely on the expected side, CONTRADICTED if entirely on the other side, otherwise INCONCLUSIVE.
2. A cell with a blocking flag is reported with no verdict; no macro verdict.
3. A model whose grid does not complete is INCOMPLETE.

Stopping rules

Single look at test after every planned job of a model; an on-machine smoke gate before the grid; infrastructure failures are relaunched from the same plan.

Verdict

INCOMPLETE (stopping rule): no verdict on H1, H2 or H3

Result

The compute ceiling was reached before any model's grid finished, so under the registered stopping rules every model on every robot is INCOMPLETE and H1, H2 and H3 get no verdict. Octo-Base finished 8 of 12 fine-tunes with their evaluations on Cobot Magic, 7 of 12 on R1 Lite and 6 of 12 on RMC-AIDA-L; Octo-Small never started, because the registration runs the whole Octo-Base grid first. No learning rate was selected and no Octo-vs-XE-0.1 gain or interval was computed. The finished grid points are listed in RC's results document as descriptive values only; they carry no pre-registered weight, are not a comparison with XE-0.1 and are not shown here. A new, separately registered study re-runs the comparison with fresh runs.

Deviations

• Three infrastructure fixes to the on-machine smoke gate, with no protocol change (dated in the registration before any grid job finished).
• Each step's registered batch of 64 ran as 4 micro-batches of 16 by gradient accumulation, because the full batch did not fit in GPU memory; the step's gradient equals the full batch's, but dropout draws differ, so runs are not bit-identical to one batch of 64.
• Per-robot compute caps were set to keep the total within the registered budget. A fine-tune with its evaluation took about 1.05 hours, against an unmeasured estimate of about 28 minutes in the registration; that, not the caps, is why the grid did not finish.

Limits

One task, three robots, grippers, open-loop metric, three seeds; the open models never saw these robots or bimanual control in pretraining.

Results reported in

Open-model baselines, results 2026-10-10 (INCOMPLETE)

Registration

2026-10-08 16:24:47 UTC, commit d4f572f in RC’s internal repository; full text available on request (sha256 cd909652d2a2026c…).

Later commits

• 2026-10-10 05:19:10 UTC, commit 53acccb, amendment: Dated deviations, written before any grid job finished and before any validation or test record was read: three infrastructure fixes to the on-machine smoke gate, with no protocol change; each step's registered batch of 64 is split into 4 micro-batches of 16 because the full batch did not fit in GPU memory, which gives the same expected gradient (dropout draws differ, so runs are not bit-identical to one batch of 64); and per-robot compute caps that keep the total within the registered budget, with a model reported INCOMPLETE for a robot whose grid cannot finish within its cap. (sha256 after it 907f6c140cb8eb4b…)

Scope

robots: Cobot Magic, R1 Lite, RMC-AIDA-L; seeds: 3; demos: 20

Question

Under XE-0.1's fair protocol on the same cells, does a fine-tuned Octo-Base have lower or higher open-loop error than XE-0.1 transfer (primary) and XE-0.1 scratch (secondary)? Published as "Baseline (RC-run)": RC fine-tunes and scores Octo, not its authors; never ranked.

Hypothesis

No directional hypothesis is registered: 21 Octo-Base grid points from the incomplete open-baselines study had already been seen, validation and test, so this study is not blind. The comparison, learning-rate selection, verdict rules and margin are fixed in the registration, and every run is fresh.

Design

Octo-Base 1.5 fine-tuned per held-out robot (Cobot Magic, R1 Lite, RMC-AIDA-L) on stack_baskets with 20 demos, 3000 steps, seeds 0 to 2 and XE-0.1's demo lists, validation and test episodes; learning-rate grid 1e-5 to 3e-4, chosen on validation only, with one pre-declared extension round at a grid edge. All 36 fine-tunes are new runs with fresh random draws; none of the 21 earlier runs enters the analysis. XE-0.1's arms are existing records at their own validation-chosen learning rates. Octo-Small is out of scope.

Metric

Open-loop translation error on the test split, zero-noise decoding; G = 1 - error(Octo-Base) / error(XE-0.1 arm), paired by seed and test episode, per robot and macro (mean of the three), with the same 95% t-interval as above; no multiplicity correction.

Decision rule

• 0. REFUSED (no verdict) if a chosen learning rate is at its grid edge after the extension round, or a cell is diverged or identical; a macro is refused if any robot is.
• 1. OCTO BETTER: the interval's lower bound is above 0.
• 2. XE BETTER: the upper bound is below 0.
• 3. NO MATERIAL DIFFERENCE: the whole interval lies within ±4%.
• 4. Otherwise INCONCLUSIVE.
• 5. INCOMPLETE (no verdict): a robot whose grid, with its extension, is not complete; the macro needs all three.

Stopping rules

Single look at test after every main and extension job is done or the compute cap is reached; before that only validation and smoke records are read. No outcome-dependent stopping or re-running; infrastructure failures are relaunched from the same plan, skipping finished jobs.

Verdict

OCTO BETTER (rule 1), macro vs XE-0.1 transfer

Result

Primary, Octo-Base vs XE-0.1 transfer, macro over the three robots: +7.0% [+3.5%, +10.4%], OCTO BETTER: fine-tuned Octo-Base has about 7% lower test error than XE-0.1's transfer arm on this protocol. Per robot: Cobot Magic +4.1% [−1.5%, +9.6%] and R1 Lite +6.6% [−1.2%, +14.5%], both INCONCLUSIVE; RMC-AIDA-L +10.2% [+4.2%, +16.3%], OCTO BETTER. Secondary, vs XE-0.1 scratch: macro +15.0% [+10.9%, +19.1%], OCTO BETTER; Cobot Magic +7.6% [−0.6%, +15.7%] INCONCLUSIVE, R1 Lite +21.3% [+13.7%, +28.8%] and RMC-AIDA-L +16.2% [+8.0%, +24.3%] OCTO BETTER. The scratch comparison is not evidence that other robots' data helps: Octo-Base differs from XE-0.1 scratch in architecture, size and recipe; only XE-0.1's own transfer-vs-scratch result isolates pretraining, and the Octo number is at most consistent with it. Rule 0 did not fire: Octo-Base chose 1e-4 on all three robots on validation, interior to its grid, so no extension ran; XE-0.1's choices are interior; no cell is diverged or identical. No multiplicity correction across the per-robot and secondary contrasts, as registered. Gain = 1 − error(Octo-Base) / error(XE-0.1 arm); positive means Octo-Base has the lower error.

Deviations

• The plan amendment before any job ran (listed below): the first run plans carried a label the job runner refuses, so the machines stopped at start-up with nothing run; jobs were byte-identical after the fix and no record was written.
• Not blind, as registered: 21 Octo-Base grid points from the incomplete open-baselines study had been seen. None was reused; every run here is a fresh random draw (training seed 1000 + seed).
• Recipe, as registered: the Octo authors' new-observation recipe at XE-0.1's demo and step budget, with each step's batch of 64 run as 4 micro-batches of 16 (one dropout draw per micro-batch).
• Automatic recovery after a GPU loss was disabled for this study; no GPU loss occurred.

Limits

Not blind (21 grid points seen before registration); Octo's authors' recipe at XE-0.1's 3000-step budget, not necessarily Octo's best fine-tune; one task, three robots, grippers, open-loop metric; XE-0.1's arms were trained once, earlier, and are paired by demo list and episode, not by training run. It does not show closed-loop task success, other tasks, other budgets, or Octo at its best possible fine-tune.

Results reported in

Octo-Base vs XE-0.1 fair comparison, results 2026-10-11 UTC, commit 975939b

Result records

tr:octo:vs_transfer:macro:20, tr:octo:vs_transfer:cobot:20, tr:octo:vs_transfer:r1lite:20, tr:octo:vs_transfer:rmc:20, tr:octo:vs_scratch:macro:20, tr:octo:vs_scratch:cobot:20, tr:octo:vs_scratch:r1lite:20, tr:octo:vs_scratch:rmc:20 in release.json

Registration

2026-10-10 16:55:28 UTC, commit 4cc0547 in RC’s internal repository; full text available on request (sha256 d3a8565b617c68e1…).

Later commits

• 2026-10-10 17:11:12 UTC, commit a3ac91c, edit: Before any job ran and before any record of this study existed: the registered run plans carried a study label the job runner refuses, so every machine stopped at start-up with nothing run. The plans now keep the runner's label and name the study separately; jobs, seeds, learning rates, demo lists and episodes are unchanged, and the old and new plan checksums are both listed in the registration. (sha256 after it 504eaf83b82be910…)