# Contribute a XEbench run

1. Run the complete task protocol on real hardware: 20 trials for arrow-key or piano sequences; 30 trials for pick and place. Keep unsuccessful trials. Mark exclusions only for documented invalid recordings or hardware conditions.
2. Keep the `results.json` produced by XEbench. It must explicitly declare `mock: false` and `protocol_complete: true`, include the required full-round count, and retain each trial's outcome and task events.
3. Make the complete run recordings inspectable at a public HTTPS URL. DexData, Hugging Face, a public video playlist, or a lab-hosted page can be used. Include failures and trial identifiers so the results can be matched to recordings. MCAP provides synchronized video and telemetry but is not required. A successful highlight clip is insufficient.
4. On /benchmark, enter the hardware, policy name, exact checkpoint / model revision, public recordings URL, and upload the results file. For an LLM controller such as GPT-6 Astra, Gemini, or Claude, also identify the provider and document the prompt, observation format, action adapter, and control frequency with the run artifacts. A policy family name alone is not an exact revision.
5. For a custom task, supply its task ID, name, protocol version, expected full-round trial count, and public HTTPS specification.
6. Review the publication consent. Contact information is optional and is stored separately from public results when online publishing is configured.

## Service availability

When online publishing is connected, the server validates the run and recomputes aggregate scores. Community results remain self-reported and do not populate the RC-verified matrix.

When publishing is not connected, the form validates locally and downloads `xebench-submission.json`. Nothing is uploaded or published. Keep the file and recordings for submission when publishing opens. The downloaded file includes any contact information you opted to provide; do not use this envelope as a public results file without removing private contact fields.

## Public scores

Choose Published runs to view submitted measurements. Preview data shows example table values and is never a submission or RC release.

The leaderboard shows success rate, a 95% Wilson interval for independent trials, trial counts and exclusions, and recordings. It orders measured rows by success rate within a selected task/protocol, with ties sharing rank. Task-specific accuracy is still recalculated by the submission service. Full-run duration and human-relative throughput appear only when their required timing records exist. Completion time over successful or timed trials alone is not HRT. Hardware, task, protocol, data, and training conditions must match for a meaningful comparison.

Keyboard and piano accuracy score sequence positions, with extra events counted as errors. Piano medium-velocity rate uses observed MIDI velocities 50–90. Pick and place reports success by object. An RC release requires a separate, reviewed cohort with pinned checkpoints/adapters and sufficient valid trials. The proposed minimum is 40 valid trials per cell; community runs need only complete their fixed round.

## Human teleoperation baseline

Human-relative throughput is `100 × (policy successful completions / policy elapsed time) / (human successful completions / human elapsed time)`. The human teleoperates the same arm + hand for pick and place, or the same mounted hand for hand-only tasks. Use the same timing rules, objects, task mix, and observation setup. Timeouts, failed attempts, recovery, and between-trial resets count toward elapsed time. The current submission API lacks full-run timing and a matched baseline, so this metric requires additional reviewed records.

## API compatibility

The form retains the `dexbench.submission.v1` envelope. The legacy field `artifacts.mcap_viewer_url` now stores the public run recordings URL regardless of host or format. `results_json_url` is an optional public HTTPS result artifact. The website sends only scoring fields from the uploaded file, not session paths or local endpoints.

Operators must apply both database migrations before enabling online submissions. The second migration permits external recording hosts and requires the policy revision. Existing community rows are preserved.