Gallery › Vibe Tracks › ML experiment tracking
ML experiment tracking
Runs as rows, the way an experiment tracker keeps them: each run's model and parameters (environment, lane, bench
version, round) and its metrics (attempts, successes, compute, success rate). A leaderboard pivot, the metric
against the step per environment, one KPI card per model with its pooled success rate, and the runs table, all from
one .base file. Illustrative data: Vibe Tracks' synthetic grasp bench, made-up models.
The .base file and its data
Every tab above is one entry of this file; the views read these rows. ⋯ in the toolbar copies the current view as YAML.
# Experiment tracking: one row per run (model, params, metrics), the views Vibe Tracks' Results tab and its
# leaderboards draw by hand. ILLUSTRATIVE data: Vibe Tracks' synthetic grasp bench (made-up models).
# Check it: node bin/data-ui.mjs check <this file>
data: runs.json
facets:
- model
- env
- lane
- bench
- { property: round, as: range }
properties:
run: { displayName: Run }
round: { displayName: Round }
started: { displayName: Started }
model: { displayName: Model }
family: { displayName: Family }
env: { displayName: Environment }
tier: { displayName: Tier }
lane: { displayName: Lane }
bench: { displayName: Bench }
attempts: { displayName: Attempts }
successes: { displayName: Successes }
compute_s: { displayName: Compute }
success_rate: { displayName: Success rate }
compute_mean_s: { displayName: Compute per grasp }
sppm: { displayName: Picks per minute }
views:
- type: pivot
name: Leaderboard
order: [model]
splitBy: [env]
value: { property: success_rate, summary: Median, display: bar }
sort:
- { property: model, direction: ASC }
- type: chart
name: Success rate by round
chart:
mark: { type: line, point: true }
encoding:
x: { field: round, title: Round (step) }
y: { field: success_rate, axis: { format: '%' }, scale: { zero: false } }
color: { field: model }
facet: { field: env, columns: 3 }
guides:
- { rule: { y: 0.55 }, label: "gate 55 %" }
hover: { title: run, fields: [success_rate, model, env, round, attempts, compute_mean_s] }
- type: kpi
name: Pooled success by model
groupBy: { property: model, direction: ASC }
sort:
- { property: round, direction: ASC }
kpi:
value: { property: successes, summary: Ratio, per: attempts }
spark: { x: round, y: success_rate }
gate: { min: 0.55 }
- type: chart
name: Compute vs success
chart:
mark: { type: point, filled: true }
encoding:
x: { field: compute_mean_s }
y: { field: success_rate, axis: { format: '%' } }
color: { field: model }
size: { field: attempts }
hover: { title: run, fields: [success_rate, compute_mean_s, model, env, round] }
- type: table
name: Runs
order: [run, round, model, env, lane, bench, attempts, successes, success_rate, compute_mean_s, sppm]
groupBy: { property: model, direction: ASC }
sort:
- { property: success_rate, direction: DESC }
display: { success_rate: bar }
fold: closed
- type: table
name: Runs by week
order: [run, started, model, env, attempts, successes, success_rate]
groupBy: { property: started, by: week }
sort:
- { property: started, direction: ASC }
display: { success_rate: bar }
- type: chart
name: Runs per week
chart:
mark: bar
encoding:
x: { field: started, by: week, title: Week }
y: { aggregate: count, title: Runs }
color: { field: model }
hover: { title: run, fields: [started, model, env, success_rate] }
// runs.json: 120 rows, 37 KB; the first 3
{
"id": "ml-experiments",
"title": "Grasp model runs (synthetic, illustrative)",
"titleProperty": "run",
"source": "vibetracks tests/fixtures/kpi/synthetic/runs_table.jsonl (vt-run/1, synthetic: made-up models)",
"properties": {
"run": {
"type": "text",
"width": 190
},
"round": {
"type": "number",
"width": 64,
"note": "the run's round, r01 -> 1: the step"
},
"started": {
"type": "date",
"width": 190
},
"model": {
"type": "text",
"width": 120,
"display": "badge"
},
"family": {
"type": "text",
"width": 70
},
"env": {
"type": "text",
"width": 160
},
"tier": {
"type": "text",
"width": 60
},
"lane": {
"type": "text",
"width": 80
},
"bench": {
"type": "text",
"width": 60
},
"attempts": {
"type": "number",
"width": 80
},
"successes": {
"type": "number",
"width": 90
},
"compute_s": {
"type": "number",
"unitText": "s",
"width": 100
},
"success_rate": {
"type": "number",
"decimals": 3,
"width": 110,
"derived": "successes / attempts"
},
"compute_mean_s": {
"type": "number",
"unitText": "s",
"width": 120,
"derived": "compute_s / attempts"
},
"sppm": {
"type": "number",
"unitText": "picks/min",
"width": 120,
"derived": "60 x success_rate / compute_mean_s"
}
},
"rows": [
{
"id": "r01-graspnet1b-seen",
"run": "r01-graspnet1b-seen",
"round": 1,
"started": "2026-09-23T08:48:00-07:00",
"model": "M1.base",
"family": "M1",
"env": "graspnet1b/seen",
"tier": "full",
"lane": "models",
"bench": "v1",
"attempts": 12,
"successes": 6,
"compute_s": 31.734,
"success_rate": 0.5,
"compute_mean_s": 2.6445,
"sppm": 11.344
},
{
"id": "r01-graspnet1b-novel",
"run": "r01-graspnet1b-novel",
"round": 1,
"started": "2026-09-23T08:52:00-07:00",
"model": "M1.base",
"family": "M1",
"env": "graspnet1b/novel",
"tier": "full",
"lane": "models",
"bench": "v1",
"attempts": 11,
"successes": 5,
"compute_s": 29.377,
"success_rate": 0.4545,
"compute_mean_s": 2.6706,
"sppm": 10.212
},
{
"id": "r01-clutter6d-hard",
"run": "r01-clutter6d-hard",
"round": 1,
"started": "2026-09-23T08:56:00-07:00",
"model": "M1.base",
"family": "M1",
"env": "clutter6d/hard",
"tier": "full",
"lane": "models",
"bench": "v1",
"attempts": 14,
"successes": 5,
"compute_s": 36.751,
"success_rate": 0.3571,
"compute_mean_s": 2.6251,
"sppm": 8.163
}
]
}README.md
# ML experiment tracking
Runs as rows, the way an experiment tracker keeps them: each run's model and parameters (environment, lane, bench
version, round) and its metrics (attempts, successes, compute, success rate). A leaderboard pivot, the metric
against the step per environment, one KPI card per model with its pooled success rate, and the runs table, all from
one `.base` file. Illustrative data: Vibe Tracks' synthetic grasp bench, made-up models.
- **App:** Vibe Tracks
- **Replaces:** Vibe Tracks' hand-rolled Results tab (the Perspective-based runs pivot and its leaderboard) and the
per-experiment metric charts its tracks draw by hand; the same shape as W&B's or MLflow's runs table and charts.
- **Shows:** `type: pivot` model x environment with the median success rate as bars; a `chart` of success rate by
round (the step), coloured by model, small multiples per environment, with the 55 % gate as a rule; `type: kpi`
grouped by model with `summary: Ratio` (Σ successes / Σ attempts, pooled, not a mean of rates), a sparkline per
model and the gate held or missed; compute vs success sized by attempts; the runs table grouped by model; the
runs grouped by ISO week (`groupBy: {property: started, by: week}`) and a bar chart of runs per week
(`x: {field: started, by: week}`).
- **Data:** `runs.json`, written by `source/build.py` from Vibe Tracks' committed fixture
`runs_table.jsonl` (in its tests/fixtures/kpi/synthetic/, 120 runs). ILLUSTRATIVE: its models and numbers are made up for
Vibe Tracks' tests; no real run log with per-run metrics is committed in Vibe Tracks or bam_ws. Success rate,
compute per grasp and picks per minute are derived per run and marked so.
The original UI it replaces
Vibe Tracks' hand-rolled Results tab (the Perspective-based runs pivot and its leaderboard) and the per-experiment metric charts its tracks draw by hand; the same shape as W&B's or MLflow's runs table and charts.
What it shows
type: pivot model x environment with the median success rate as bars; a chart of success rate by round (the step), coloured by model, small multiples per environment, with the 55 % gate as a rule; type: kpi grouped by model with summary: Ratio (Σ successes / Σ attempts, pooled, not a mean of rates), a sparkline per model and the gate held or missed; compute vs success sized by attempts; the runs table grouped by model; the runs grouped by ISO week (groupBy: {property: started, by: week}) and a bar chart of runs per week (x: {field: started, by: week}).
Data
runs.json, written by source/build.py from Vibe Tracks' committed fixture runs_table.jsonl (in its tests/fixtures/kpi/synthetic/, 120 runs). ILLUSTRATIVE: its models and numbers are made up for Vibe Tracks' tests; no real run log with per-run metrics is committed in Vibe Tracks or bam_ws. Success rate, compute per grasp and picks per minute are derived per run and marked so.