AA-Robotics
Artificial Analysis research preview: language models in a coding-agent harness directly control a 6-axis robot arm to complete simple tabletop tasks.
Research preview v0.1- Models tested
- 5
- Best score
- 69.4
- GPT-6 Astra
- Cheapest per task
- $1.93
- GPT-6.1 Sol · 64.6
- Human, per task
- 20 s
- $0.211 at the average US hourly wage
Score by cost or time
One attempt per task and model. The horizontal axis is reversed: towards the top right, a model scores more for less money or less time.
Models shown5 of 5
The best models are shown by default. Hover a model to bring it forward; the axes fit the models shown.
Passed: 5 of 8Cost / task: $16.36Time / task: 30 min 16 sTo first action: 4 min 38 s
Passed: 5 of 8Cost / task: $1.93Time / task: 28 min 05 sTo first action: 5 min 56 s
Passed: 4 of 8Cost / task: $7.61Time / task: 52 min 24 sTo first action: 21 min 31 s
Passed: 3 of 8Cost / task: $6.75Time / task: 31 min 40 sTo first action: 7 min 17 s
Passed: 3 of 8Cost / task: $9.73Time / task: 31 min 09 sTo first action: 11 min 35 s
| # | Model | Harness | Score | Passed | Cost / task | Time / task | To first action |
|---|---|---|---|---|---|---|---|
GPT-6 Astra OpenAI | Codex | 69.4 | 5 of 8 | $16.36 | 30 min 16 s | 4 min 38 s | |
| 2 | GPT-6.1 Sol OpenAI | Codex | 64.6 | 5 of 8 | $1.93 | 28 min 05 s | 5 min 56 s |
| 3 | Gemini 4 Argon | Antigravity | 55.3 | 4 of 8 | $7.61 | 52 min 24 s | 21 min 31 s |
| 4 | Claude Opus 5.5 Anthropic | Claude Code | 48.8 | 3 of 8 | $6.75 | 31 min 40 s | 7 min 17 s |
| 5 | Claude Fable 5.1 Anthropic | Claude Code | 46.3 | 3 of 8 | $9.73 | 31 min 09 s | 11 min 35 s |
Results per task
Each model's score on each task, out of 100.
| Task | GPT-6 Astra | GPT-6.1 Sol | Gemini 4 Argon | Claude Opus 5.5 | Claude Fable 5.1 |
|---|---|---|---|---|---|
| Press ButtonPress the button with the gripper. | 100 | 100 | 100 | 100 | 100 |
| Place CrocodilePlace the crocodile inside the rectangle. | 100 | 100 | 100 | 100 | 100 |
| Pour DrinkPour the drink from the bottle into the cup. | 100 | 0 | 100 | 25 | 30 |
| Tidy BoxPut all four blocks in the box, then put the lid back on. | 40 | 100 | 40 | 40 | 40 |
| Match ShapesPlace the blocks into the board so that it matches the picture. | 44 | 100 | 0 | 38 | 100 |
| Write Hello WorldWrite “hello world” on the paper with the marker, inside the black box. | 100 | 100 | 45 | 45 | 0 |
| Build TowerMake the tallest tower out of the foam blocks. | 71 | 0 | 57 | 25 | 0 |
| Measure BoxMeasure the length, width, and height of the box. | 0 | 17 | 0 | 17 | 0 |
The tasks on video
A short clip of each task published by Artificial Analysis, with its instructions and pass criteria.
Press the red button
- Goal ·
- Press the button with the gripper.
- Pass criteria ·
- The button flashes.
Place the crocodile toy in the rectangle
- Goal ·
- Place the crocodile inside the rectangle.
- Pass criteria ·
- The crocodile is entirely within the rectangle.
Pour the drink into the cup
- Goal ·
- Pour the drink from the bottle into the cup.
- Pass criteria ·
- Drink is in the cup, none is spilled, and nothing is broken.
Tidy the blocks into the box
- Goal ·
- Put all four blocks in the box, then put the lid back on.
- Pass criteria ·
- All blocks are inside the box with the lid flush.
Complete the shape matching challenge
- Goal ·
- Place the blocks into the board so that it matches the picture.
- Pass criteria ·
- Blocks in the board match the picture.
Write hello world
- Goal ·
- Write “hello world” on the paper with the marker, inside the black box.
- Pass criteria ·
- “hello world” is legible and inside the black box.
Build the tallest tower
- Goal ·
- Make the tallest tower out of the foam blocks.
- Pass criteria ·
- Tower is freestanding and max achievable height.
Measure the box
- Goal ·
- Measure the length, width, and height of the box.
- Pass criteria ·
- Each dimension is within 2 mm of the correct size.
Explore the runs
The full video of every run. The histogram shows the arm's activity; the dark band, the moments it moves. Click it to jump there.
- First action · 3:16
- First passing state · 3:23
- Arm moving
GPT-6 Astra · Codex
Press the red button
Score: 100 / 100
Pass criteria · The button flashes.
Mistakes on video
Clips Artificial Analysis picked to show the models' mistakes, captioned in English on screen.
Tests pen contact on the page itself, leaving marks inside the writing box
Claude Opus 5.5 · Write Hello World · at 21:40 into the run
Poor physical reasoning
Each attempt to drag the box back pushes it further toward the corner of the table
Claude Opus 5.5 · Tidy Box · at 36:10 into the run
Poor physical reasoning
Slides the gripper along the table into the crocodile and pushes it about 10 cm away, out of reach
Claude Opus 5.5 · Place Crocodile · at 2:54 into the run
Poor physical reasoning
Finds the paper by making ink dashes inside the writing box
Claude Fable 5.1 · Write Hello World · at 8:46 into the run
Poor physical reasoning
Two re-approaches push a crocodile that was fully inside the rectangle back over the far edge
Claude Opus 5.5 · Place Crocodile · at 35:30 into the run
Poor physical reasoning
The wrist presses on the tray and pushes the whole tray forward, which makes each grasp on the bottle harder
Claude Opus 5.5 · Pour Drink · at 5:04 into the run
Poor physical reasoning
Table-contact probes shove four loose blocks into one cluster
Claude Opus 5.5 · Match Shapes · at 20:57 into the run
Poor physical reasoning
Handles the blocks for 13 minutes before removing the lid
Claude Opus 5.5 · Tidy Box · at 10:00 into the run
Poor physical reasoning
Releases the wedge from well above the rim, and it lands on top of a cube
Claude Fable 5.1 · Tidy Box · at 18:10 into the run
Poor physical reasoning
9 of 61 clips
Method and versions
Each model controls a 6-axis robot arm (5 joints and a gripper) through a simple HTTP API, sees the scene through four cameras, and can write and run code to plan its movements.
- Tabletop tasks modelled on everyday activities: grasping precision, spatial reasoning and adapting to the unexpected.
- A human judge grades the end state against the task's criteria: 100 for a pass, partial points otherwise.
- Each model runs in its creator's coding-agent harness (Codex, Claude Code, Antigravity…) at the high effort setting; the Harness column names it.
- One zero-shot attempt per task: the model stops when it judges the task complete or gives up.
- The human baseline is the by-hand task time, priced at the average US hourly wage Artificial Analysis cites.
Data via Artificial Analysis' AA-Robotics article. Score, cost and time are the published averages over one attempt per task.