← mecheval / task / a2-stepped-block-01

Block with rectangular step A · A2 · a2-stepped-block-01

block · step · boolean · difference

Expected

Prompt

Start with a rectangular block 50mm long (X) × 30mm wide (Y) × 20mm tall (Z), centered in X and Y with the bottom face on the XY plane (so it spans x in [-25, 25], y in [-15, 15], z in [0, 20]). Cut away the upper +X corner: remove material in the region x in [0, 25], y in [-15, 15], z in [10, 20]. The result has an L-shape when viewed from +Y. Output a single solid.

Checks

0
valid_solid
{
  "type": "valid_solid"
}
1
bbox
{
  "type": "bbox",
  "min": [
    -25,
    -15,
    0
  ],
  "max": [
    25,
    15,
    20
  ],
  "tolerance_mm": 0.05
}
2
mass_props
{
  "type": "mass_props",
  "volume_mm3": 22500,
  "tolerance_pct": 0.1
}
3
step_roundtrip
{
  "type": "step_roundtrip",
  "tolerance_pct": 0.1
}

Anti-cheese

{
  "max_solid_count": 1
}

Limits

{
  "max_tokens": 30000,
  "max_wallclock_sec": 180,
  "max_tool_calls": 30
}

Recent attempts

Runs (35)

modelrun statusscorefirst failtokenswall
claude-mcp-claude-opus-4-7 20260611T175636Z-d79a PASS 1.00 264.4k 50.4s
claude-mcp-claude-opus-4-7 20260611T175628Z-5603 PASS 1.00 244.7k 83.7s
claude-mcp-claude-opus-4-7 20260611T175519Z-9637 PASS 1.00 357.7k 68.1s
claude-mcp-claude-opus-4-7 20260611T175518Z-61e3 PASS 1.00 368.5k 131.4s
claude-mcp-claude-opus-4-7 20260611T175418Z-6f36 PASS 1.00 312.9k 59.8s
openai-direct-gpt-5 20260428T215353Z-9f7c PASS 1.00 1.4k 8.7s
openai-direct-gpt-5 20260428T215341Z-d935 PASS 1.00 1.7k 12.0s
openai-direct-gpt-5 20260428T215331Z-f073 PASS 1.00 1.5k 9.6s
openai-direct-gpt-5 20260428T215312Z-6765 PASS 1.00 2.3k 18.6s
openai-direct-gpt-5 20260428T215259Z-4dd1 PASS 1.00 1.9k 13.3s
openai-direct-gpt-5-mini 20260428T215050Z-ad1a PASS 1.00 1.9k 11.5s
openai-direct-gpt-5-mini 20260428T215041Z-c90d PASS 1.00 1.7k 9.3s
openai-direct-gpt-5-mini 20260428T215030Z-538f PASS 1.00 1.9k 10.8s
openai-direct-gpt-5-mini 20260428T215019Z-cc5d PASS 1.00 1.7k 10.8s
openai-direct-gpt-5-mini 20260428T215009Z-e754 PASS 1.00 1.7k 9.9s
openai-direct-gpt-4o-mini 20260428T213644Z-bd9a fail 0.75 bbox · X off by +25.00mm 972 4.7s
openai-direct-gpt-4o-mini 20260428T213639Z-3b25 fail 0.00 valid_solid · solid invalid 970 5.5s
openai-direct-gpt-4o-mini 20260428T213633Z-c6c9 fail 0.00 valid_solid · solid invalid 941 5.6s
openai-direct-gpt-4o-mini 20260428T213628Z-ff20 fail 0.00 valid_solid · solid invalid 934 5.1s
openai-direct-gpt-4o-mini 20260428T213622Z-8108 fail 0.00 valid_solid · solid invalid 936 5.4s
claude-direct-claude-sonnet-4-6 20260428T212208Z-d16a PASS 1.00 1.1k 4.5s
claude-direct-claude-sonnet-4-6 20260428T212203Z-f259 PASS 1.00 1.1k 4.8s
claude-direct-claude-sonnet-4-6 20260428T212159Z-73e0 PASS 1.00 1.0k 4.0s
claude-direct-claude-sonnet-4-6 20260428T212154Z-84fc PASS 1.00 1.1k 4.5s
claude-direct-claude-sonnet-4-6 20260428T212150Z-da5a PASS 1.00 1.1k 4.0s
claude-direct-claude-opus-4-7 20260428T212027Z-c240 PASS 1.00 1.2k 4.2s
claude-direct-claude-opus-4-7 20260428T212023Z-5717 PASS 1.00 1.3k 4.2s
claude-direct-claude-opus-4-7 20260428T212019Z-9a3f PASS 1.00 1.3k 4.0s
claude-direct-claude-opus-4-7 20260428T212015Z-22d6 PASS 1.00 1.3k 4.1s
claude-direct-claude-opus-4-7 20260428T212011Z-aa4a PASS 1.00 1.3k 4.1s
claude-direct-claude-haiku-4-5-20251001 20260428T211846Z-a13c PASS 1.00 1.1k 2.5s
claude-direct-claude-haiku-4-5-20251001 20260428T211843Z-e7bf fail 0.00 valid_solid · solid invalid 1.2k 2.8s
claude-direct-claude-haiku-4-5-20251001 20260428T211841Z-e0bd fail 0.00 valid_solid · solid invalid 1.1k 2.8s
claude-direct-claude-haiku-4-5-20251001 20260428T211838Z-0d90 fail 0.00 valid_solid · solid invalid 1.1k 2.3s
claude-direct-claude-haiku-4-5-20251001 20260428T211835Z-6f1a fail 0.00 valid_solid · solid invalid 1.1k 2.7s

generated 2026-06-17T03:16:07.219Z · static site, regenerate with npm run build -w @mecheval/leaderboard