← mecheval / model / openai-direct-gpt-5-mini

openai-direct-gpt-5-mini

133 run blobs across 26 tasks

All runs

taskrun statusscorefirst failtokenswall
a3-spherical-dome-block-01 20260429T002607Z-248c PASS 1.00 — 2.5k 35.4s
a3-spherical-dome-block-01 20260429T002535Z-dca3 PASS 1.00 — 2.4k 32.5s
a3-spherical-dome-block-01 20260429T002503Z-ab62 PASS 1.00 — 2.3k 31.5s
a3-spherical-dome-block-01 20260429T002444Z-4a7a PASS 1.00 — 2.5k 19.5s
a3-spherical-dome-block-01 20260429T002409Z-9ba5 fail 0.50 bbox · Z off by +15.00mm 2.5k 34.6s
a3-three-tangent-cylinders-01 20260429T002329Z-ebd3 PASS 1.00 — 2.5k 40.0s
a3-three-tangent-cylinders-01 20260429T002253Z-8404 PASS 1.00 — 2.8k 35.6s
a3-three-tangent-cylinders-01 20260429T002213Z-aa7b PASS 1.00 — 2.6k 39.9s
a3-three-tangent-cylinders-01 20260429T002132Z-19a6 PASS 1.00 — 2.6k 41.1s
a3-three-tangent-cylinders-01 20260429T002041Z-f292 PASS 1.00 — 3.0k 50.7s
a3-hex-bolt-pattern-01 20260429T001937Z-27fe PASS 1.00 — 4.0k 63.8s
a3-hex-bolt-pattern-01 20260429T001848Z-d8b5 PASS 1.00 — 3.2k 48.3s
a3-hex-bolt-pattern-01 20260429T001738Z-51f5 PASS 1.00 — 4.3k 69.7s
a3-hex-bolt-pattern-01 20260429T001637Z-d3a3 PASS 1.00 — 3.6k 61.3s
a3-hex-bolt-pattern-01 20260429T001548Z-ba39 PASS 1.00 — 3.1k 48.0s
a3-cross-shaft-01 20260429T001508Z-7980 fail 0.75 step_roundtrip · STEP drift on 1/1 solid 2.6k 39.8s
a3-cross-shaft-01 20260429T001423Z-1f2b fail 0.75 step_roundtrip · STEP drift on 1/1 solid 2.4k 45.2s
a3-cross-shaft-01 20260429T001336Z-3b34 fail 0.75 step_roundtrip · STEP drift on 1/1 solid 2.6k 46.9s
a3-cross-shaft-01 20260429T001255Z-9522 fail 0.75 step_roundtrip · STEP drift on 1/1 solid 2.6k 41.2s
a3-cross-shaft-01 20260429T001214Z-1863 fail 0.75 step_roundtrip · STEP drift on 1/1 solid 2.6k 40.6s
a3-octagonal-flange-01 20260429T001046Z-de66 fail 0.67 bbox · X off by -1.65mm 4.5k 87.8s
a3-octagonal-flange-01 20260429T000804Z-28c7 fail 0.00 valid_solid · no valid solid produced 8.2k 162.1s
a3-octagonal-flange-01 20260429T000616Z-ba90 fail 0.00 valid_solid · solid invalid 6.0k 108.0s
a3-octagonal-flange-01 20260429T000445Z-9852 fail 0.67 bbox · X off by -1.65mm 5.6k 90.6s
a3-octagonal-flange-01 20260429T000254Z-2be7 fail 0.83 bbox · X off by -1.65mm 6.3k 109.9s
a3-cross-shaft-01 20260428T233319Z-c959 fail 0.75 step_roundtrip · STEP drift on 1/1 solid 2.3k 20.4s
a3-octagonal-flange-01 20260428T233213Z-12bc fail 0.83 bbox · X off by -1.65mm 6.6k 65.6s
a3-octagonal-flange-01 20260428T233052Z-2da0 fail 0.00 valid_solid · no valid solid produced 8.2k 80.7s
a3-octagonal-flange-01 20260428T232730Z-a3bf fail 0.83 bbox · X off by -1.65mm 7.3k 95.1s
a3-octagonal-flange-01 20260428T232553Z-a6bb fail 0.83 bbox · X off by -1.65mm 7.1k 96.3s
c-reacher-01 20260428T215515Z-9e23 fail 0.20 body_valid · check not implemented 6.3k 48.7s
c-reacher-01 20260428T215422Z-a8a3 fail 0.20 body_valid · check not implemented 6.6k 52.6s
a2-stepped-pyramid-01 20260428T215252Z-a730 PASS 1.00 — 1.9k 11.3s
a2-stepped-pyramid-01 20260428T215241Z-d236 PASS 1.00 — 1.9k 11.2s
a2-stepped-pyramid-01 20260428T215228Z-110f PASS 1.00 — 1.9k 13.0s
a2-stepped-pyramid-01 20260428T215206Z-0046 PASS 1.00 — 2.0k 22.0s
a2-stepped-pyramid-01 20260428T215154Z-03db PASS 1.00 — 2.0k 11.8s
a2-cube-with-pocket-01 20260428T215143Z-51b8 PASS 1.00 — 1.8k 10.9s
a2-cube-with-pocket-01 20260428T215133Z-c4d2 PASS 1.00 — 1.8k 10.1s
a2-cube-with-pocket-01 20260428T215123Z-d2a4 PASS 1.00 — 1.7k 9.8s
a2-cube-with-pocket-01 20260428T215112Z-a503 PASS 1.00 — 1.9k 11.5s
a2-cube-with-pocket-01 20260428T215102Z-b263 PASS 1.00 — 1.7k 9.6s
a2-stepped-block-01 20260428T215050Z-ad1a PASS 1.00 — 1.9k 11.5s
a2-stepped-block-01 20260428T215041Z-c90d PASS 1.00 — 1.7k 9.3s
a2-stepped-block-01 20260428T215030Z-538f PASS 1.00 — 1.9k 10.8s
a2-stepped-block-01 20260428T215019Z-cc5d PASS 1.00 — 1.7k 10.8s
a2-stepped-block-01 20260428T215009Z-e754 PASS 1.00 — 1.7k 9.9s
a2-channel-bracket-01 20260428T214957Z-77ce PASS 1.00 — 1.7k 12.0s
a2-channel-bracket-01 20260428T214946Z-670c PASS 1.00 — 1.7k 10.8s
a2-channel-bracket-01 20260428T214935Z-661e PASS 1.00 — 1.7k 10.7s
a2-channel-bracket-01 20260428T214926Z-8e1b PASS 1.00 — 1.7k 9.1s
a2-channel-bracket-01 20260428T214916Z-e045 PASS 1.00 — 1.7k 10.2s
a2-bolt-circle-block-01 20260428T214848Z-5917 PASS 1.00 — 3.8k 27.7s
a2-bolt-circle-block-01 20260428T214822Z-6efd PASS 1.00 — 3.4k 25.1s
a2-bolt-circle-block-01 20260428T214753Z-1307 PASS 1.00 — 3.6k 29.1s
a2-bolt-circle-block-01 20260428T214716Z-8d5a PASS 1.00 — 3.5k 35.9s
a2-bolt-circle-block-01 20260428T214641Z-e4de PASS 1.00 — 4.3k 35.2s
a2-mounting-rail-01 20260428T214626Z-2d24 PASS 1.00 — 2.3k 14.7s
a2-mounting-rail-01 20260428T214607Z-010f PASS 1.00 — 2.6k 18.8s
a2-mounting-rail-01 20260428T214549Z-a7e7 PASS 1.00 — 2.4k 17.9s
a2-mounting-rail-01 20260428T214529Z-d6a9 PASS 1.00 — 2.2k 20.1s
a2-mounting-rail-01 20260428T214500Z-6e7a PASS 1.00 — 2.2k 28.3s
a2-finned-block-01 20260428T214443Z-b783 PASS 1.00 — 2.5k 17.4s
a2-finned-block-01 20260428T214427Z-0107 PASS 1.00 — 2.5k 16.1s
a2-finned-block-01 20260428T214407Z-4b48 PASS 1.00 — 2.4k 20.1s
a2-finned-block-01 20260428T214352Z-6b51 PASS 1.00 — 2.2k 14.4s
a2-finned-block-01 20260428T214332Z-5d4b PASS 1.00 — 2.7k 20.2s
a2-cubemark-01 20260428T214258Z-dac3 PASS 1.00 — 4.0k 33.4s
a2-cubemark-01 20260428T214214Z-c29c fail 0.00 valid_solid · solid invalid 4.2k 43.8s
a2-cubemark-01 20260428T214149Z-082b PASS 1.00 — 3.8k 24.6s
a2-cubemark-01 20260428T214125Z-6361 PASS 1.00 — 3.4k 23.6s
a2-cubemark-01 20260428T214058Z-e322 PASS 1.00 — 3.8k 26.7s
a2-tee-bracket-01 20260428T214039Z-b3f5 PASS 1.00 — 2.6k 18.3s
a2-tee-bracket-01 20260428T214025Z-92ae PASS 1.00 — 2.3k 14.8s
a2-tee-bracket-01 20260428T214010Z-ce3b PASS 1.00 — 2.3k 14.7s
a2-tee-bracket-01 20260428T213952Z-fb3c PASS 1.00 — 2.6k 18.1s
a2-tee-bracket-01 20260428T213933Z-bd29 PASS 1.00 — 2.7k 18.9s
a2-square-flange-01 20260428T213913Z-4be8 PASS 1.00 — 2.7k 19.6s
a2-square-flange-01 20260428T213849Z-b80b PASS 1.00 — 2.8k 23.1s
a2-square-flange-01 20260428T213827Z-7089 PASS 1.00 — 3.0k 21.8s
a2-square-flange-01 20260428T213805Z-cbe9 PASS 1.00 — 2.8k 22.3s
a2-square-flange-01 20260428T213743Z-a9f5 PASS 1.00 — 3.1k 21.0s
a2-washer-01 20260428T213736Z-b325 PASS 1.00 — 1.1k 7.1s
a2-washer-01 20260428T213726Z-3c4a PASS 1.00 — 1.4k 10.3s
a2-washer-01 20260428T213717Z-ccb0 PASS 1.00 — 1.3k 8.8s
a2-washer-01 20260428T213707Z-50c4 PASS 1.00 — 1.2k 9.7s
a2-washer-01 20260428T213655Z-fc3d PASS 1.00 — 1.4k 11.2s
a2-l-bracket-01 20260428T213635Z-f681 PASS 1.00 — 2.7k 20.5s
a2-l-bracket-01 20260428T213615Z-46d8 PASS 1.00 — 2.3k 19.8s
a2-l-bracket-01 20260428T213558Z-f74c PASS 1.00 — 2.1k 17.2s
a2-l-bracket-01 20260428T213532Z-c667 PASS 1.00 — 2.6k 25.6s
a2-l-bracket-01 20260428T213509Z-c90f PASS 1.00 — 2.5k 22.6s
a2-flanged-cap-01 20260428T213435Z-71b4 fail 0.86 mass_props · volume off by 0.6% 3.3k 33.5s
a2-flanged-cap-01 20260428T213348Z-86aa fail 0.86 mass_props · volume off by 0.6% 3.1k 46.8s
a2-flanged-cap-01 20260428T213317Z-2bd7 PASS 1.00 — 3.2k 30.3s
a2-flanged-cap-01 20260428T213232Z-abde PASS 1.00 — 3.0k 43.8s
a2-flanged-cap-01 20260428T213147Z-826f PASS 1.00 — 3.3k 44.1s
a1-pipe-01 20260428T213138Z-12c7 PASS 1.00 — 1.1k 9.2s
a1-pipe-01 20260428T213129Z-e64e PASS 1.00 — 1.3k 9.2s
a1-pipe-01 20260428T213116Z-644c PASS 1.00 — 1.5k 12.1s
a1-pipe-01 20260428T213105Z-ac87 PASS 1.00 — 1.2k 10.8s
a1-pipe-01 20260428T213053Z-932c PASS 1.00 — 1.3k 12.0s
a1-block-01 20260428T213036Z-2f19 PASS 1.00 — 1.2k 16.8s
a1-block-01 20260428T213024Z-f42d PASS 1.00 — 1.3k 12.2s
a1-block-01 20260428T213013Z-8f6f PASS 1.00 — 1.2k 11.0s
a1-block-01 20260428T213004Z-99ac PASS 1.00 — 1.1k 9.0s
a1-block-01 20260428T212952Z-5fa5 PASS 1.00 — 1.3k 12.2s
a1-cone-01 20260428T212945Z-893c PASS 1.00 — 960 6.7s
a1-cone-01 20260428T212939Z-d8f2 PASS 1.00 — 1.0k 6.6s
a1-cone-01 20260428T212930Z-b46c PASS 1.00 — 1.0k 8.9s
a1-cone-01 20260428T212920Z-0a8f PASS 1.00 — 1.1k 9.5s
a1-cone-01 20260428T212914Z-d7cd PASS 1.00 — 960 5.8s
a1-sphere-01 20260428T212908Z-8ba2 PASS 1.00 — 917 6.6s
a1-sphere-01 20260428T212903Z-6914 PASS 1.00 — 789 4.9s
a1-sphere-01 20260428T212856Z-9ffd PASS 1.00 — 917 6.4s
a1-sphere-01 20260428T212850Z-ddc0 PASS 1.00 — 856 5.6s
a1-sphere-01 20260428T212844Z-d9f4 PASS 1.00 — 917 6.5s
a1-stepped-shaft-01 20260428T212831Z-72f3 PASS 1.00 — 1.5k 12.5s
a1-stepped-shaft-01 20260428T212818Z-4f83 PASS 1.00 — 1.5k 13.4s
a1-stepped-shaft-01 20260428T212756Z-e826 PASS 1.00 — 1.3k 21.4s
a1-stepped-shaft-01 20260428T212744Z-6ce8 PASS 1.00 — 1.3k 12.3s
a1-stepped-shaft-01 20260428T212732Z-d512 PASS 1.00 — 1.3k 12.1s
a1-plate-01 20260428T212655Z-e0fb fail 0.00 valid_solid · solid invalid 3.0k 37.0s
a1-plate-01 20260428T212624Z-a391 PASS 1.00 — 2.9k 30.7s
a1-plate-01 20260428T212550Z-a979 PASS 1.00 — 2.8k 33.5s
a1-plate-01 20260428T212513Z-336d PASS 1.00 — 2.4k 37.3s
a1-plate-01 20260428T212443Z-9cd5 PASS 1.00 — 2.6k 29.6s
a1-cube-01 20260428T212434Z-929d PASS 1.00 — 1.1k 9.5s
a1-cube-01 20260428T212424Z-97ee PASS 1.00 — 1.1k 9.5s
a1-cube-01 20260428T212414Z-8b89 PASS 1.00 — 1.1k 10.3s
a1-cube-01 20260428T212405Z-5e4d PASS 1.00 — 1.1k 8.7s
a1-cube-01 20260428T212350Z-c018 PASS 1.00 — 1.1k 14.9s
a1-cube-01 20260428T212237Z-5584 PASS 1.00 — 1.1k 12.3s
mecheval · an evaluation suite by Municipal Robotics
sibling project: vcad. · github

generated 2026-06-17T03:16:07.292Z · static site, regenerate with npm run build -w @mecheval/leaderboard