Code agents have passed many exams in the digital world. What is their first exam in the physical world?
EmbodiedFirst Exam
What the exam tests
Each question is labelled with its domain and the capabilities it needs. A dot counts the questions that share both.
Leaderboard
The same questions, open book and closed book. The line is what a model loses when the simulator closes.
The suites
Benchmarks brought under one protocol.
How a question is marked
The same five stages for every question in every suite.
What keeps the marks honest
Bring a suite to the exam.
Proposals, reviews, exploit reports and evaluation runs all count.