5 October 2026 · 2 min read
How do you measure a mind?
The SI has seven base units and none of them is for intelligence. That gap explains most of the confusion in AI progress claims.
The International System of Units, the SI, defines seven base units: the second, the metre, the kilogram, the ampere, the kelvin, the mole and the candela. Everything a physicist measures is built from those. There is no unit for intelligence, which is an awkward fact for a field that claims to be producing more of it every year. It is also the private joke in this site's address.
IQ is a ranking
The best-known number, IQ, is a position within a human population. Scores are scaled so the average is 100 and the standard deviation is 15. An IQ score says where a person stands relative to other people on a set of tests. It has no zero, no natural upper end, and no meaning for a system that was not part of the population.
Benchmarks saturate
AI research measures progress with benchmarks: fixed sets of questions with known answers. They work well until they are solved. Scores climb to the ceiling, and a benchmark at the ceiling can no longer separate a good system from a great one. Test questions also leak into training data, so a high score may reflect memory more than ability. And once a benchmark becomes the goal, it gets optimised directly, which is Goodhart's law again.
Better instruments
- Task length. How long a task, measured in the time it takes a skilled person, can a system complete on its own? Evaluation groups such as METR track this, and it has the advantage of not having a ceiling.
- Novelty. Performance on problems built to be unlike anything in the training data, which tests reasoning over recall.
- Learning efficiency. How many examples a system needs to pick up a new skill. People need few. This is closer to what the word intelligence means in everyday use.
- Real work. Whether the output is good enough that someone would pay for it, sustained over weeks and not only in a demonstration.
A rule for readers
When a result is announced, ask three things. What exactly was measured? Could the system have seen the answers? Does the measure still have room at the top? A claim that survives those questions is worth attention. Most headlines do not.
The scale on the front page is drawn without numbers for this reason.