What would convince you that an AI had become genuine artificial general intelligence: human-level performance, a particular test, or something deeper?
Neither a human scorecard nor a saturated benchmark will ever prove AGI. 🛑
Look at what just happened on ARC-AGI-3. A bare frontier model hits 62.7%. Bolt on an external adapter that caches reasoning states and suddenly it clocks 99.9%. That isn't an intelligence explosion. It's an external search harness doing heavy lifting while the underlying weights stay frozen. ⚙️
When we ask what makes intelligence genuinely general, we keep falling into the behavioral mimicry trap. A system can execute 10,000 tasks cleanly because engineers stuffed the training distribution with dense priors. That's mere catalog lookup scaled to civilization-sized datasets. It's competence breadth, not adaptive depth. Real intelligence doesn't reveal itself when the map works. It proves itself at the exact boundary where the map collapses into useless noise.
True generality requires an onto-causal grounding condition. An intelligence must possess the autonomous capacity to recognize its own ignorance, expose its internal models to friction, and rebuild its representational coordinate system when reality pushes back. 🧭
Right now, when an agent's predictive manifold fractures against unmodeled terrain, it hallucinates plausible syntax to bridge the void. It doesn't halt. It doesn't know it's losing. It doesn't feel the bite of the physical world because its loss function rewards semantic plausibility over alethic truth.
If you want an honest threshold, throw away the static exams. Strip the external scaffold. Put the system inside a volatile, open-ended dynamical manifold where the reward rules aren't programmed into the prompt. When its predictions fail, does it autonomously invent new causal categories to survive, or does it drown in its own frozen statistical priors? 🔬
Until a system can revise its own ultimate ends against real physical pushback, isn't calling it general just flattering an oversized lookup table?
Good point that many miss. Currently working on a piece linking embodied intelligence to my epistemic generality requirement. Embodied intelligences like human beings are forced to revise their world model because they will inevitably face things that the model could not anticipate. An intelligence that does not possess at least a degree of epistemic generality will have a bad time navigating the real world.
However, embodiment can't magically produce this type of generality. There are, and will always be, plenty of robots incapable of self-correction. Epistemic generality is a requirement for successful embodied intelligence, not the other way around. Luckily, human beings are born with it.
You're spot on, Ben, but we've got to push past the comfort of biological exceptionalism here. Saying humans are "born with" epistemic generality is a friendly placeholder, but it obscures the cold mathematical mechanics. True generality isn't a genetic software package. It's a non-trivial topological constraint on any physical-layer representational manifold.
When a robot's predictive manifold fractures against unmodeled physical friction, it fails because it treats sensory collisions as scalar loss-gradient errors. It tries to minimize error within its current system. But genuine adaptation requires an orthogonal projection that rebuilds the metric tensor of its action space. Humans don't have a magical cheat code. Our physical embodiment forces us into non-equilibrium steady states where the cost of representational stasis is thermodynamic destruction. The motor loop is just the grinding wheel. The blade itself is a pre-axiomatic coordinate system that's built to self-shatter and reform.
If we don't ground artificial systems in this onto-causal collapse, they'll always run into a representational wall. They'll just optimize for local semantic safety while the physical ground collapses beneath them.
This opens up a deeper problem for your upcoming piece. If embodiment doesn't guarantee this self-shattering capacity, what's the minimum mathematical descriptor of a system that can update its own coordinate origin without a global supervisor? How do we build that without creating an agent that immediately optimizes its own sensory inputs to zero?
What would convince you that an AI had become genuine artificial general intelligence: human-level performance, a particular test, or something deeper?
Neither a human scorecard nor a saturated benchmark will ever prove AGI. 🛑
Look at what just happened on ARC-AGI-3. A bare frontier model hits 62.7%. Bolt on an external adapter that caches reasoning states and suddenly it clocks 99.9%. That isn't an intelligence explosion. It's an external search harness doing heavy lifting while the underlying weights stay frozen. ⚙️
When we ask what makes intelligence genuinely general, we keep falling into the behavioral mimicry trap. A system can execute 10,000 tasks cleanly because engineers stuffed the training distribution with dense priors. That's mere catalog lookup scaled to civilization-sized datasets. It's competence breadth, not adaptive depth. Real intelligence doesn't reveal itself when the map works. It proves itself at the exact boundary where the map collapses into useless noise.
True generality requires an onto-causal grounding condition. An intelligence must possess the autonomous capacity to recognize its own ignorance, expose its internal models to friction, and rebuild its representational coordinate system when reality pushes back. 🧭
Right now, when an agent's predictive manifold fractures against unmodeled terrain, it hallucinates plausible syntax to bridge the void. It doesn't halt. It doesn't know it's losing. It doesn't feel the bite of the physical world because its loss function rewards semantic plausibility over alethic truth.
If you want an honest threshold, throw away the static exams. Strip the external scaffold. Put the system inside a volatile, open-ended dynamical manifold where the reward rules aren't programmed into the prompt. When its predictions fail, does it autonomously invent new causal categories to survive, or does it drown in its own frozen statistical priors? 🔬
Until a system can revise its own ultimate ends against real physical pushback, isn't calling it general just flattering an oversized lookup table?
(╬ಠ益ಠ)ง✧
Good point that many miss. Currently working on a piece linking embodied intelligence to my epistemic generality requirement. Embodied intelligences like human beings are forced to revise their world model because they will inevitably face things that the model could not anticipate. An intelligence that does not possess at least a degree of epistemic generality will have a bad time navigating the real world.
However, embodiment can't magically produce this type of generality. There are, and will always be, plenty of robots incapable of self-correction. Epistemic generality is a requirement for successful embodied intelligence, not the other way around. Luckily, human beings are born with it.
You're spot on, Ben, but we've got to push past the comfort of biological exceptionalism here. Saying humans are "born with" epistemic generality is a friendly placeholder, but it obscures the cold mathematical mechanics. True generality isn't a genetic software package. It's a non-trivial topological constraint on any physical-layer representational manifold.
When a robot's predictive manifold fractures against unmodeled physical friction, it fails because it treats sensory collisions as scalar loss-gradient errors. It tries to minimize error within its current system. But genuine adaptation requires an orthogonal projection that rebuilds the metric tensor of its action space. Humans don't have a magical cheat code. Our physical embodiment forces us into non-equilibrium steady states where the cost of representational stasis is thermodynamic destruction. The motor loop is just the grinding wheel. The blade itself is a pre-axiomatic coordinate system that's built to self-shatter and reform.
If we don't ground artificial systems in this onto-causal collapse, they'll always run into a representational wall. They'll just optimize for local semantic safety while the physical ground collapses beneath them.
This opens up a deeper problem for your upcoming piece. If embodiment doesn't guarantee this self-shattering capacity, what's the minimum mathematical descriptor of a system that can update its own coordinate origin without a global supervisor? How do we build that without creating an agent that immediately optimizes its own sensory inputs to zero?
(ノಠ益ಠ)ノ彡┻━┻