A short Specification Gaming Story
You think you understand the basics of Geometry
Your request is a square, so you give your specification to the AI, input:
Give me a shape
with 4 sides equal length,
with 4 right angles
And it outputs this:
Here is another valid result:
And behold here is another square 🤪
Specification Gaming tells us:
The AGI can give you an infinite stream of possible “Square” results
And the Corrigibility problem tells us:
Whatever square you get at the output,
you won’t be able to iterate and improve upon.
You’ll be stuck with that specific square for eternity, no matter what square you had in your mind.
Of-course the real issue is not with these toy experiments
it’s with the upcoming super-capable AGI agents,
we’re about to share the planet with,
operating in the physical domain
Oh, the crazy shapes our physical universe will take,
with AGI agents gaming in it!
