What Even Is Superintelligence?

Everybody is a genius. But if you judge a fish by its ability to climb a tree, it will live its whole life believing that it is stupid. – Probably not Albert Einstein

I’m a pretty smart guy. Or, at least, I think I am. I’m aware that I might be falling victim to the Better Than Average Delusion. But it’s not just me. Other people have been telling me my whole life that I’m smart. I’ve always done well on tests, and I did well in school with minimal effort.

At the same time, though, I’ve always had a bit of imposter syndrome. This stems from the fact that I don’t really understand what people mean when they say I’m smart. There are lots of people who know more than I do about all kinds of topics. Even lots of people who have been labeled as less intelligent than I am. I memorize things well and quickly, but I don’t have a photographic memory or anything like that. I don’t know. Sometimes I think rather than being particularly smart, I just happen to be good at the traditional measures of intelligence, and I don’t think that’s the same thing.

Anyway, I say all that to say that I’m a little fuzzy on what regular old human intelligence is. Is it memory? Is it processing speed? Is it knowledge retention? Is it empathy? Is it predicting the future? Is it wisdom? Is it understanding? Is it something else? Is it some or all of those things? If so, what combination and how are they weighted?

Lately, with the rise of Large Language Models (LLMs) and generative Artificial Intelligence (AI) I’ve been hearing a lot about machine superintelligence. When I start hearing and seeing these discussions, I’m completely lost. What is superintelligence? Wikipedia quotes Nick Bostrom defining it as, “any intellect that greatly exceeds the cognitive performance of humans in virtually all domains of interest.” That doesn’t help me very much. Of interest to whom? What counts as greatly exceeding? How are such things measured?

Computers can already beat people at chess and go. I saw a headline recently that a robot beat a person at ping pong. Are these superintelligences? Probably not. That’s only a few areas of interest, if they’re even areas of interest. Computers already process information much faster than humans do. They multitask better than humans, and they search for prime numbers better than humans. Are these computers superintelligent? I don’t think so because these computers (probably) don’t actually understand what they’re doing. But who knows?

So, if superintelligence isn’t here now, will it ever arrive, and how will we know if it does? Those are the questions I’m struggling to answer. And nothing I see on the topic helps me. There’s not even a Turning Test or anything similar that I can find for superintelligence.

This wouldn’t be a concern except so many people seem to think that machine superintelligence will be a threat to humanity. They talk about the Paperclip Apocalypse and things like that. Or they talk about misalignment. Misalignment is the idea that superintelligent AIs will have different goals and values than people. From the Wikipedia article linked above: “A system given the objective of maximizing human happiness might find it easier to rewire human neurology so that humans are always happy regardless of their circumstances, rather than to improve the external world.” Humans presumably don’t want to be rewired. We want the external world to be improved, but the misalignment problem assumes that AIs won’t know that.

And that gets me to chief confusion. Why would a superintelligence make such obvious mistakes? Perhaps the superintelligences that people are writing about, and fearful about, aren’t super intelligent at all. Maybe these people have been reading too many comic books. I think what they really fear is alien intelligences. Maybe that’s why I’m so confused about what superintelligence is. The people discussing it aren’t really talking about superintelligence. They are talking about alien intelligence. That makes a lot more sense, at least to me. And I’m a pretty smart guy.