The first ASI will be decentralized
Because the first NSIs are
Part of the worry of artificial super intelligence1 (ASI) is that it will be in the hands of specific companies or specific people. (Yes there are many other worries, let’s talk about one at a time.) My claim here is simple: the first ASI will be decentralized. This doesn’t mean that zero subsequent ASIs will be centralized, but it should reduce our prior, as I explain below.
Why would the first ASI be decentralized? Let’s first look at a couple forms of
Natural Super Intelligence
These are natural (i.e., not humanly designed) systems that have super-human intelligence: cultural evolution and the market economy. We could call them NSIs.
The cultural evolutionary theorists never find individual intelligence that impressive by itself.
This point is sufficiently argued by Joseph Henrich, Robert Boyd, and Peter Richardson, among others. The process of social learning through cultural evolution gave humans the edge we have; our individual smarts did not. To borrow a Whitehead quote from F.A. Hayek: “Civilization advances by extending the number of important operations which we can perform without thinking about them.”2 Our languages, norms, routines, written and unwritten institutions, practices, rituals, even our behavioral biases—these all team up to give humans a collective edge as a species. We learn almost everything from others and then tinker on the smallest margins. Only with our collective brain and cultural iteration are we able to claim status as kings and queens of the globe.
The market economy is the more familiar NSI; we are all very intimate with it. The social economic problem is that we somehow need to know what things to make and how to make it (production), who to give it to (distribution), and what to make in the future (innovation). We could do this a whole bunch of different ways, but we’d like to do it in the way that is efficient and conserves our resources.
The best way we have stumbled upon so far is the market economy. Everyday billions of people make judgements about what is right and true; they share those judgements by making decisions in the market, voting with their dollars. Then, prices adjust and coordinate everyone’s behavior to efficiently communicate if we should be collectively consuming or producing more or less of each thing. Everyday, when billions of people engage in exchange, we exchange opinion on how to use our resources and cognitive energy. The outcome is a system of productive specialization and social cooperation surpassing the highest intellect and most sophisticated algorithms.
Both of these systems work through the decentralized actions of individuals, coordinated through system-wide mediums. Thus we should expect with
Decentralized ASI
As long as we see increasingly agentic-like behavior from regular, human-level and below AIs (which we have), we should expend them to start a social order.
Consider a simple, likely, scenario. I will hire my non-ASI AI assistant to do something, say, go collect some data. It will then auction this task off on an AI job board, willing to pay some fee that I authorize. Some other AI—the one specialized, or with the appropriate privileges—picks up the job. My assistant gives me the data.
Notice that this is no different than me hiring my grocer to collect some bananas. In both cases I, the principal, am hiring an agent to tap into the market economy to fulfil a task at the least cost to society.
Should I call that decentralized ASI? Well, collecting data is a task humans can do. But if my assistant is truly agentic, it will likely have other goals than what I ask, ones I can’t exactly comprehend. In pursuit of its own goals, it may hirer some other agents or communicate with them in some way.
Drawing from what we learn about NSIs, once these non-ASI AIs are interacting as individual agents, it would quite surprising if they didn’t collectively show some emergent super intelligence. Which monies will they use? Will they trust each other? How will they specialize and trade? How quickly will their cultures shift and evolve? No, I will not try to speculate.
But whatever this collective AI brain is, it is decentralized. We should ask,
So what?
The good news is that the first ASI will be decentralized and thus very likely to not be in the hands of a single bad actor.
The bad news (maybe?) is that the first ASI will be here before we know it, before any one lab can conjure up enough compute on their own.
Again, the first ASI being decentralized doesn’t mean that all other ASIs will decentralized. But once this system / market / society gets off the ground, it will likely shift the rent race from achieving the ASI to directing parts of it. Specialization and trade within the AI network will be more important than the one neural network to rule them all.
“Any intellect that greatly exceeds the cognitive performance of humans in virtually all domains of interest” See here.
Chapter 2, Constitution of Liberty.