Chollet on Intelligence

The issue is Control.

Let an ultraintelligent machine be defined as a machine that can far surpass all the intellectual activities of any man however clever. Since the design of machines is one of these intellectual activities, an ultraintelligent machine could design even better machines; there would then unquestionably be an “intelligence explosion,” and the intelligence of man would be left far behind. Thus the first ultraintelligent machine is the last invention that man need ever make, provided that the machine is docile enough to tell us how to keep it under control.

I think generally, intelligence is very situational, much more so then people who write about intelligence explosions expects.

If intelligence is fundamentally linked to specific sensorimotor modalities, a specific environment, a specific upbringing, and a specific problem to solve, then you cannot hope to arbitrarily increase the intelligence of an agent merely by tuning its brain — no more than you can increase the throughput of a factory line by speeding up the conveyor belt.

And again, the analogy seems correct, intelligence comes from environment stimulus and interaction, if a “general AI” is simply the most gifted problem solver, it still does not allow it to take over the world. Ultimately any human design would just maximize the latent intelligence, but the expressed intelligence still requires factoring in the environment and context that the design exist in.

I think it’s also the case proponents of exponential intelligence growth also ignore bottleneck factors of expressing intelligence, be it resources, lack of prior incremental improvements, and/or deliberate limits. Human intelligence is the result of incremental collaborative slow improvement, which goes against the explosive singularity models proposed.

• Intelligence is situational
• Intelligence exist within the confines of the system which creates it
• Expressed intelligence is externalized, distributed and collaborated.
• Recursively self improving system has diminishing returns, and results in linear, at best sigmoidal growth in progress.
• We are ALREADY recursive improving in intelligence.

Instead of worrying about the singularity, we should worry about how we assume machine learning works and create the harmful biases, and treat how we use these tools too impact others first. Machines are not scary, humans misusing machines are.

I think the risks are all human decisions ultimately, surveillance, misuse of data, adversarial interactions, negative reinforcements, rabbit holes, the common thread is human in the loop doing what’s short term beneficial, and not something with correct social utility. To combat this mass population manipulation is the true challenge of the 2020s+. This decade will let us know whether we are bothered to regain the control of our digital and physical selves.

Don’t use AI as a tool to manipulate your users; instead, give AI to your users as a tool to gain greater agency over their circumstances.

As for the machines, I would simply, turn it off /s.