Two books I read recently helped me tremendously to understand what’s happening now with Artificial Intelligence: Prediction Machines and Power and Prediction (affiliate links) by Ajay Agrawal, Joshua Gans, and Avi Goldfarb.
The authors are economists and professors at the University of Toronto, one of the leading institutions studying the economic impacts of AI. Their books helpfully frame the advent of AI as just one in a long series of economic paradigm shifts, each of which has been closely studied and dissected before.
This article is a summary of the ideas I found most enlightening and impactful, and my interpretation of how they apply to modern knowledge work.
AI Is Prediction
It’s easy to get caught up in all the hype around AI’s potential, but at the fundamental economic level, AI is about dramatically lowering the cost of prediction.
When you ask ChatGPT “What is the capital of Delaware?”, the machine learning algorithm that powers it doesn’t “know” the answer in any sense. Instead, it is predicting “What does the user want to hear?”
But what exactly is prediction?
We use that word in a day-to-day sense of “guessing what’s going to happen,” but here we’re using it in a much broader sense. Prediction is the process of “filling in missing information.” It takes data you have, such as shoppers’ buying habits in a store, and uses it to generate data you don’t have, such as what they’re likely to buy next.
Think of each time you swipe your credit card at a store. The merchant has no idea who you are or what your creditworthiness might be. No human at the credit card provider knows either.
It is an algorithm that crunches the data it has about you, as well as the specifics of the transaction you’re trying to make, and then in seconds provides an answer: Approved or Not Approved. That is an example of a prediction not about the future, but about whether this transaction is authorized right at this moment.
Machines predict whether a smudge on an x-ray might be a tumor. They predict whether the person looking into the iPhone camera is its owner or not. They predict what you might want to buy next on Amazon, which links you might want to click on Google, and even when to automatically apply the brakes in your car to avoid a collision.
Prediction isn’t only about predicting what will happen in the future – it is about predicting what data a user might want.
The Mass Production of Prediction
Putting on an economist’s lens, we can think of modern AI as a dramatic fall in the cost of prediction. Just as the factories of the Industrial Revolution started churning out mass-produced consumer goods, making previously rare and expensive items suddenly cheap and commonplace, we are witnessing the rise of “prediction factories” doing much the same with information.
Many of the highest-paying jobs requiring the most training involve making predictions. Doctors predict the right diagnosis, lawyers the right legal argument, and engineers the most effective piece of code to accomplish a function. Previously it was tremendously expensive to access the predictions of these highly trained professionals, but the price of that particular commodity is now plummeting.
What happens when the price of something drops? We typically find many more ways to use it. In other words, as the price drops, demand goes up.
The economist William Nordhaus documented how the price of artificial light plummeted by a factor of 400 from the early 1800s to today. We replaced kerosene with electricity, and it became trivially inexpensive to light our homes and offices at all hours.
In response, our consumption of artificial light exploded! We learned to use light carelessly, with abandon, illuminating everything from our garden pathways at home to empty streets at night.
The invention of computers similarly made arithmetic – a valuable commodity previously requiring highly trained bookkeepers – so cheap that we use it now without a second thought. We didn’t just consume more arithmetic – we also found radically innovative and unexpected new ways to use it: to play digital music, manipulate digital images, send digital messages, and consume digital content in all forms.
Note that the digital spreadsheet eliminated the most time-consuming activity for bookkeepers: doing arithmetic calculations. You might think that the entire profession would have disappeared as a result. But products like VisiCalc (the very first spreadsheet computer program) actually made their work more valuable overall. You could easily calculate the expected return for various projects, and then run multiple scenarios using different assumptions.
The same people who had laboriously calculated the answers by hand in the past were now best positioned to ask the right questions of the new computerized spreadsheet. Instead of making them obsolete, the new technology augmented them with superpowers.
What will happen as the cost of prediction goes from “expensive” to “practically free”? We need to understand a couple other points to find out.
AI Decouples Prediction And Judgment
One reason prediction is currently so expensive and so few are qualified to provide it is that it is bundled with another rare quality: judgment.
When you ask your doctor for a diagnosis, you don’t just want to know your odds of survival (which is a prediction). You also want to know the course of treatment they recommend. That takes judgment. The judgment of what to do next always depends on the prediction of what will happen if we do.
“If you take this test, we’ll know what’s causing your pain.” “If you take these pills, your symptoms will be alleviated.” “If you undergo this surgery, your prognosis will improve.”
In other words, judgment has always been bundled together with prediction, which makes it even more difficult and expensive to access them in combination.
While prediction says what is likely to happen, judgment assigns value to the possible outcomes. It is the skill of determining the reward or profit of an outcome. Prediction says what will likely happen; judgment decides how much that outcome is worth to you.
To exercise judgment, you need to know what you want, and how much it matters to you to get it. It is about making tradeoffs to achieve the highest value possible from the options at hand.
Both prediction and judgment are combined to produce a crucial capability: decision-making. Decision-making is at the core of most occupations today, not to mention every moment of our daily lives.
Schoolteachers decide the best way to educate their students. Managers decide who to recruit and who to promote. Police officers decide how to handle potentially dangerous situations. Parents decide how much screen time their kids can have.
As humans, we have always had to do these three things – prediction, judgment, and decision-making – together.
But now that prediction is being unbundled and made superabundant, we have the chance for the first time to break this process down into its constituent parts. We can start to think clearly about which human activities will then diminish in value, and which ones will increase in value.
AI Allows Us to Replace Rules With Decisions
As talented as humans are at making decisions, it is also tremendously taxing for us.
Think about the last time you made a major purchase for your home, like an oven, mattress, or new car. It probably took days if not weeks of serious reflection, researching and comparing options, identifying essential features, weighing tradeoffs, and of course, determining what you could afford.
To mitigate the tremendous mental and energetic cost of decision-making, we’ve developed countless rules as a society to guide our actions. We always stick to the right on streets and walkways (or left, in certain countries). We always take our trash out to the curb on certain pickup days. We always start work around the same time and end at the same time.
We may chafe at some of these rules, but there’s no question they save us a staggering amount of cognitive effort. So much so that we add a whole new set of rules to govern our personal lives: always eat this, never eat that; always shop here, never shop there; always do it this way, never do it that way.
We form these habits and rules when it costs too much to optimize our decisions. We decide not to decide. Just try to imagine how difficult life would be if you had to invent a completely new routine every day.
AI is a disruption to this rules-based order in which we live. If our rules exist to save us energy, and the reason we need to save energy is because decisions are so taxing, and one of the key inputs to decisions are predictions, then by making predictions cheap and plentiful, AI changes the very basis on which our society runs.
Here’s an example: We typically follow the rule to arrive at the airport two hours before the scheduled departure of our flight. This rule is so ingrained in us that it feels almost like a law of nature.
But it’s helpful to examine why this rule exists in the first place. It comes down to the large degree of uncertainty surrounding our arrival at the airport and getting to our gate. We don’t know what traffic will be like, how long the parking or dropoff will take, how long the lines will be at check-in and security, or any one of a number of other delays or snafus that might arise.
We can’t easily acquire the information that would be needed to reduce that two-hour buffer, so we make do: we leave early, bring lots of reading material or entertainment devices with us, and assume we’ll be waiting for a long time.
Now imagine if you had an AI that could take in all the needed information – your personal habits and time needed to get ready; local traffic and weather patterns; details about the layout and accessibility of the departing airport, and even data like the typical on-time departure rates for the airline and specific flight you’re taking. From all this data it could generate a customized airport departure time that told you exactly when to leave to maximize your chances of making your flight while minimizing the time you’ll have to wait.
Rather than blindly following a hard-wired rule like “two hours before,” you’re now following a personalized recommendation suited to your situation. In other words, because the prediction of how long it will take to arrive at your flight has become so cheap, you can now make the decision of when to leave, rather than following a rule.
Instead of making do with uncertainty, you now have the option to optimize your decision. You gain access to more options, without having to pay the cost of calculating them. You can contemplate more complexity with less risk. Rather than uncertainty being a threat or something to tolerate, it opens up more pathways for you to choose from.
When the Price of Something Falls, Its Complements Increase in Value
There’s another piece to the puzzle we have to understand: when the price of something falls, anything that is a “complement” to it will gain in value.
For example, imagine there was a technological breakthrough in battery technology and all cars could now be powered virtually for free via built-in solar panels. People would take many more trips and drive many more miles in response, leading to a rise in the value of all the complements to driving: drive-through fast-food and coffee places, roadside rest stops and convenience stores, and even streaming music and audiobooks.
If we think of prediction as just such a raw material that has suddenly plummeted in price, we should expect anything that is a complement to prediction – that is consumed along with prediction – to become more valuable.
This includes data because high-quality predictions require a lot of data. Any data that is difficult to acquire or that changes frequently will be disproportionately valuable, and those who know how to acquire it will be sought after.
It includes judgment – the skill of knowing which predicted outcomes are valuable and why. Judgment is based mostly on experience, and thus human experience of unusual or complex events will become far more valuable. Paradoxically, the more we automate business and society, the fewer opportunities humans will have to acquire hard-won experience, thus making it even more valuable.
It also includes action – we still need a human to take most of the actions inspired by the predictions that machines make. People who know how to effectively navigate the tremendous complexity and uncertainty of the real world, and to advance particular ideas or causes or points of view, will be more valuable than ever.
And finally, a sense of perspective is a complement to prediction that will rise in value. One definition of wisdom is “broad framing” – not losing sight of the big picture and keeping seemingly intractable problems and crises in perspective. As each element of decision-making gets increasingly unbundled into ever more specialized and narrow algorithms, this kind of broad perspective will become even more challenging to maintain than it already is.
At the same time, human-generated prediction will fall in value as it faces more and more competition from the “prediction factories” of AI.
For example, cab drivers were once required to memorize all the roads in their city to be able to efficiently route passengers to their destination. This knowledge would often take them years to acquire and was highly prized and well-compensated. But as soon as we all gained smartphone-based navigation, the value of a cab driver’s ability to predict the best route to a location dropped to nothing.
AI Deployment is Going to Take a Long Time
AI pioneer Andrew Ng, who founded the Google Brain project and was chief scientist at Baidu, proclaimed that “AI is the new electricity. It has the potential to transform every industry and to create huge economic value.”
We can look to history for examples of what it looks like when an epochal new technology transforms society. And using electricity as a guide, that history suggests the spread of AI will take a long time.
Thomas Edison’s invention of the lightbulb in 1879 is hailed as one of the most revolutionary moments in human history. And yet, two decades after this groundbreaking moment, electricity had barely begun to be adopted – only 3% of US households had it. That number was barely higher in factories.
However, fast-forward another couple of decades, and electricity was everywhere, reaching half the US population. Understanding what happened in those “in-between times” is crucial to predicting how AI will likely play out in our lifetimes.
Electricity was a challenge to the existing dominant source of power in the late 19th century: steam power.
Steam power was reliable but highly inefficient, losing 30–85% of its potential before it could be applied. Steam power would typically enter a factory at a single source, a 3-inch drive shaft of iron or steel to which belts and pulleys could be attached throughout the rest of the building. This system had one serious drawback: you either had to power the entire factory at full steam, or not at all. You couldn’t use power in small doses, or in different locations.
By changing from steam power to electricity, factories unlocked enormous advantages. They could be built far away from cities, in cheaper and more spacious areas. They could mount electric drives to individual machines, making them portable for the first time. It became cheaper to use power in smaller doses, only if and when needed, rather than powering the entire factory. And it was cleaner and more consistent as well.
But why did it take four decades before such a powerful innovation spread to half the country? It was because it couldn’t be tacked on to existing factories. The very concept of a factory had to be reinvented, and new ones had to be designed and built from scratch in new and untested locations before the benefits could be realized. New kinds of organizations had to be created that were capable of designing and running such factories. Only then did electrification show up in the productivity statistics, and in a big way.
Electricity was such a radical breakthrough that it required a new system-level solution. It decoupled energy use from its source, challenging deep assumptions that had been embedded in manufacturing for decades if not centuries.
Artificial Intelligence is of similar magnitude: by decoupling prediction from the rest of the decision-making process, it is reshaping how intelligence is accessed and applied. In turn, that is shifting where and how we as humans apply our own thinking to solve problems.
Just as with the rise of electricity, to fully realize the benefits of AI we will need completely new system-level solutions and even new kinds of organizations that we probably can’t envision today. Just think about how cars were vastly better than horses once widely available, but cars needed gas stations, good roads, and a whole new set of laws to function.
Which means AI will also take a long time to be deployed – probably decades. Technological change happens fast, but the social technologies needed to deploy it evolve slowly.
Three Case Studies of AI-Driven Prediction
Let’s take a look at three case studies, explained in detail in Prediction Machines and Power and Prediction, of what will happen once we have access to abundant AI-generated predictions:
- College admissions
- School bus drivers
- Weather forecasts
College Admissions
The challenging, time-consuming process of applying to college can be reframed as a prediction problem: each university is trying to predict which of the applicants is most likely to succeed in completing their degree.
Now imagine if the predictions required to make admissions decisions were free and instantaneous. Admissions advisors could run an algorithm that told them with a high degree of confidence who are the best candidates.
The cost of applications might fall to zero since they don’t require human time to evaluate. Schools might not even need applications – perhaps they could simply access your online profiles (with your permission) and generate an admission decision based on that data.
The admissions process itself might reverse: instead of an inbound process of prospective students applying to certain universities, those universities could proactively reach out to people whom their algorithm suggests would make excellent students. Not only would this save a lot of people a lot of effort, it might even lead to a wider applicant pool of people who wouldn’t normally consider higher education.
To take this a step further, imagine if this approach was extended throughout a student’s college career. The algorithm could predict which tutors and supplementary classes they would need at various points to maximize the value of their education. Rigid rules such as “everyone follows the same curriculum” and “everyone does the same homework” could be replaced with far more personalized assignments precisely targeted to address their weaknesses.
School Bus Drivers
If self-driving cars become fully automated, the job of bus drivers might seem an obvious candidate for elimination. But consider that bus drivers do much more than drive: they supervise a large number of kids, look out for and protect them from hazards, and maintain discipline inside the bus, among other responsibilities.
If the act of driving became automated, they might be able to spend much more time on these other, arguably more valuable tasks. That might elevate the job into a caretaking and educating role, which would require more training and higher standards but also might be better compensated.
The lesson, often repeated elsewhere, is that AI often automates a task, but rarely an entire job. If there is even one element of a job requiring human involvement, then a human must still perform it. And they can often use all the time that’s been freed up on higher value, more complex tasks.
Weather Forecasts
Over the last few decades, weather forecasts have become far more accurate, timely, and precise. Many of us now depend on daily and hourly forecasts without a second thought, and they’re rarely mistaken.
But a weather forecast is a “one-size-fits-all” service. It is the same forecast for everyone living in a city or region. Imagine if those forecasts could be personalized at the individual level.
You’d be given a temperature forecast that took into account your personal sensitivity to humidity and wind. The prediction for rain would consider your personal risk of being caught outside without a jacket or umbrella, based on where you live and even what you’re wearing. Your personal weather forecast could advise you to make certain decisions – such as wearing different clothes, taking an alternate route to work, or bringing your plants in from the back porch.
This example illustrates that the barrier to being able to offer more personalized products and services is often automation. We could generate the personalized weather forecast described above today, but it would take too much human time to be practical.
If AI took over the process, we should be able to switch from generic, uniform rules to thousands or millions of individualized recommendations, on everything from weather to traffic to job-seeking to having good conversations.
Five Questions to Thrive in the Age of AI
AI represents the rise of a new source of competition for us humans. Not only do we have to worry about other people taking our jobs, intelligent machines will substitute for an increasing number of the tasks we perform as well.
At this point, you may be wondering, “How can I use these ideas to protect and advance my own career and life?”
There are five questions suggested at various points in these two books that I think are worth asking yourself:
1. Which rules I’m currently following could be replaced with a decision?
The rules we follow throughout our days protect us from having to make too many decisions. But in protecting us, they also limit us. It takes perceptiveness and meta-cognition (the awareness of your own thinking) to even notice when you’re following a rule.
If you’re an investment analyst who researches new companies and writes reports with your findings, what rules are you following in the preparation of those reports? Do you always start the same way, looking at the same sources of data, evaluating them with the same criteria, and presenting your results in the same format?
Those are all rules. How can you use AI to customize or automate one or more of those steps so you have the ability to make a more finely-tuned decision about how to execute them?
2. How can I enhance my judgment using AI prediction?
Now that prediction and judgment have been decoupled, you will probably have to do much less prediction, but make more decisions with a greater degree of judgment. How can you use the newly plentiful supply of predictions to hone your decision-making skills?
For example, if you’re a graphic designer, you no longer have to create a new image from scratch hoping or guessing that it will be what your client wants. Instead, you can generate dozens of options with the click of a button. Could you apply your design skills to create a step-by-step process of iteration that includes your client, bringing them into the design process as a value-added service?
Imagine an immersive viewing room where AI-generated images move slowly across the walls in succession. Thematic music fills the air to create a sensory experience. The images get tweaked and tuned in real time based on which images most resonate with them, using eye-tracking technology and heart rate variability data. By the time they step out of the viewing, they have an image in hand which is better than you could have created upfront.
3. How can I find and access new, unconventional sources of data?
One of the clearest lessons of recent AI models is that it takes an enormous amount of data to train them. There are serious concerns that humanity’s total supply of data will only last so long. We’ll have to constantly find and create more.
Given that fact, how can you put yourself in a position of securing access to new, untapped, hidden, secret, or quickly changing information? Perhaps it’s data about the natural world, such as how forests are growing or the movement of deep-sea oil reserves. Maybe it’s highly sensitive data, like people’s deepest fears or secret preferences (acquired ethically I hope).
All that data can be stored in a system of personal knowledge management – which I call a Second Brain – so you always have it ready to feed into any AI (or human) you’re interacting with. We offer in-depth training in how to build such a system as a best-selling book and a self-paced course.
Until AI can directly access the full richness of reality, both in the physical world and the psychological one, it will depend on us to learn and improve its understanding of the world.
4. What is the smallest number of decisions I can make to achieve my mission?
This one may seem counterintuitive, but in the future, any decision that requires a human in the loop will be far slower, more expensive, and more complex (and maybe in some situations even more risky) than one handled purely by AI. Decisions that can be made solely by AI will happen at AI speed, whereas human decisions will have to proceed through the much slower world of human minds, relationships, and institutions.
Given this new predictive superpower we now have access to, there will be tremendous value in structuring an activity so that it requires as few human decisions as possible.
Imagine two non-profit microfinance organizations working to end poverty in Sub-Saharan Africa. They are both well-intentioned, well-resourced, and good at what they do. But one of them has automated the process of deciding who should be given a micro-loan using AI.
Over time they are likely to make lending decisions faster, improve the accuracy of those decisions by incorporating data on the results, changing how the staff operates to further improve the algorithm, and using flexible lending criteria that takes into account more variables than any human could consider.
This AI-driven nonprofit could well achieve a more effective, fair, and customized lending portfolio than its counterpart, leading to lower costs that further frees up funds for more lending. By minimizing the surface area of their organization that is exposed to humans, and thus the decisions that humans have to make, they could greatly accelerate their impact.
5. What information, if supplied to AI, would allow me to make better predictions?
Predictions are like precisely engineered products, finely tuned for very specific purposes. Although we are early in the AI revolution and a few general-purpose platforms like GPT-4 are dominating the industry, that is likely to change as more companies enter.
Even if it doesn’t, there will always be a need for us to provide the context and details the AI needs to make a prediction. This is why almost any ChatGPT prompt can benefit from more rules and guidelines, specific details, background context, and examples for how to understand what you’re asking.
Most of our information management tasks, from reading to notetaking to file management to content curation to social media, will have to change in a world shaped by AI. We’ll need to think about the apps and platforms we use to orchestrate all that information as a system, and consciously design it to feed us just the right information we need to craft the next AI prompt.
The better the quality of the data you have access to, and the easier it is to access it, the more you will find yourself empowered and unleashed by AI, rather than threatened by it.
Follow us for the latest updates and insights around productivity and Building a Second Brain on Twitter, Facebook, Instagram, LinkedIn, and YouTube. And if you’re ready to start building your Second Brain, get the book and learn the proven method to organize your digital life and unlock your creative potential.
- POSTED IN: Artificial Intelligence, Cognitive science, Futurism, Habits, Technology