A thesis on software eating the world
This is an unauthorized summary of the 30,000-word blog series Breaking Smart, by Venkatesh Rao of ribbonfarm, which I believe to be among the most important writing in recent years on innovation, productivity, and problem solving.
The series attempts to answer the question “What exactly does it mean to say ‘software is eating the world’?”, and ends up laying out a new framework for how to understand and promote innovation in companies, in society, and in life.
It is a massively condensed summary, with a mix of direct quotes, paraphrasing, rewording, my own commentary, and MANY details and examples left out, which means there are likely big gaping holes. To fill them, go read the full version here.
In the past, companies were designed to predict and plan for the future. But the faster things change, the less value in planning and prediction, for three reasons:
- You may get it wrong, leading to wasted effort and resources
- The more resources you spend, the more attached you get to the sunk costs of your vision
- You end up suppressing the chaos and serendipity that are the essence of innovation
The results are common across all big, centralized projects in the modern world: massive cost overruns, extended delays, failed launches, damaging unintended consequences, and broken, unusable systems.
These failures don’t happen despite planning and prediction, they happen because of them.
They happen because we insist on planning and predicting a world that is no longer predictable.
In the slow-moving, traditional business environment, problem solving was a straightforward 3-step process:
- Problem selection: choose a clear and important problem
- Resourcing: capture resources by promising to solve it
- Solution: solve the problem within promised constraints
This process is so familiar, you’re probably thinking “Isn’t that the definition of problem solving?”
But this “waterfall” approach gets more problematic the more change accelerates, for two reasons:
- Rigidity
Choosing a problem based on “importance” usually means accepting established ways of looking at and thinking about it. Defining a realistic vision of what “success” means is necessary to get people on board, but this often means limiting creative possibilities for those who come later.
2. Positive uncertainty
We often think of uncertainty in business as a negative, but it doesn’t have to be. Uncertainty can also bring unexpectedly good ideas and novel ways of solving problems. But staying too fixated on a vision and the step-by-step plan to get there makes it hard to take advantage of these things. Problems have to be solved in pre-approved ways. Sticking to “the plan” ends up turning any source of uncertainty into a negative. Not a good strategy in a world of accelerating change.
Now that we’ve torn down the “old way,” what would it look like to solve problems in such a way that it gets better the faster we go?
In three words:
Immersion. Experimentation. Leverage
Immersion means intentionally exposing yourself to streams of ideas, people, and new capabilities, not with the goal of knowing everything, but to stay sensitized to developing opportunities and threats. The faster the stream goes, the more ideas and information you get exposed to.
Experimentation recognizes that our technological, networked world presents us with quickly falling downsides (for example, the cost of starting a business, or the costs of failure), as well as rapidly expanding upsides (opportunities to reach more people in less time than ever before).
The only strategy that makes sense in such a situation is placing many small bets in many different directions, in hopes of riding the huge upside of any idea that happens to succeed. More experimentation leads to more failure, but that’s actually a good thing: the faster you fail, the faster you learn, because it is usually faster and cheaper to learn from failure than to attempt to anticipate and plan for every single thing that could go wrong.
Leverage is the ability to rapidly shift resources to new, more fruitful directions. This ability becomes paramount when the best path forward cannot be predicted, and even when you find it, it can change quickly and unexpectedly.
That’s the short version. Read on for more details and examples.
So what can you do to promote this type of problem solving?
This is the big question to be answered. Otherwise this is just an amusing theory.
Immersion
The dominant paradigm for understanding the world has switched from “containers” to “streams.”
Containers are the boxes we draw around things to understand them. Industries, companies, departments, semesters, packages, silos — these are the units we break the world down into. They served us well in making sense of a world that was relatively static, simple, and slow-changing.
The problem with containers is that they encourage us to view reality in what behavioral psychologists call “functionally fixed” ways: people, ideas, and things have fixed, single meanings. But now the world is better understood via the metaphor of “streams.”
A formal definition of a stream is an “open, non-hierarchical flow of real-time information from multiple overlapping networks.” Think of social media. Twitter is not a single monolithic network; it is multiple overlapping networks, each person occupying a unique intersection of friends, colleagues, thought leaders, admirers, brands, and celebrities, as well as ideas, jokes, memes, and hashtags.
Bringing this idea back to the broader world, a stream is “a life context formed by all the information flowing towards you via a set of trusted connections — to free people, ideas and resources — from multiple networks, both technological and not.”
Think of the main streets of a thriving city. Commercial networks overlap with transport networks, political networks, family networks, friendship networks, environmental networks, etc. All these networks not only coexist, but actually enhance each other, constantly presenting unrelated people, ideas, and resources in unexpected juxtapositions.
This is the source of the creative buzz you feel walking through downtown Manhattan. Just strolling down the street you are confronted with explosively original combinations of textures, sounds, images, languages, people, atmospheres. The combinatorial possibilities are endless enough that people with completely different tastes and lifestyles can occupy the same physical spaces, finding inspiration in different facets of the stream.
It’s worth noting that much of the incredible creativity and resilience of cities — they continue to survive and thrive even through war, disaster, and economic and sociological disruption — comes from this diversity of networks.
Coming back full circle to technology, this explains why the Internet as a whole exhibits a similar creativity and resilience. Physical and social constraints break down as everyone talks to everyone, across time and space and social class. Niche communities that could never meet in person find a gathering place online. The voices of dissidents and outsiders can be heard just as easily as the biggest brands. We shouldn’t be surprised that the Internet, like cities, is simultaneously one of the greatest sources of innovation in the world today, and also virtually impossible to destroy.
Individuals and companies in this new era innovate by exposing themselves to as many different streams as possible, especially ones that provide:
- rich feedback
- self-correction
- ongoing improvement
- continuous learning
This goes beyond attending events and reading newsletters. The most innovative businesses, instead of controlling access and locking down their most valuable assets, make them as open and accessible as possible. Tesla realized that, to innovate as quickly as possible, they needed not only to obtain access to the best ideas and engineers, they needed to give access to their patents to the broader industry, even their competitors.
They understood that true innovation doesn’t happen in secret R&D labs; it happens through thriving ecosystems with as many diverse people as possible working on a problem in as many different ways as possible. This is sometimes called the “luck of networks” — by increasing the size and scope of the network, you make your own luck.
The Open Source movement, Google’s Android operating system, Big Data technology, the Arduino hardware experimentation kit, and the OpenROV underwater robot are all examples of extremely difficult problems being solved not by meticulous planning and prediction, but freewheeling communities of tinkerers.
Experimentation
There is a consequence of accelerating change that is not immediately obvious: the faster things change and improve, the more your current thinking represents a constraint on the future efforts of yourself and others.
Your capabilities will be so reliably greater a year from now than today, it makes sense to replace at least some long-term planning around current constraints with a different activity that provides greater short-term rewards: tinkering. We can define “tinkering” as trial-and-error, iterative improvement. It is testing and adaptation designed to uncover unexpected new opportunities, without fixed goals.
But tinkering can equally be defined as “play,” and in this way it mimics the random walk of children’s games. This is an uncomfortable analogy in professional contexts (we don’t like the idea that we go to work to “play games”), but it reflects a historical insight: the future of work lies in today what we consider play. The well-paid virtual reality designers of today were the “video game addicts” of yesteryear. The drone pilots of today were the model airplane enthusiasts of the past. There is someone in a garage somewhere with an amusing hobby that will one day be a multi-billion dollar industry.
An important aspect of tinkering is that it is driven by people’s natural curiosity; what tantalizes and draws them for reasons they cannot necessarily explain. Following this curiosity leads in the direction of maximal interestingness — a sort of hunch that asking a certain question will lead to fruitful answers. This is the true purpose of the much vaunted “collaboration” — it is a mechanism for averaging and combining the interestingness instincts of people with diverse backgrounds and skills, in the hope that it will lead to profitable opportunities that no one would find on their own.
Examples of this type of open collaboration include opt-in staffing of projects (allowing people to choose their work based on what interests them), promiscuous forking (creating parallel versions of a project to allow them to evolve in different directions), and teams drawn from across roles, functions, and departments (to combine different flavors of curiosity).
A key concept to ensure that this experimentation process remains connected to the real world is RERO (Release Early, Release Often). The temptation and danger in an environment of accelerating change is to wait longer before releasing your creation into the world. You want to run one more test, add one more feature, make the quality just a little higher, gather just a bit more data, fix one more bug, and then you’ll be ready.
This tendency is dangerous because the outside world doesn’t stop changing as you add your finishing touches; it goes on evolving just as quickly and unpredictably. And the longer you wait to release, the longer it takes to capture the feedback and learning that represent the real metrics of success in the marketplace.
The RERO principle counteracts this tendency by scheduling small, frequent releases. The concept originates in software, where updates can be pushed over the air at low cost. But it is rapidly becoming necessary in all industries. The only way for execution to track rapidly shifting priorities is to get constant, concrete feedback from your users. The only way to do that is to give them new product updates frequently. The only way to do that is to build this rapid release cycle into the culture and product development process itself.
Making experimentation the core of your business paradoxically requires developing the ability to abandon failed experiments as quickly as possible, thereby minimizing sunk costs.
Leverage
In the past, the concept of leverage (doing more with less) was associated with the strengths of the traditional corporation: economies of scale, accumulated financial capital, market power, brand equity. In modern times these strengths are just as often liabilities, as they make it more difficult to adapt to changing conditions.
This principle attempts to explain the new forms of leverage that are arising as software eats the world and large companies along with it.
Until now we’ve ignored the role of corporate structure and strategy. Immersion and Experimentation are largely bottom-up, decentralized phenomena. But there is still a role for leaders: to align people and resources behind the experiments that happen to succeed. The most innovative idea in the world is worthless if it is allowed to die on the vine.
The question becomes “How can leaders facilitate decision making without stifling the randomness essential to breakthrough ideas?”
What’s needed is a change in how leaders view their role. Traditionally, consensus-seeking was about getting everyone to agree on a detailed long-term vision, while leaving immediate next steps fuzzy. As our time horizon moves closer and closer, it makes more sense to do the reverse: define next steps clearly, while deliberately leaving the long-term vision as fuzzy as possible.
This not only avoids wasting resources on long-term plans that are likely to change anyway, it’s also much easier to get people to reach rough consensus than to update a complex, detailed plan. Both these advantages allow you to take action quicker, like the startups everyone so admires.
The reason this feels scary is that the purpose of long-term planning is to uncover risks and constraints. The assumption is that by uncovering them, the company will be able to mitigate them. But in an era of accelerating change, predicting the risks is impossible. In an era of rapidly expanding capabilities through technology, the constraints you identify may not be as limiting as you think.
It now makes more sense to have an abundance mindset that assumes rapidly expanding resources, capabilities, and opportunities. This lens allows you to see the new forms of leverage popping up everywhere. There is no definitive list of “leverage points” in this new environment. The idea is to create your own.
Google Captcha utilizes a previously “wasted” resource — the few seconds of people’s attention used to decipher captchas and prove they’re not a computer — for a very useful purpose: deciphering the street addresses captured by its Street View vehicles.
For a more basic example, consider the humble hashtag. It takes no special skills or tools, hardly any knowledge of computers, is not limited to any particular platform or social network, cannot be patented or copyrighted, and is available to anyone at any time. Yet if a particular hashtag goes viral, it will quickly reach a number of people greater than the entire global publishing industry of a century ago. By creating a hashtag, any individual has at least the potential to publish and distribute their ideas through a soft network piggybacking on the global web.
This is a manifestation of Gall’s Law, from the study of complex systems:
“A complex system that works is invariably found to have evolved from a simple system that worked. A complex system designed from scratch never works”
Someone could create a centralized R&D department somewhere full of PhDs, behavioral science experts, and powerful supercomputers, with the mission of generating new viral ideas “from scratch.” But I guarantee you they could not compete in terms of quality OR quantity with the citizen scientists, Do-It-Yourselfers, self-trackers, guerrilla marketing agencies, biohackers, etc. that each day publish more content online than you could consume in a lifetime.
The best ideas have to be grown, not built. The greenhouse is replacing the factory floor.
The Luck of Networks
You may have wondered what the title of this series, “Breaking Smart,” refers to. My interpretation is that this new approach to problem solving implies a radical shift: being “smart” is no longer about raw brainpower, or having all the answers.
The greatest source of leverage in the world today is network effects, but harnessing it takes a new mindset and set of skills:
It requires reacting to uncertainty by opening things up, instead of locking them down, so that more unexpected things can happen. Witness the #ASAPbio movement of research scientists pushing to publish their results online instead of waiting six months or more for journal publication.
It requires getting good at breaking assumptions about how resources can be used, about questioning zero-sum thinking that sees all resources in terms of their scarcity. Uber and Airbnb unleashed billions in economic value not by creating new products, but increasing the utilization rate of our largest assets that sit unused the majority of the time — our cars and houses.
It requires staying open to and recognizing serendipitous solutions to problems — solutions that may be so original and out-of-left-field that they may not even look like solutions at first. The rise of video and teleconferencing has probably done more to alleviate traffic and pollution than any environmental initiative, yet this sort of indirect impact is rarely appreciated.
The smartest way to solve a problem is no longer to attack it directly. Problems are too complex, interconnected, and chaotic to yield to brute force, analytical attack. You have to create a network to defeat a network.
The smart approach is to use surplus and spillover effects: to fill the vessel with enough hyper-reactive chemicals and catalysts to maximize the chances that it will boil over with interesting compounds. If you’re lucky it will become auto-catalytic, feeding itself through cycles of growth and decay. If you’re really lucky you’ll be standing by to benefit from the compounds that emerge, not controlling but feeding the reaction.
Luck may seem a strange thing to depend on for business or professional success. Luckily, you can make your own luck to a certain degree, by making your information and your organization as transparent, searchable, and hackable as possible. It is the organizations that open up and dissolve internal and external barriers the fastest that will succeed in this new environment. By doing so, the very definition of what the company is will become inseparable from the networks and streams in which it moves.
The best way to surf the stream, it seems, is to become part of the stream.
Follow us for the latest updates and insights around productivity and Building a Second Brain on Twitter, Facebook, Instagram, LinkedIn, and YouTube. And if you're ready to start building your Second Brain, get the book and learn the proven method to organize your digital life and unlock your creative potential.
- POSTED IN: Book summary, Creativity, Entrepreneurship, Futurism, Strategy