We perpetually interact with our technologies. On the one hand they serve us, and on the other hand they control us.¹ Computers, smartphones, and the infrastructure surrounding them now mediate much of our communications, affecting not only whom we can reach and who can reach us but also what we can say and what we can hear. Our communications tools free our language and our thinking and also govern them. Our technologies affect not only how we “make sense” but also what we mean by “making sense.”

The proliferation of sensors, smart-connected products (Internet of Things), the measurements they generate (big data), on-demand computing (the cloud), and pattern-finding software (AI) are changing how individuals and organizations interact. New distributed structures challenge established centralized organizations. Boundaries between inside and outside are blurring. And everywhere, more and more of what we do is recorded.

As we design with these new technologies, they offer new tools and new materials on which to work, but they also change the design process and the roles designers play in it.

##Technologies driving the data economy

The information revolution emerged over the last 50 years, as a series of technology waves transformed the way we communicate, do business, and organize our lives. Computers evolved from rarified research tools, to corporate mainframes, to department mini-computers, to personal computers for individuals. The PC made business digital. The Internet connected everything. And smartphones made computing ubiquitous: always on and always connected.

Now we are on the cusp of another technology wave at least as large as the previous waves of the PC, the Internet, and the smartphone. Surprisingly, this new wave does not yet have a commonly agreed upon name. Google CEO Sundar Pichai calls it “AI First,” building on the mantra “mobile first” and suggesting that the focus of software design and development, which moved from desktop to mobile, is now moving to AI.² Siri cofounder Dag Kittlaus has called it the era of “assistants.”³ Tech marketers have promised “digital transformation,” “social business,” and “a smarter planet.” The Economist has proclaimed that data will be for the twenty-first century what oil was for the twentieth century—an enabler of new technologies, new products, and new businesses.⁴ Each of these terms is a lens on the new wave, though none fully describes it.

The new wave is driven by five core technologies, each combining with the others and reinforcing them all.

Sensors — In 2016, about 1.5 billion smartphones were sold, each chock-full of sensors: a touch screen, a camera or two or three, a microphone, moisture sensor, proximity sensor, light sensor, motion sensor, and more. That’s a lot of sensors. The huge volumes drive down costs, and the race for features drives up innovation. Most sensors are printed on chips, making them very small, leading to sensors being installed all around us (e.g., 400,000 CCTV cameras in London alone), on us (e.g., activity trackers and smart fabrics), and even in us (e.g., ID chips for pets and blood glucose sensors for people).

IoT — The Internet of Things (also GE’s Industrial Internet and Cisco’s Internet of Everything) refers to sensors and machines connected to the Internet—everything from your home thermostat to private satellites to your car. In addition to sensors and communications chips, IoT devices contain microprocessors, making them “smart.” These “smart-connected products” communicate via the Internet with centralized services, sharing local data with “headquarters” and receiving instructions in return. The centralized services often support continuous monitoring, remote diagnostics, process control, predictive analytics, and soon autonomous operations. Along the way, they also generate a flood of data.

Big data — Measuring has become big business (see also Trend: Resilient Organizations). Three examples: 1) A Nest Learning Thermostat (a special purpose computer) records every change you make to its settings. 2) In the last five years, ~500 satellites were launched; in the next 5 years, ~5,000 will be launched. Currently, each pixel in Google Earth is updated, every ~2 years; in the next 5 years, it may be every 20 minutes. In addition to increasing the frequency of observations, satellite resolution is increasing, too, from 15–30 meters per pixel to 10–30 cm and higher. Satellites generate petabytes of data per year (a million gigabytes). 3) Closer to home, the average car contains roughly 30 microprocessors, 80 sensors, and 100 million lines of code. Each car may generate a terabyte of data per day. With 1 billion cars in the world, that’s about a zettabyte of data per day (one million petabytes). Not all of it goes to the cloud. Not yet.

In addition to physical sensors, “virtual sensors” collect oceans of data, too. Pretty much every action anyone takes online is logged. What did you search for? Which link did you click? How long did you stay on a page? That’s oceans of data about customers’ aggregate choices and your particular activities; all of it, just begging to be analyzed.

The cloud — On network diagrams, engineers represent connections to the Internet as lines to a cloud icon. “Cloud computing” refers to the massive processing and storage resources offered as on-demand Internet services by Amazon, Google, Microsoft, and others. Yet again, increases in scale have reduced unit cost—this time driving the marginal cost of computer processing to almost nothing—and have turned processing into a “utility” much like electric power. What this means is that even the smallest startup can spin-up a super-computer-in-the-cloud (i.e., a distributed system with thousands or tens of thousands of cores).

And in turn, it means Amazon, Facebook, Google, and a few others have become “data refineries,” with processing power rivaling the U.S. intelligence agencies. All this processing power can be brought to bear to find patterns in big data—if you have the right software.

AI — Artificial intelligence refers to a range of software technologies from common statistical methods to convolutional neural networks, sometimes called machine learning (ML) or deep learning (DL). What AI systems are learning is to see and speak—computer vision (CV) and natural language processing (NLP). At heart, AI systems find patterns. That is, they match chunks of data. Recent advances have come not so much from improved algorithms as from the huge amounts of data now available to the algorithms. Self-driving cars work largely because Google and others have collected tons of data on driving, not because of significant breakthroughs in AI software.

All five technologies— sensors, IoT, big data, the cloud, and AI—work together in a larger system, what we might call an “AI platform” or, more prosaically, a “data refinery.” Sensors in smart-connected products collect data on their environment, users, and usage. They send data to servers in the cloud. There, vast server “farms” offer high-volume computer processing on demand—low-cost, distributed super-computers that apply software algorithms to find patterns in the data—and improve daily operations.

All of this has changed how individuals and organizations interact.

##New structures in the data economy

Such data refineries have been evolving on the web for more than 10 years. Amazon predicts which books you will buy. Facebook predicts which posts you will read. Google predicts which ads you will click. Netflix predicts which movies you will watch. Palantir predicts whether you belong to a gang (and notifies your local police). And a host of finance companies predict the likelihood they will be paid for each transaction, before deciding to approve or decline your next purchase.

In a sense, we have entered “willingly” into the panopticon (though perhaps not entirely with informed consent)—an era in which everything we do is measured and recorded. In 1999, Sun Microsystems CEO Scott McNealy called privacy a “red herring,” saying, “You have zero privacy, anyway… Get over it.” In the “surveillance economy,” the guiding principle is: “If you’re not paying for the [proximate] product, you are the [ultimate] product.”⁵

Scores — Influence scores abound, not just “likes,” “up-votes,” and “page-rank,” but also your “Klout Score” (aggregated social media status), FICO score (credit rating), “airline miles” (your value to the airline, determining where you sit and when you board), “h-index” (how widely an academic author is cited), and more. Companies even have scores: for example, “Net Promoter” (how likely customers will recommend it to a friend). Now China is taking measurement to the next level, implementing a social credit system, what The Atlantic calls an “algorithmically determined . . . index of [every citizen’s] trustworthiness and virtue,” which will determine their access to, “well, everything.”⁶ While the system is still in development, already, Chinese authorities are using the social credit score to limit travel.

Digital twins — Data refineries aren’t just for tracking consumers (or citizens). They’re also for running big businesses. GE has built “digital twins”—software models of its jet engines and steam turbines—using sensor data to predict faults and head off potential trouble with preventative maintenance. GE’s smart-connected product systems even enable its customers in the electric power business to “arbitrage” maintenance. A utility may decide to defer maintenance (recognizing an increased cost later) in order to keep a generator online for an impending heat wave (which will raise electric power demand and spike spot market prices)—calculating that the value of the power generated in the heat wave will more than offset the higher maintenance costs. Siemens has a similar platform. IBM is attempting to build one for healthcare. Satellite companies are teaming with commodities traders to monitor the global network of supply chains in order to predict changes in supply and demand—in real time. Of course, governments are interested, too. The U.S. Department of Defense is building One World Terrain, “a single 3D geospatial database for use in next-generation simulations…”—and as a “comprehensive synthetic training environment” (STE).

Smart-connected products — Harvard Business School professor Michael Porter has described “how smart, connected products are transforming competition” among existing businesses—“and creating entirely new industries.”⁷ Porter also explains how individual smart-connected products can link with other smart-connected products in what he calls “product systems,” and how product systems can link with one another in “systems of systems.” As an example of a smart-connected product, Porter offers a farm tractor (an almost self-driving vehicle). The tractor forms the hub of a farm equipment system, including tillers, planters, harvesters, and more. The farm equipment system is itself part of a large farm management system, which also includes systems for seed optimization, irrigation, weather, and perhaps hedging and trading.

Carnegie Mellon University computer science and HCI professor Jodi Forlizzi has called such systems “product-service ecologies,” emphasizing their organic nature and the interaction of many actors in complex networks (see also Trend: Aggregation and Curation).

Platforms — Product-service ecologies, smart-connected product systems, and the larger data economy are built on platforms—systems that leave room for others to add functionality and create value. For example, Apple sells its iPhone platform to customers who download and run apps made by third-party developers. Successful platforms nurture reciprocal relationships. Apps attract users, users attract developers, and developers make apps. More users means more developers, which means more apps—reinforcing the cycle by attracting still more users to the platform.

Platforms often rest on other platforms in a “stack.” A Facebook app runs on Facebook, which runs on browsers, which are themselves apps running on an OS, which runs on microprocessors. A Facebook app may also run on phone operating systems, which run on phones. And a Facebook app also runs on a cloud-based “back-end,” which has its own stack of platforms (server, database, OS, hardware). All these elements together (and even more to support advertising) comprise the Facebook product-service ecology.

Blurring boundaries — In the data economy, boundaries are not always clear or fixed. Competitors may also be collaborators. Suppliers may also be customers. Employees may also be constituents whose wishes matter. And vice versa.

In an earlier era, producers and consumers exchanged goods for money. Little else was involved. Transactions were mostly anonymous, one-time events. Now, anonymity is disappearing as businesses collect data on every interaction with customers, and one-time transactions give way to ongoing relationships. At the same time, new communications technologies are giving consumers more access to one another and to the people who run businesses, further blurring boundaries between inside and outside. In many cases, a purchase becomes membership in a brand—a decision to join a tribe.

##Changes in making sense

Technology affects what we can say and hear, which affects what counts as “making sense.”

University of California, Berkeley linguistics professor George Lakoff has described two ways of explaining change: “direct causation” and “systemic causation.”⁸ Direct causation focuses locally. Change in X causes change in Y. Much of science and engineering is built on understanding direct causation (e.g., bacteria cause sickness). Systemic causation looks more globally. Change in X may lead not only to change in Y but also to change in Z and A and B and C and a “cascade” of other changes (e.g., trophic cascades, cell-signaling pathways, quorum sensing, and what philosopher Gilles Deleuze calls “rhizomes” of thought and culture). Also, the change in X that led to change in Y may lead back to change in X, in a feedback loop (e.g., penicillin may kill bacteria and increase the resistance of bacteria). Increasingly, the data economy requires us to think systemically and holistically (see also Trend: Complex Problems).

Designer Manuel Lima has described a similar shift in structures of knowledge—from hierarchies (trees) to networks (webs).⁹ For example, we used to think in terms of org charts (a tree showing who works for whom); now we think in terms of social networks (webs showing the many ways people are connected). We used to think of the “tree of knowledge” (a taxonomy classifying everything we know); now Google is building a “knowledge graph” (a web defining relationships between concepts). Increasingly, the data economy requires us to think in terms of networks and relationships.

Science itself—our preeminent way of explaining things—is undergoing a shift. Sensors, IoT, big data, the cloud, and AI are changing science, too. Data, once scarce, is now super-abundant. Extrapolating from small samples is becoming continuous measurement of everything. Answering simple questions has become multivariate analysis.

The “new science” is less about discovering “universal” laws and more about building models that evolve in real-time as ever more data becomes available. The new science is emerging first in large organizations, because it requires infrastructure. Yet the infrastructure of the cloud will make the new science accessible to small teams and even individuals. In fact, “citizen scientists” have a bigger role to play than ever before (see also Trend: Aggregation and Curation).

What’s more, we will all become “Sunday scientists”—performing millions of little experiments on ourselves, as we use the super-abundance of data, cloud computing, and AI-as-a-service to find patterns and improve our health, finances, and happiness.

On the one hand, these tools promise more access more quickly (e.g., as peer review becomes publishing preliminary results). On the other hand, the challenge to central authorities by distributed networks will make discerning fact from fiction increasingly difficult.

##Changes in design practice
The changes in technology that are changing the economy—and changing how we communicate and what it means to make sense—are also changing both what we design and how we design.

What we design — The traditional concern of designers (i.e., the form of objects) has broadened to include the structure of systems (i.e., smart-connected products and product-service ecologies). We are designing “finished” products less and less. Instead, we are designing platforms—creating opportunities in which others can design—performing a sort of “meta-design.”

We are also grappling with new opportunities. Data is becoming a fundamental resource—a material of design. Software architect Tim Misner has suggested that defining what data will be needed (in order to design the next generation of a product or to evolve a product during operation) will become a key part of the design process. Likewise, AI is becoming a fundamental resource—a material of design. Finding opportunities to find patterns and build models will become a key part of the design process, as well. One result will be new modes of interacting with computers, including interfaces that are more collaborative and more like conversation with people.

How we design — Designers are also recognizing they are enmeshed in the systems they are trying to design. Like natural ecologies, product-service ecologies cannot be “designed” in the traditional sense—they cannot be planned completely in advance by someone at “the top.” Rather, designers (and software developers and product managers) must create conditions from which successful platforms and ecologies may “emerge,” enabling them to grow over time as many factors interact.

The fact that designers are enmeshed in the systems they are trying to design suggests shifts in several dimensions of design practice:

Going forward, designers also face a number of issues related to ethics. We are already grappling with safeguarding data and privacy. What is “informed consent” in this context? Should individuals own all the data their actions generate? Recently, preventing “theft of attention” (incentives that addict users to services) has become something of a cause. Yet, among some designers, “behavior change” is still discussed as an aspiration. Bias in algorithms has also been in the press, along with questions about transparency. What exactly are the algorithms doing? What factors are they using to make decisions? Questions about the consequences of automation and AI rise up occasionally, but with little serious debate. In the United States, 13 million people earn their living by driving. What happens when their jobs are replaced by self-driving vehicles? The need for augmentation (increasing abilities, instead of efficiencies) deserves much more attention.

These and other issues related to ethics will challenge designers in the data economy and require new competencies.

###Endnotes

1. Ashby, W. R. (1956). An Introduction to Cybernetics. New York, NY: John Wiley and Sons, Inc.
2. Boulding, K. (1956). “General Systems Theory: The Skeleton of Science.”
3. Dubberly, H. (2018). Designing Within Systems. Presentation at Rochester Institute of Technology.
4. Dubberly, H. (2017). “Datafication” — The Rise of Big Data and the Application of AI to Everything. Presentation at Stanford University.
5. Dubberly, H. (2014). A Systems Literacy Manifesto. All Rights Reserved © 2018 AIGA.
6. Forlizzi, J. (2008). “The Product Ecology: Understanding Social Product Use and Supporting Design Culture.” International Journal of Design, Volume 2, Number 1.
7. Kelly, K. (2017). The Inevitable: Understanding the 12 Technological Forces That Will Shape Our Future. New York, NY: Penguin Books.
8. Pangaro, P. (2017). Conversation Is More Than Interface. Presentation at Interaction Design Association Conference.
9. Pangaro, P. (2015). Interaction Design and 21^st Century Work.

Originally published in AIGA Design Futures. View Original Publication

An introduction to AIGA Design Futures. View Introduction

Download PDF