The
voice-activated gadget in the corner of your bedroom suddenly laughs
maniacally, and sends a recording of your pillow talk to a colleague. The clip
of Peppa Pig your toddler is watching on YouTube unexpectedly descends into
bloodletting and death. The social network you use to keep in touch with old
school friends turns out to be influencing elections and fomenting coups.
Something
strange has happened to our way of thinking – and as a result, even stranger
things are happening to the world. We have come to believe that everything is
computable and can be resolved by the application of new technologies. But
these technologies are not neutral facilitators: they embody our politics and
biases, they extend beyond the boundaries of nations and legal jurisdictions
and increasingly exceed the understanding of even their creators. As a result,
we understand less and less about the world as these powerful technologies
assume more control over our everyday lives.
Across the
sciences and society, in politics and education, in warfare and commerce, new
technologies are not merely augmenting our abilities, they are actively shaping
and directing them, for better and for worse. If we do not understand how
complex technologies function then their potential is more easily captured by
selfish elites and corporations. The results of this can be seen all around us.
There is a causal relationship between the complex opacity of the systems we
encounter every day and global issues of inequality, violence, populism and
fundamentalism.
Instead of a
utopian future in which technological advancement casts a dazzling,
emancipatory light on the world, we seem to be entering a new dark age
characterised by ever more bizarre and unforeseen events. The Enlightenment
ideal of distributing more information ever more widely has not led us to
greater understanding and growing peace, but instead seems to be fostering
social divisions, distrust, conspiracy theories and post-factual politics. To
understand what is happening, it’s necessary to understand how our technologies
have come to be, and how we have come to place so much faith in them.
In the 1950s,
a new symbol began to creep into the diagrams drawn by electrical engineers to
describe the systems they built: a fuzzy circle, or a puffball, or a thought
bubble. Eventually, its form settled into the shape of a cloud. Whatever the
engineer was working on, it could connect to this cloud, and that’s all you
needed to know. The other cloud could be a power system, or a data exchange, or
another network of computers. Whatever. It didn’t matter. The cloud was a way of
reducing complexity, it allowed you to focus on the issues at hand. Over time,
as networks grew larger and more interconnected, the cloud became more
important. It became a business buzzword and a selling point. It became more
than engineering shorthand; it became a metaphor.
Today the
cloud is the central metaphor of the internet: a global system of great power
and energy that nevertheless retains the aura of something numinous, almost
impossible to grasp. We work in it; we store and retrieve stuff from it; it is
something we experience all the time without really understanding what it is.
But there’s a problem with this metaphor: the cloud is not some magical faraway
place, made of water vapour and radio waves, where everything just works. It is
a physical infrastructure consisting of phone lines, fibre optics, satellites,
cables on the ocean floor, and vast warehouses filled with computers, which
consume huge amounts of water and energy. Absorbed into the cloud are many of
the previously weighty edifices of the civic sphere: the places where we shop,
bank, socialise, borrow books and vote. Thus obscured, they are rendered less
visible and less amenable to critique, investigation, preservation and
regulation.
Over the last
few decades, trading floors around the world have fallen silent, as people are
replaced by banks of computers that trade automatically. Digitisation meant
that trades within, as well as between, stock exchangescould happen faster and
faster. As trading passed into the hands of machines, it became possible to
react almost instantaneously. High-Frequency Trading (HFT) algorithms, designed
by former physics PhD students to take advantage of millisecond advantages,
entered the market, and traders gave them names such as The Knife. These
algorithms were capable of eking out fractions of a cent on every trade, and
they could do it millions of times a day.
Something
deeply weird is occurring within these massively accelerated, opaque markets.
On 6 May 2010, the Dow Jones opened lower than the previous day, falling slowly
over the next few hours in response to the debt crisis in Greece. But at
2.42pm, the index started to fall rapidly. In less than five minutes, more than
600 points were wiped off the market. At its lowest point, the index was nearly
1,000 points below the previous day’s average, a difference of almost 10% of
its total value, and the biggest single-day fall in the market’s history. By
3.07pm, in just 25 minutes, it recovered almost all of those 600 points, in the
largest and fastest swing ever.
In the chaos
of those 25 minutes, 2bn shares, worth $56bn, changed hands. Even more
worryingly, many orders were executed at what the Securities Exchange
Commission called “irrational prices”: as low as a penny, or as high as
$100,000. The event became known as the “flash crash”, and it is still being
investigated and argued over years later.
One report by
regulators found that high-frequency traders exacerbated the price swings.
Among the various HFT programs, many had hard-coded sell points: prices at
which they were programmed to sell their stocks immediately. As prices started
to fall, groups of programs were triggered to sell at the same time. As each
waypoint was passed, the subsequent price fall triggered another set of
algorithms to automatically sell their stocks, producing a feedback effect. As
a result, prices fell faster than any human trader could react to. While
experienced market players might have been able to stabilise the crash by
playing a longer game, the machines, faced with uncertainty, got out as quickly
as possible.
Other theories
blame the algorithms for initiating the crisis. One technique that was
identified in the data was HFT programmes sending large numbers of
“non-executable” orders to the exchanges – that is, orders to buy or sell
stocks so far outside of their usual prices that they would be ignored. The
purpose of such orders is not to actually communicate or make money, but to
deliberately cloud the system, so that other, more valuable trades can be
executed in the confusion. Many orders that were never intended to be executed
were actually fulfilled, causing wild volatility.
Flash crashes
are now a recognised feature of augmented markets, but are still poorly
understood. In October 2016, algorithms reacted to negative news headlines
about Brexit negotiations by sending the pound down 6% against the dollar in
under two minutes, before recovering almost immediately. Knowing which
particular headline, or which particular algorithm, caused the crash is next to
impossible. When one haywire algorithm started placing and cancelling orders
that ate up 4% of all traffic in US stocks in October 2012, one commentator was
moved to comment wryly that “the motive of the algorithm is still unclear”.
At 1.07pm on
23 April 2013 Associated Press sent a tweet to its 2 million followers:
“Breaking: Two Explosions in the White House and Barack Obama is injured.” The
message was the result of a hack later claimed by the Syrian Electronic Army, a group affiliated to
Syrian president Bashar al-Assad. AP and other journalists quickly flooded the
site with alerts that the message was false. The algorithms following breaking
news stories had no such discernment, however. At 1.08pm, the Dow Jones went
into a nosedive. Before most human viewers had even seen the tweet, the index
had fallen 150 points in under two minutes, and bounced back to its earlier
value. In that time, it erased $136bn in equity market value.
Computation is
increasingly layered across, and hidden within, every object in our lives, and
with its expansion comes an increase in opacity and unpredictability. One of
the touted benefits of Samsung’s line of “smart fridges” in 2015 was their
integration with Google’s calendar services, allowing owners to schedule
grocery deliveries from the kitchen. It also meant that hackers who gained
access to the then inadequately secured machines could read their owner’s Gmail
passwords. Researchers in Germany discovered a way to insert malicious code
into Philips’s wifi-enabled Hue lightbulbs, which could spread from fixture to
fixture throughout a building or even a city, turning the lights rapidly on and
off and – in one possible scenario – triggering photosensitive epilepsy. This
is the approach favoured by Byron the Bulb in Thomas Pynchon’s Gravity’s
Rainbow, an act of grand revolt by the little machines against the tyranny of
their makers. Once-fictional possibilities for technological violence are being
realised by the Internet of Things.
In Kim Stanley
Robinson’s novel Aurora, an intelligent spacecraft carries a human crew from
Earth to a distant star. The journey will take multiple lifetimes, so one of
the ship’s jobs is to ensure that the humans look after themselves. When their
fragile society breaks down, threatening the mission, the ship deploys safety
systems as a means of control: it is able to see everywhere through sensors,
open or seal doors at will, speak so loudly through its communications
equipment that it causes physical pain, and use fire suppression systems to
draw down the level of oxygen in a particular space.
This is
roughly the same suite of operations available now from Google Home and its
partners: a network of internet-connected cameras for home security, smart
locks on doors, a thermostat capable of raising and lowering the temperature in
individual rooms, and a fire and intruder detection system that emits a
piercing emergency alarm. Any successful hacker would have the same powers as
the Aurora does over its crew, or Byron over his hated masters.
Before
dismissing such scenarios as the fever dreams of science fiction writers,
consider again the rogue algorithms in the stock exchanges. These are not
isolated events, but everyday occurrences within complex systems. The question
then becomes, what would a rogue algorithm or a flash crash look like in the
wider reality?
Would it
look, for example, like Mirai, a piece of software that brought down large
portions of the internet for several hours on 21 October 2016? When researchers
dug into Mirai, they discovered it targets poorly secured internet connected
devices – from security cameras to digital video recorders – and turns them
into an army of bots. In just a few weeks, Mirai infected half a million
devices, and it needed just 10% of that capacity to cripple major networks for
hours.
Mirai, in
fact, looks like nothing so much as Stuxnet, another virus discovered within
the industrial control systems of hydroelectric plants and factory assembly
lines in 2010. Stuxnet was a military-grade cyberweapon; when dissected, it was
found to be aimed specifically at Siemens centrifuges, and designed to go off
when it encountered a facility that possessed a particular number of such
machines. That number corresponded with one particular facility: the Natanz
nuclear facility in Iran. When activated, the program would quietly degrade
crucial components of the centrifuges, causing them to break down and disrupt
the Iranian enrichment programme.
The attack was
apparently partially successful, but the effect on other infected facilities is
unknown. To this day, despite obvious suspicions, nobody knows where Stuxnet
came from, or who made it. Nobody knows for certain who developed Mirai,
either, or where its next iteration might come from, but it might be there,
right now, breeding in the CCTV camera in your office, or the wifi-enabled
kettle in the corner of your kitchen.
Or
perhaps the crash will look like a string of blockbuster movies pandering to
rightwing conspiracies and survivalist fantasies, from quasi-fascist
superheroes (Captain America and the Batman series) to justifications of
torture and assassination (Zero Dark Thirty, American Sniper). In Hollywood,
studios run their scripts through the neural networks of a company called
Epagogix, a system trained on the unstated preferences of millions of
moviegoers developed over decades in order to predict which lines will push the
right – meaning the most lucrative – emotional buttons. Algorithmic engines
enhanced with data from Netflix, Hulu, YouTube and others, with access to the
minute-by-minute preferences of millions of video watchers acquire a level of
cognitive insight undreamed of by previous regimes. Feeding directly on the
frazzled, binge-watching desires of news-saturated consumers, the network turns
on itself, reflecting, reinforcing and heightening the paranoia inherent in the
system.
Game
developers enter endless cycles of updates and in-app purchases directed by A/B
testing interfaces and real-time monitoring of players’ behaviours. They have
such a fine-grained grasp of dopamine-producing neural pathways that teenagers
die of exhaustion in front of their computers, unable to tear themselves away.
Or
perhaps the flash crash will look like literal nightmares broadcast across the
network for all to see? In the summer of 2015, the sleep disorders clinic of an
Athens hospital was busier than it had ever been: the country’s debt crisis was
in its most turbulent period. Among the patients were top politicians and civil
servants, but the machines they spent the nights hooked up to, monitoring their
breathing, their movements, even the things they said out loud in their sleep,
were sending that information, together with their personal medical details,
back to the manufacturers’ diagnostic data farms in northern Europe. What
whispers might escape from such facilities?
We are able to
record every aspect of our daily lives by attaching technology to the surface
of our bodies, persuading us that we too can be optimised and upgraded like our
devices. Smart bracelets and smartphone apps with integrated step counters and
galvanic skin response monitors track not only our location, but every breath
and heartbeat, even the patterns of our brainwaves. Users are encouraged to lay
their phones beside them on their beds at night, so that their sleep patterns
can be recorded. Where does all this data go, who owns it, and when might it
come out? Data on our dreams, our night terrors and early morning sweating
jags, the very substance of our unconscious selves, turn into more fuel for
systems both pitiless and inscrutable.
Or perhaps the
flash crash in reality looks exactly like everything we are experiencing right
now: rising economic inequality, the breakdown of the nation-state and the
militarisation of borders, totalising global surveillance and the curtailment
of individual freedoms, the triumph of transnational corporations and
neurocognitive capitalism, the rise of far-right groups and nativist
ideologies, and the degradation of the natural environment. None of these are
the direct result of novel technologies, but all of them are the product of a
general inability to perceive the wider, networked effects of individual and
corporate actions accelerated by opaque, technologically augmented complexity.
In New
York in 1997, world chess champion Garry Kasparov faced off for the second time
against Deep Blue, a computer specially designed by IBM to beat him. When he
lost, he claimed some of Deep Blue’s moves were so intelligent and creative
that they must have been the result of human intervention. But we understand
why Deep Blue made those moves: its process for selecting them was ultimately
one of brute force, a massively parallel architecture of 14,000 custom-designed
chess chips, capable of analysing 200m board positions per second. Kasparov was
not outthought, merely outgunned.
By the time
the Google Brain–powered AlphaGo software took on the Korean professional Go
player Lee Sedol in 2016, something had changed. In the second of five games,
AlphaGo played a move that stunned Sedol, placing one of its stones on the far
side of the board. “That’s a very strange move,” said one commentator. “I
thought it was a mistake,” said another. Fan Hui, a seasoned Go player who had
been the first professional to lose to the machine six months earlier, said:
“It’s not a human move. I’ve never seen a human play this move.”
AlphaGo went
on to win the game, and the series. AlphaGo’s engineers developed its software
by feeding a neural network millions of moves by expert Go players, and then
getting it to play itself millions of times more, developing strategies that
outstripped those of human players. But its own representation of those
strategies is illegible: we can see the moves it made, but not how it decided
to make them.
The late Iain
M Banks called the place where these moves occurred “Infinite Fun Space”. In
Banks’s SF novels, his Culture civilisation is administered by benevolent,
superintelligent AIs called simply Minds. While the Minds were originally
created by humans, they have long since redesigned and rebuilt themselves and
become all-powerful. Between controlling ships and planets, directing wars and
caring for billions of humans, the Minds also take up their own pleasures.
Capable of simulating entire universes within their imaginations, some Minds
retreat for ever into Infinite Fun Space, a realm of meta-mathematical
possibility, accessible only to superhuman artificial intelligences.
Many of
us are familiar with Google Translate, which was launched in 2006, using a
technique called statistical language inference. Rather than trying to
understand how languages actually worked, the system imbibed vast corpora of
existing translations: parallel texts with the same content in different
languages. By simply mapping words on to one another, it removed human
understanding from the equation and replaced it with data-driven correlation.
Translate
was known for its humorous errors, but in 2016, the system started using a
neural network developed by Google Brain, and its abilities improved
exponentially. Rather than simply cross-referencing heaps of texts, the network
builds its own model of the world, and the result is not a set of
two-dimensional connections between words, but a map of the entire territory.
In this new architecture, words are encoded by their distance from one another
in a mesh of meaning – a mesh only a computer could comprehend.
While a
human can draw a line between the words “tank” and “water” easily enough, it
quickly becomes impossible to draw on a single map the lines between “tank” and
“revolution”, between “water” and “liquidity”, and all of the emotions and
inferences that cascade from those connections. The map is thus
multidimensional, extending in more directions than the human mind can hold. As
one Google engineer commented, when pursued by a journalist for an image of
such a system: “I do not generally like trying to visualise
thousand-dimensional vectors in three-dimensional space.” This is the unseeable
space in which machine learning makes its meaning. Beyond that which we are
incapable of visualising is that which we are incapable of even understanding.
In the
same year, other researchers at Google Brain set up three networks called
Alice, Bob and Eve. Their task was to learn how to encrypt information. Alice
and Bob both knew a number – a key, in cryptographic terms – that was unknown
to Eve. Alice would perform some operation on a string of text, and then send
it to Bob and Eve. If Bob could decode the message, Alice’s score increased;
but if Eve could, Alice’s score decreased.
Over
thousands of iterations, Alice and Bob learned to communicate without Eve
breaking their code: they developed a private form of encryption like that used
in private emails today. But crucially, we don’t understand how this encryption
works. Its operation is occluded by the deep layers of the network. What is
hidden from Eve is also hidden from us. The machines are learning to keep their
secrets.
How we
understand and think of our place in the world, and our relation to one another
and to machines, will ultimately decide where our technologies will take us. We
cannot unthink the network; we can only think through and within it. The
technologies that inform and shape our present perceptions of reality are not
going to go away, and in many cases we should not wish them to. Our current
life support systems on a planet of 7.5 billion people and rising depend on
them. Our understanding of those systems, and of the conscious choices we make
in their design, remain entirely within our capabilities. We are not powerless,
not without agency. We only have to think, and think again, and keep thinking.
The network – us and our machines and the things we think and discover together
– demands it.
Computational
systems, as tools, emphasise one of the most powerful aspects of humanity: our
ability to act effectively in the world and shape it to our desires. But
uncovering and articulating those desires, and ensuring that they do not
degrade, overrule, efface, or erase the desires of others, remains our
prerogative.
When Kasparov
was defeated back in 1997, he didn’t give up the game. A year later, he
returned to competitive play with a new format: advanced, or centaur, chess. In
advanced chess, humans partner, rather than compete, with machines. And it
rapidly became clear that something very interesting resulted from this
approach. While even a mid-level chess computer can today wipe the floor with
most grandmasters, an average player paired with an average computer is capable
of beating the most sophisticated supercomputer – and the play that results
from this combination of ways of thinking has revolutionised the game. It
remains to be seen whether cooperation is possible – or will be permitted –
with the kinds of complex machines and systems of governance now being
developed, but understanding and thinking together offer a more hopeful path
forward than obfuscation and dominance.
Our
technologies are extensions of ourselves, codified in machines and
infrastructures, in frameworks of knowledge and action. Computers are not here
to give us all the answers, but to allow us to put new questions, in new ways,
to the universe.
Rise of the
machines: has technology evolved beyond our control? By James Bridle. The Guardian, June 15, 2018.
The Age of
Complexity is an extract from the book ‘New
Dark Age: Technology and the End of the Future’’ It was published by Verso in the UK in June 2018,
and the US in July 2018.
More extracts here :
Inside
the infinite imagination of a computer. Dazed & Confused, September 17, 2018.
AR
In your
book, you talk about the notion that we’re in a state of knowing more about the
world than ever, but we have less and less agency to change it, and we need to
develop a kind of literacy around these computing systems. But it seems like we
could develop literacy and still not gain any real power over the systems.
JB
Oh,
absolutely. That’s certainly possible. I don’t think that there’s any kind of
anything that will guarantee you some kind of magical power over things. In
fact, the hope that you can do so is itself kind of dangerous. But it’s one of
the routes that I explore to a possibility of gaining some kind of agency
within these systems.
One of
the ways that I approach these problems is through one particular form of
systemic literacy that I’ve developed through my work and my studies, but I
also think it’s generalizable. I think anyone can get there from a background
in any number of disciplines. And understanding that literacy is transferable
and that we all have the capabilities to apply it to think clearly about
subjects that seem difficult and complex is one of the main thrusts of the
book.
AR
You’ve
given examples in the past of ways that people could resist “inevitable”
technological progress, like taxi drivers making salt traps for self-driving
cars. What else could they do?
JB
I did a
whole bunch of projects around self-driving cars, which also included building
my own — poorly, but in a way that helped me learn how it’s done — so that I
gained an understanding of those systems, and possibly as a result would be
able to produce a different kind of self-driving car, essentially. In the same
way that anyone who tries to work on these systems, build them themselves, and
understand them has the possibility of shaping them in a totally different way.
The
autonomous trap is another approach to some of the more threatening aspects of
the self-driving car. It’s quite a sort of aggressive action to literally stop
it. And I think working with and attempting to stop and resist are both super
useful approaches, but they both depend on having some level of understanding
of these systems.
AR
These seem like individual solutions to some extent. How do you deal with situations like climate change, where you need really large-scale systemic change?
JB
There’s
a couple of things I talk about regarding climate in the book, and one of them
is to be really, really super direct about the actual threat of it, which is
horrific, and it’s kind of so horrific that it’s difficult for us to think
about. Simply the act of articulating that — making it really, really clear,
exploring some of the implications of it — that kind of realism is a super
necessary act.
We’re
still fighting this rear-guard action of, “Oh, it’s manageable,” “Oh, we can
mitigate it,” or “It’s not really real.” We’re still, despite everything we
know, everything people say, stuck in this ridiculous bind where we seem
incapable of taking any kind of action. And, for me, that’s part and parcel of
this continuous argument we have over numbers and facts and figures and the
data and information that we’re gathering, as though this is some kind of
argument that has to be won before we do anything. That excludes the
possibility of doing anything concrete and powerful and present.
AR
How does
it feel to be a critic of these technologies for years and suddenly see people
start agreeing with you?
JB
I think
there’s a lot of people right now who find themselves in the position of being
“Well yes, this is exactly what we meant,” you know? I remember having
conversations years ago with someone saying, “What’s the worst that can happen
with someone having all this data centralized?” And my answer to that was,
“Well, the worst thing that can happen is that fascists take over and have
control of that data.” And a few years ago, that felt like the worst possible
thing, completely unimaginable. And here we are today — when fascism is alive
and well in Europe, and growing in certain ways in the US as well. So it’s
suddenly not so remote.
But at
the same time, people who have been thinking about this for a while have also
been building things that are capable of mitigating that. So while I argue
against everything being magically fixed, putting this all out in the open in
certain ways does start to make some kind of difference. The really important
thing, I think, is to constantly frame this as a struggle. Which, again, we
kind of don’t often do, particularly in the context of technology — where we
see this stuff as a kind of ongoing, always upward unstoppable march.
Technology
always walks this kind of weird knife edge. It becomes hard for us to
understand and change — everything disappears behind glass, inside little black
boxes. But at the same time, if you do manage to crack them open just a little
bit, if you get some kind of understanding, everything suddenly becomes really
quite starkly clear in ways that it wasn’t before. I’m kind of insisting on
that moment being the moment of possibility — not some kind of weird imaginary
future point where it all becomes clear, but just these moments of doubt and
uncertainty and retelling of different stories.
AR
Speaking
of stories, you reference authors like H.P. Lovecraft and Iain M. Banks in New
Dark Age. How is fiction shaping the way we deal with this future?
JB
A lot of
the way that we think of technology, and the internet in particular, has been
really shaped by the ideas of it that came along before the thing itself
arrived, right? Just as our ideas of space exploration are completely shaped by
fantasies of space exploration from long before we got to space practically.
The really interesting science fiction to me now happens kind of in the next
week or the next year at most because it’s so obvious to us how little we can
predict about long-term futures, which really, for me, is more of a reflection
of reality than reality is a reflection of science fiction.
I’m
unsure about the value of stories to pull us in a particular direction. Most
science fiction writers insist that all their fiction is really about the
present, so they’re really just different ways of imagining that.
AR
Jeff
VanderMeer has also said that futuristic dystopias are a way of shifting real
problems “over there” out of reality.
JB
Yeah,
exactly. There’s a whole genre of design fiction as well that posits these
political things as design objects as a way to kind of pull those futures into
being. But I always think there’s something very risky about that, because it
also positions them as somewhere else, right? Not as tools that we have access
to in the present. And VanderMeer’s fiction is pretty interesting, because
while it’s obviously somewhat future-oriented, it’s also deeply about the weird
and strange and difficult to understand.
I think
that is better than what I said before, really. That is the most interesting
current within science fiction right now: not imaginings of weird futures,
utopian or dystopian, but ones that really home into how little we understand
about the world around us right now.
AR
How do
we critique the idea of inevitable, upward progress without overly
romanticizing the past? In the US, criticism of automation gets tied up with calls
to protect jobs that fit a stereotypical 20th century white, male vision of
work.
JB
There’s
always that danger of romanticization, it’s true. It’s still being played out.
That also comes about because of our really narrow view of history — that we
have these quite small and very essentially made-up histories of things that
we’re so acculturated to. So one of the things I try to do in the book is pull
out these alternative histories of technology, and that’s another current
that’s quite strong at the moment.
I just
read Claire Evans’ book Broad Band, about the number of women involved in the
creation of the internet as we know it today. Many of the characters, real
people in her book, they’re not just engineers and programmers. They’re also
community moderators and communicators, people who shaped the internet just as
much as people who wrote the lines of code.
And so
as soon as you dig up that history, you then can’t help but understand the
internet as something that’s very different in the present. And therefore you
can understand the future as something else as well. So if we talk about
automation, then one of the works we can do is not just to hark back to some
kind of golden age, but to trouble that legacy as well, to talk about who
worked then and under what conditions, you know?
There’s
always technological resistance. Like the Luddites, who are pretty well-known
now, but the fact is that the Luddites weren’t smashers of technology; they
were a social movement, performing a very violent and direct form of critique
of the destruction of their livelihoods, of what those machines were doing. And
so now we have many, many other tools of critique for that. But by retelling
these stories, by understanding them in different ways, it’s possible to
rethink what might be possible in the present.
James
Bridle on why technology is creating a new dark age. By Adi Robertson. The Verge. July 16, 2018.
Halfway
through James Bridle’s foreboding, at times terrifying, but ultimately
motivating account of our technological present, he recounts a scene from a
magazine article about developments in artificial intelligence. The journalist
is asking a Google engineer to give an image of the AI system developed at
Google. The engineer’s response was, ‘I do not generally like trying to
visualise thousand-dimensional vectors in three-dimensional space.’ A few pages
later, discussing the famous example of grandmaster Garry Kasparov losing a
series of six chess matches to IBM supercomputer Deep Blue, Bridle quotes Fan
Hui, an experienced Go player, describing the Google-developed AlphaGo
software’s defeat of professional Korean Go player Lee Sedol at the
2,500-year-old strategy game: ‘“It’s not a human move. I’ve never seen a human
play this move.” And then he added, “So beautiful.”’
The first challenge for proving a system’s
intelligence is image cognition: AI are trained for facial recognition or to
scan satellite imagery. Still, technology is not primarily considered a visual
problem, even if new technologies’ effect on our lives is the subject of
countless movies which are often, to echo Bridle’s title, quite dark. Bridle, a
visual artist whose artworks consider the intersection of technology and
representation, from the shadows cast by drones to the appearance of stock
images in public space, does not focus his book on representations of
technology, but rather on a different visual problem: invisibility. In his
introduction, Bridle warns that society is powerless to understand and map the
interconnections between the technological systems that it has built. What is
needed, the artist claims, is an understanding that ‘cannot be limited to the
practicalities of how things work: it must be extended to how things came to
be, and how they continue to function in the world in ways that are often
invisible and interwoven. What is required is not understanding, but literacy.’
Literacy,
in Bridle’s use, is beyond understanding, and is the result of our struggle to
conceive — to imagine, or describe — the scale of new technologies. A lot of
the examples in the book are visual and descriptive, providing new imagery to
help his readers picture some of the issues that should concern them but are
hard to imagine since they happen far from the eye. In a chapter dedicated to
complex systems, Bridle describes Amazon warehouses that employ a logistics
technique called ‘chaotic storage’ which manages the goods on floors whose
organisation is not based on any order a human can grasp — alphabetised books,
homeware in a specific department — but on an algorithmic logic that makes the
system incomprehensible to its employees. The workers carry handheld devices
that direct them across the facility: they are incapable of intervening with
the machine’s choice, incapable of seeing its reason. Even when things are made
visible, it’s also a reflection of darkness: when IBM developed the Selective
Sequence Electronic Calculator, it was installed on a ground-floor shop on East
57th Street in Manhattan. The President of IBM at the time, Thomas J. Watson,
wanted the public to see the SSEC, so that they would feel assured that the
machine is not meant to replace them. The publicity photos of the IBM calculator,
operated by a woman in a former shoe store, do not expose what was actually
happening: the SSEC was being used to run simulations of hydrogen bomb
explosions, carried out in full view in a storefront in New York City.
New Dark
Age is neatly divided into ten chapters, each titled with a single word
beginning with the letter C: ‘Chasm’ is the introduction, and one of the most
valuable sections of the book, discussing how technological acceleration has
changed society and charting the impossibility of seeing clearly how these
changes affect every aspect of our day-to-day lives: ‘new technologies,’ writes
Bridle, ‘do not merely augment our abilities, but actively shape and direct
them, for better and worse. It is increasingly necessary to be able to think
new technologies in different ways, and to be critical of them, in order to
meaningfully participate in that shaping and directing.’ The next chapter,
‘Computation’, is a short history of computers, in which Bridle explores the
interwoven history of computational development and warfare, especially atomic
warfare during the Cold War. A chapter called ‘Cognition’ is dedicated to
artificial intelligence, and one titled ‘Complicity’ discusses surveillance and
systems of control via technology. ‘Concurrency’ takes up an example Bridle has
written about before — and which was picked up by major newspapers and
television news — and expands it. The initial essay was titled ‘Something is
wrong on the internet’; Bridle published it on Medium because, he explained in
a short paragraph preceding the piece, he didn’t want the materials he was
writing on ‘anywhere near’ his own website. Looking at YouTube videos, Bridle
was pointing to several disturbing, weird, dark clips purportedly served up to
toddlers. Things like the ‘wrong head trope’, which involves Disney characters
whose heads are separated from their bodies, floating onscreen to the sound of
nursery rhymes until they are matched with the right bodies or a bloody video
of Peppa Pig going to the dentist. Bridle describes ‘a growing sense of
something inhuman’ in the proliferation of these — which isn’t necessarily
related to the origin of these videos but, rather, to the way they are
distributed to children: via an algorithm that serves them disturbing content
because it is set to autoplay. Bridle links this example with a discussion of Russian
interference with foreign elections via the distribution of misinformation, and
he also brings in the Ashley Madison hacks, which exposed that the dating site
for married people had tens of thousands of fake, automated female accounts
that interacted with men: paid subscribers who shelled dollars to interact with
a piece of software attached to a photograph of a woman. The content directed
at us, whether created by state propaganda, corporations in search of
advertising dollars and paid subscriptions, or simply spammers, creates the
same results — confusion, deception, a relationship to power (state or
corporate) that is constantly reasserted by the information we are served up.
‘This is how the world actually is,’ Bridle says, ‘and now we are going to have
to live in it.’ (And raise our children in it.)
In
‘Climate’, a summary of technology’s effect on and impact by climate change,
Bridle outlines the endless cycle in which abuse of resources affects a system
that uses those resources both to study and monitor the climate. For example,
cable landing sites, where the submarine cables connecting the internet reach
the shore, are especially vulnerable to sea level rise — which is ironic since
the internet is also a major player in climate change. The power data centres
require accounted for 2 per cent of global emissions in 2015, which is about
the same carbon footprint as commercial aviation. Cryptocurrencies and
blockchain software, so often discussed in emancipatory terms, since they have
the potential to decentralise financial systems, require the same amount of
energy as nine American homes per day to complete a single transaction;
blockchain will use up the same amount of electricity as the entire United
States by the end of 2019. In Japan, predictions are that by 2030 digital
services will require more power than the nation can generate. The network’s
voracious consumption of power isn’t just the responsibility of the NSA data
centres, but also the end-users. ‘We need to be more responsible about what we
use the internet for,’ Bridle quotes Ian Bitterlin, a UK expert on data
centres: ‘Data centres aren’t the culprits – it’s driven by social media and
mobile phones. It’s films, pornography, gambling, dating, shopping – anything
that involves images.’
Which
suggests the missing chapter—or approach—in the book: Culture. Bridle is an
artist and the visual examples he puts forth are some of the highlights of the
book, especially when considering its subtitle: ‘the end of the future’. The
end, that is, of something we’ve always imagined. There is a lovely short
section where Bridle writes about the Concorde, the supersonic passenger plane
that British Airways and Air France stopped flying in 2003. Bridle describes
growing up in the London suburbs under the flight path to Heathrow Airport and
hearing, every evening at 6.30 p.m., the rumble of the plane, its futuristic,
sleek, triangular design an image of the future that died with the end of the
Concorde flights. These stunning few paragraphs on design and its impact on the
popular psyche follow a discussion of clean-air turbulence (another terrifying
result of climate change, where flights experience extreme turbulence in
unforeseen areas) and precede a simple conclusion: that futuristic inventions
and designs like the Concorde are the exception and the rule is small in-flight
adjustments, like slightly better wingspan leading to slightly better fuel
mileage. These two pages set up an idea about what we cannot see: Bridle cites
philosopher Timothy Morton’s idea of the ‘hyperobject’, which is a thing that
is too big to see in its entirety and thus to comprehend. Climate, for Bridle,
is a hyperobject — which we only perceive through its effects: a melting ice
sheet, for example — ‘but so are nuclear radiation, evolution, and the
internet.’
The
things we cannot see are not always imperceptible because they are too large to
comprehend, but because they are intentionally obfuscated. The simple example
is the language we use when discussing technology — the ‘cloud’ for a series of
links between servers; ‘open’ is a decentralised resource, but open-source is
also a method of building free software using business-friendly,
hivemind-labour. The ‘democratising’ potential of the internet is hailed by
multinational corporations, those same corporations that stand to benefit from
the positive PR of the ‘freedom’ that platforms like Twitter promote. Without
the use of scare quotes, these ethereal, abstract terms press an understanding
of the internet as an ecosystem with its own rules, and one that is presented
as intangible and ubiquitous. The far-from-simple example is Bridle’s
discussion of high-frequency trading. In a chapter titled ‘Complexity’, Bridle
describes a bicycle ride from Slough, just west of London, to Basildon near the
eastern coast of the UK. The 60-plus-mile journey cuts through the heart of the
City, London’s financial hub. The City and its cluster of glass towers is the
public image of the UK’s finance sector, but the transactions that fuel it are
made out of sight, in warehouses like the Euronext Data Center (the European
outpost of the New York Stock Exchange) in Basildon and the LD4 Data Centre
(the London Stock Exchange) in Slough. The glass towers, the stock exchanges
designed like Greek temples, are now symbolic, empty signs: they stand for
something that is totally invisible, that happens in warehouses on the
outskirts of the city. Conjure an image from 1980s films about Wall Street and
its culture: on the trading floor, men shouting, fighting, running while
carrying slips of papers in their hands. Replace it with the image of men
sitting in offices, pressing the refresh button again and again on their
desktop computers. Then replace that image, too. Financial transactions were
always dependent on speed, but as computing power and network speeds have
increased, the speed of these exchanges has accelerated to leave these men
behind.
Now
computers are trading with other computers in countryside locations where space
and power are available, but there is no symbolic imagery. ‘Financial systems
have been rendered obscure, and thus even more unequal,’ Bridle writes. The
chapter on complexity is also the one that talks most about the effects of the
meeting of capital — another C word — and technology on the societies we live in,
especially in terms of labour. This is the chapter to include a long discussion
of Uber’s relationship to its drivers as contractors (who they force to listen
to anti-union podcasts) and the charting of Amazon’s storage facilities. These
networks are not invisible, they are made to look invisible. And the stakes of
opacity are the impossibility of organising, both as employees and as citizens.
Could there be an Occupy movement around obfuscated spaces like data centres on
the peripheries of cities?
Bridle’s
conclusion begins with an event — the 2013 Google Zeitgeist conference. Held
annually in an exclusive hotel in Hertfordshire, England, it’s a private
gathering — though some of the meetings are posted on Google’s ‘zeitgesitminds’
page, TED-talk style — for executives and politicians. At the 2013 conference,
Google CEO Eric Schmidt publicly discussed the emancipatory power of
technology. Schmidt talked about how technology, and particularly cell phones
and their built-in cameras, could prevent atrocities by exposing them —
‘somebody would have figured out and somebody would have reacted to prevent
this terrible carnage.’ His example was the Rwandan genocide, which, he
described, had to be planned, ‘people had to write it down’. An image of those
plans would have leaked, Schmidt is certain, and ‘somebody would have reacted’.
Bridle summarises easily: ‘Schmidt’s — and Google’s — worldview is one that is
entirely predicated on the belief that making something visible makes it
better, and that technology is a tool to make things visible.’ But of course,
the UN, the USA, Belgium and France all had access to intelligence information,
including radio broadcasts and satellite imagery from Rwanda, and ‘somebody’
didn’t react. Bridle cites a report on Rwanda, noting it could have been the
conclusion of his book, too: ‘any failure to fully appreciate the genocide
stemmed from political, moral, and imaginative weaknesses, not informational
ones.’
The
incapability to understand the scale and impact of technology on the lives of
human beings is not a visual problem, it is a problem of imagination. One of
the significant achievements of Bridle’s book is that it challenges the idea
that to participate in the conversation about technology requires prior
technical knowledge. Rather, Bridle points out, the fight is against the
intentional obfuscation of systems, and that is before we even consider machine
vision: to counter Schmidt’s idea of technology as a tool to make things
visible, we need to criticise the role of technology in the creation of that
image. Considering these complex questions of representation, maybe we should
look to visual artists in order to see a reflection of the world we live in,
and see that to point to the darkness is a way of shining a light. For the
informed reader of technology criticism, New Dark Age will not be a revelation.
Bridle’s research is impressive and the knowledge, examples and concerns he
lays out are proposed in an organised, systemic fashion. As a summary of
discussions spanning many disciplines, from finance to entertainment and
climate change, Bridle’s book is not a primer, but a crucial illustration of
just how intertwined these concerns are.
New Dark
Age takes its title from H.P. Lovecraft’s ‘The Call of Cthulhu’ — ‘that we
shall either go mad from the revelation or flee from the deadly light into the
piece and safety of a new dark age’ —
but then goes on to cite a line from Virginia Woolf’s diaries: ‘the future is
dark, which is the best thing the future can be.’ This book is not a collection
of prophecies; it is a commitment to the present. ‘Nothing here is an argument
against technology: to do so would be to argue against ourselves,’ writes
Bridle. He insists that what is needed is not understanding, but a new
language, new metaphors — a new image — that would allow us to look at the
darkness directly and — hopefully — begin to see.
