When tech changes our view of the world

What happens when augmented reality collides with the internet of everything? Some cyber prophets envision an enhanced mode of existence called the mirrorworld. But is the mirror already broken?

Looking back at the future can provide an interesting, if somewhat embarrassing, perspective on the history of innovation. Every new year brings new buzzwords, sounds of revolutions and paradigm shifts. But the truth is our ability to predict the future was never very astute.

One year the prophets assure us that the smart home is sure to change our lives, then 3D printers are on the verge of making factories obsolete, next comes the internet of things, virtual reality, augmented reality, blockchain, self-driving cars, artificial intelligence… Yet somehow these predictions repeatedly miss their mark. Another year goes by and neither society nor our lives are as thoroughly transformed as we envisioned. Instead other, uncharted tracks toward the future open up. Suddenly there are kids running around chasing pokémon all over town, sharing the streets with adults zooming recklessly through traffic on rented electric scooters.

To be fair, many of the innovations that were promised have, technically speaking, come to fruition. Augmented reality, for instance, has been available and widespread for many years. A few years ago, I visited the Googleplex in Mountainview and tried the (at the time) bleeding edge Google Glass AR glasses. The tech works just fine and seems useful to boot. It just turns out that it is not quite useful enough for people to actually want to buy and wear Google’s glasses outside of very specific use cases; at least not yet.

Google Maps, on the other hand, is AR that has already found a natural place in our lives. It is a great example of how technology can fundamentally change our perception of the world around us, while staying out of the way. Google Maps adds a digital layer of information onto the physical world, full of useful information about our surroundings, and we have gradually become more dependent on this digital layer, to the point where it now is second nature. Meanwhile carrying a paper map, or even calling someone for directions, now seems strangely archaic.

An intriguing idea

This February Wired Magazine published a massive think piece on AR, written by the magazine’s founding editor, Kevin Kelly. The headline reads: “AR will spark the next big tech platform – call it mirrorworld”. The buzzword alarm goes off whenever some tech missionary combines the words next and big in a headline, but the idea Kelly delineates is an intriguing one:

“The mirrorworld doesn’t yet fully exist, but it is coming. Someday soon, every place and thing in the real world—every street, lamppost, building, and room—will have its full-size digital twin in the mirrorworld. For now, only tiny patches of the mirrorworld are visible through AR headsets. Piece by piece, these virtual fragments are being stitched together to form a shared, persistent place that will parallel the real world.”

This concept did not originate with Kelly. The term was first coined by Yale University computer scientist David Gelernter in his 1991 book “Mirror worlds”, where he wrote that this technology “will revolutionize the use of computers, transforming them from (mere) handy tools to crystal balls which will allow us to see the world more vividly and see into it more deeply”.

The vision of the mirrorworld is essentially a merging of augmented reality with the internet of everything, another ambitious tech prophecy that is taking its sweet time to come true. The IOE is the vision of a world where your bed, your refrigerator, your stove, your car, and your shoes are all connected and exchanging information. In a world where everything is connected the potential of augmented reality is vastly enhanced.

To understand what the mirrorworld would mean, it is helpful to consider self-driving cars. At present they are using cameras and other sensors to collect information from the world they operate in, meticulously building a digital model of the streets around them that their algorithms then struggle to make sense of. But they are also aided by the aggregated data that is collected, uploaded to the cloud and then accessible to other cars, for instance in the form of maps and information about the current traffic situation.

In an internet of everything scenario the cars on the street would be less dependent on making sense of the world through computer vision. Cars, bicycles, pedestrians, traffic signs, bridges and the roads themselves would be in constant communication with each other, sharing information and together creating fine-grained digital versions of the world – mirrorworlds if you like. AR eyeglasses, too, would be far more powerful than the current state of the art if they could instantly extract information from every little thing they were looking at.

A world built on ones and zeros

The vision of the mirrorworld is shared by many players in this fledgling industry. Magic Leap is one of the companies developing AR hardware to allow us to access the information layers in new ways. They call their version of the mirrorworld the “Magicverse”. Meanwhile, futurist author Charlie Fink calls it a world “painted with data”, and Microsoft’s Mike Pell likes to talk about spatial computing and “smart information” that presents itself precisely when we need it, based on its understanding of our location and context.

These visionaries are all referring to a vast new world built by ones and zeros that, with the help of AR and AI, alters the physical world around us. This mirrorworld – or whatever you choose to call it – is an exciting prospect. But the bleak reality is that there will never be a single, shared mirrorworld. Instead, there will be many mirrorworlds.

Digital layers are added to our world at a rapid pace. In just a few decades mobile phones have altered our sense of place, space and time. We tend to imagine the next technological milestone to be amazing and transformative while what we already have seems boring and routine. But it can be helpful to zoom out from the here and now, and imagine what we would think of today’s smartphones if they were presented to us in the 1970’s. From that viewpoint we are carrying around tiny miracles of human innovation in our pockets, and they happen to be amazing augmented reality tools. In the words of cyberpunk author William Gibson: the future is already here – it is just not very evenly distributed.

Today, the overwhelming majority of the world’s population does not own a smartphone. As more digital layers are added to the world, there is no reason to believe that they will be accessible to all, nor distributed equally. We will be piecing together shards of these mirrorworlds in different ways, depending on what we want and what we can afford.

Every powerful tech company in the world will want to own, control and profit from these mirrorworlds, which means there will be competing shards or layers. Alphabet – the parent company of Google and Waymo – is particularly well situated to rule these worlds. They are leading the AI race, and with Google Street View cars and Waymo’s fleet of self-driving cars they are documenting the physical world, inch by inch. With the help of Android smartphones, they can track smartphone users as they move not only on the web but through the physical world as well. Google Maps, as mentioned before, is an impressive digital recreation of the planet, with maps of varying definition covering the entire globe. In some places the granularity is no greater than showing the names of roads, in other places you can effectively walk the virtual streets of cities, and even find your way inside building complexes like airports and shopping malls.

All are investing heavily

As amazing as this is, Google Maps is just in its infancy. 15 years ago, it did not exist. Try to imagine what it will be 20 years from now.

Alphabet competitors such as Facebook, Apple, Microsoft, Amazon, Baidu, Wechat and Tencent are behind at this point, but they are all investing heavily in technologies like augmented reality, virtual reality and artificial intelligence. As competing mirrorworlds collide, we will see reality fracture into different layers and experiences. If you think this sounds strange, I can reassure you: it will soon seem familiar, even unremarkable. In fact, it is already happening. Imagine three people moving through a city. One is a foodie, one is a Tinder user, one is a Pokémon Go player. The way they experience the city will be very different from each other: the foodie with an eye on the interactive map leading the way to the hottest restaurants and microbreweries; the Pokémon Go player immersed in the hunt for “pokéstops” visible only to those with the app active; the Tinder user keenly aware of potential dates in the immediate vicinity. And then, of course, there is the rare flaneur who experiences the city au naturel, oblivious of the many layers of data available.

Philosophers have argued for centuries about the existence of an objective reality.

Philosophers have argued for centuries about the existence of an objective reality, whether or not we can perceive it and should expect that we share the same experience of it. Plato, in ancient Greece, believed that such a reality existed but that it was inaccessible to humans, who only perceive its shadow. Immanuel Kant followed in Plato’s footsteps two millennia later, distinguishing between objects as things-in-themselves and as phenomena, the latter being our perceptions of these objects, filtered through our sensibilities.

The prophecy of the mirrorworld is a grand one, a promise of enhanced access to and understanding of the world. But the most important takeaway here is this: As we integrate technology ever more intimately into our lives, our sensibilities are changing, and with them the very phenomena of objects around us. In the words of David Gelernter, tech now indeed helps us “see the world more vividly and see into it more deeply”.

Moving into the future, our shared experience of the world is coming apart, piece by piece, altered as it filters through a multitude of digital layers. To each his own mirrorworld, presented by Google or Facebook – and filled with highly relevant advertising.