The Impulse to Capture, Understand, and Share

Paradigm shifts, which come in storms of new technologies, maim big companies and create new stars. The iPhone marginalized both Eastman Kodak and Nokia. Both companies were dominant at one point, but new technology created new stars. The same will happen in the Spatial Computing decade, which is just beginning. The storm will be violent and swift but to understand why this is so, we need to go back to that earlier time.

Ansel Adams first visited Yosemite in 1916 when he was 12 years old, carrying a new camera he had been given as a gift. He was so stricken by the beauty he saw everywhere that he tried to capture it, with the intent of sharing it with friends and family back home. He was frustrated, his son Michael told us, that his photos didn't properly capture anything close to what he experienced in real life, and that drove him to spend the rest of his life trying to find ways to improve photography so that his photos would more closely match what he experienced.

That act has been followed by millions of visitors each year who stand in the spots where he captured Half Dome or El Capitan, but it is by visiting Adams' home in Carmel, California that you see the ties between photography of the past and where we are going next. In that home, you see the photography of the past. His darkroom still holds his dodging tools, enlarger, and the bottles of chemicals Adams used to develop, stop, and fix the silver halides that we all see in Adams' photography today. That was hardly the only technology Adams used in making his photographs, but it gives insight into Adams' creative process, which has been studied by many photographers even today.

In fact, that same creative process has been trained into many cameras' metering systems and, now, in modern phones from Huawei and Apple, has been taken further thanks to Computer Vision systems that make photos sharper, better exposed, and more colorful.

Photo credit: Robert Scoble. Michael Adams, Ansel Adams son, shows us Yosemite Valley and tells us how, and where, his dad took famous photos, with him in tow.

What we also learned is that Adams created much of Kodak's advertising, by using a tripod he invented that enabled him to take "wrap around" photography by shooting several images one next to the other. A phone's panoramic mode basically does the same thing, without needing a tripod with notches in it to properly align the images. A computer in your phone now does the work that Ansel used to do. What would Adams think about digital photography and the hordes of people taking photos on phones and other devices? Or AIs that "improve" photography by searching large databases and replacing things, like blurry overexposed moons with great-looking properly exposed ones, something that Huawei's latest phones do? Adams' son says that if Ansel were alive today, he would be right there along with other innovators: pushing the technology of photography even harder in an attempt to get closer to the natural world.

Why? He, and many environmentalists such as John Muir, played key roles in protecting Yosemite as a National Park, as well as getting many to visit the park through their work. Their idea was that if they could just show people the natural world better, they would be able to get people to travel to see it. If they could travel to see it, they might change their attitudes toward nature and change their polluting ways. This point of view is more needed today now that we can see man's impact on Earth is far deeper and more dangerous to our long-term survival than even Adams could see 100 years ago.

Who fills Ansel's shoes today? Or who will need new forms of images the way Kodak did to hang in Penn Station, New York, like it did with Ansel's photos? People like Ross Finman. He runs an Augmented Reality lab in San Francisco, California, for Niantic, the company who built Pokémon Go. We can see a world where games built on Niantic's platforms will change our behavior quite deeply. We saw thousands of people running across New York's Central Park just to catch a coveted rare Pokémon. This is a new behavior-change platform, and Finman and his team are the ones building the technology underneath it. Isn't behavior change what advertisers are paying for? Niantic's platforms understand the world better than most and were started by the team behind Google Earth, so they've been at this for quite some time.

His dad, Paul, and mom, Lorna, met at Stanford University where they both got PhDs. His was in electrical engineering, hers was in physics. Both are passionate about science and share a sizeable warehouse where Paul continues to develop new technologies. We visited this lab because it is where Ross developed his love of robotics (he later went onto Carnegie Mellon to study that very subject, and afterward started a company, Escher Reality, which was sold to Niantic. More on that in a bit). The first Niantic game to see his work was "Harry Potter: Wizards Unite," which shipped last year.

Outside of the lab in a huge field is an autonomous tractor moving around showing us that this isn't your ordinary Idaho farmer. This field is where Ross first developed his SLAM algorithms, which we now call the AR Cloud, and are the basis for how Augmented Reality works (and how that tractor navigates around the field). SLAM stands for "Simultaneous Location and Mapping" and builds a 3D map of the field, which a computer then can navigate around.

Augmented Reality glasses and Autonomous Vehicles, along with other robots, and virtual beings, all use SLAM to "see."

Photo credit: Robert Scoble. An autonomous tractor rolls around in a field surrounding Paul Finman's lab in Ceour d'Alene, Idaho. This field is where Ross Finman did his work building the Augmented Reality technology now being used in Niantic's Harry Potter game.

The SLAM/AR Cloud system that Finman is developing isn't directly connected to Ansel Adams' form of chemical-based photography, but there is a tie to both Adams' impulse to capture and share and Finman's. Ansel's work was analog. Finman's work is digital. Unlike earlier digital photographers, Finman's system captures the real world as a 3D copy instead of a 2D grid, which was closer to what Ansel did. Finman is the Ansel Adams of this new polygon-based age. His work will let us be in the photograph. Unlike Adams, though, we can use that data to train AIs that will keep us, or at least something that looks, moves, and sounds like us, around―potentially forever.

This process of understanding the real world, converting it into digital information, and then processing it into a form that can be transmitted and displayed (developers call it a mesh) is the basis of everything you will read about in this book.

Understanding the gap between the real world (we think of it as the analog world, because the photons, that is, light, that hits our eyes from, say, a sunset, or the audio that hits our ears, arrive as a smooth wave) and the digital one, which is quantized into streams of numbers, is key.

This is very different from the digital world that underpins Spatial Computing, which has been sliced and diced into polygons or quantized into streams of numbers playing on Spotify. You see it when you see virtual beings, like avatars, and there isn't enough resolution to properly "fool" your mind. The industry calls this "the uncanny valley" for the difference between our emotional response to seeing, say, a real human singer, and an artificial one.

Finman studies this uncanny valley, and in his work at Niantic he is working to more completely understand the real world, build systems that have deeper situational awareness, and then close the gap between how we sense the real world and how we perceive, and emotionally respond to, this new, digital, world.

The impulse, that Finman has to capture, understand, and share the world using new technology, is the same that Adams had with his camera, and that is what ties both of them into our prime directive to become better human beings.

Soon, Finman will bring us powers to capture, understand, and share our world that Adams probably would never have fathomed. Our children will be able to jump into family dinner, either remotely, or in the future go back in time to experience what it was like to have their mom serve them dinner. A new way of remembering your life and everything in it is on the way with Spatial Computing.

Soon, too, will analog experiences be relegated to special places like racetracks or rare vacations to beautiful places. Both things that increasingly you'll need to be wealthy to experience. The storm of change is about to see everything turn digital and that has deep implications for games, entertainment, and how we capture, understand, or share our own lives with others. Soon, we will be interacting with this virtual world, and the innovation teams are already gaining new skills to build new interfaces with this new world.

We won't be surprised if, in a few decades, Finman and other innovators like him are celebrated the way we celebrate Adams today as a true pioneer that pushed technology to its ultimate edge. We also won't be shocked if a few huge companies or product lines disappear over the next decade.

To understand more of the gap between this polygon-based digital age and the analog age of media that is coming to a close, we need to visit Neil Young's studio. He invited us in to listen to some of his music on a two-inch analog tape. He wanted to show us what we lost when we moved music from analog tape and vinyl records to listening on Spotify on our phones.

His audio engineer, John Nowland, played us "Harvest Moon" on that analog tape, and then we listened to it in digital form after that analog master was turned into slices. 600,000 of them a second, which is about 12x more than you will hear on a CD. So, pretty high resolution.

We could still hear a difference. "The closer you were to the analog, the more natural it felt," Nowland told us. "You lose the nuances and detail when you squash it down." That interview is at https://youtu.be/Ta-RvERB6Ac.

Photo credit: Rocky Barbanica. Robert Scoble meets with Neil Young's audio engineer, John Nowland, and listens to music in both analog and digital formats.