Photos by Reid Godshaw.
Standing in the University of Toronto’s Humanistic Intelligence Lab may be the closest thing to being inside Dr. Steve Mann’s head. The lab is divided into two rooms, each crammed with circuit boards, soldering irons, and computers from the bygone era of floppy discs and dial-up modems. Stacks of boxes filled with manuals, prototypes, tools and other electronics nearly reach the ceiling, and spread all around are esoteric devices like the Model 300A Harmonic Wave Analyzer and the Krohn-Hite 3202 Dual Channel Variable Filter – both more than seventy years old. Pioneering scientists once used these contraptions to measure sound and light waves. They arguably belong in a museum.
For Mann, a tenured professor at the school, the anachronistic mess is deliberate. “Since childhood, I’ve been fascinated with how things work, and thus surrounded myself with things like lock-in amplifiers, cathode-ray oscillographs, and the like,” he says. “To lead a team to invent the future, we need to understand what came before. I see so many labs where they throw away all the old equipment, and have forgotten certain fundamental aspects of the past.”
“There’s a lot of stories in this lab,” says Ken Yang, who met Mann as an MBA student at the University of Toronto, and now works to help commercialize many of his inventions. “When you’re in Steve’s labs, you’re like an archaeologist, just digging through legacies.”
It’s the type of space you get familiar with when you work with an inventor and academic who, by his own admission, has been called a “freak” and a “weirdo,” but also “the father of wearable computing.”
Steve sits near his accolades and some of the artwork he made, using radio waves from smartphones to burn into the wood upon which they are mounted. This work, entitled “Burner Phone,” is an example of real augmented reality.
Born in Hamilton, Ontario, Mann – now in his late 50's, a husband and a father – has always had a sense of wonder when it comes to technology and inventions. In 1966, when Mann was four years old, his grandfather taught him how to weld. Mann noticed that the protective helmet’s small, tinted viewing window almost completely obstructed the welder’s vision. That realization led to his later role in the invention of a technology many of us utilize daily: High Dynamic Range, or HDR, which is the stitching together of multiple images with a range of exposures to make a clearer, more true-to-life picture. The technology – now used in all types of cameras, from five-pound DSLRs to iPhones and iPads – was vital in his development of the device he wears almost 24/7 today, the EyeTap, which many consider the predecessor to Google Glass.
Mann’s father – whom Mann describes in his book Cyborg: Digital Destiny and Human Possibility in the Age of the Wearable Computer as a “lifelong tinkerer and experimenter” – taught him about electronic circuits before he learned to read, write or do arithmetic. Mann and his brother, who became an artificial intelligence professor, built a “bugging system” to record their parents’ conversations, as well as an “early warning system” that would make a sound every time anyone came near their room. He also may have built the world’s first Walkman. “As a child, I removed the stereo head from a large cassette deck and installed it in a specifically modified portable battery-operated dictating machine,” he wrote in Cyborg. “Then I added two high-quality sound reproduction amplifiers so I could listen to music while walking or jogging.” He heard only what he wanted to hear. No radio ads, no music he didn’t like, no car horns and screeching brakes. “While many people scoffed at this invention,” Mann continued, “I quickly realized its possibilities not only as a portable music device, but also as a way to drown out the distractions of what was becoming an increasingly hostile world.”
As a teenager, while volunteering at a local radio and television repair shop (he was too young to get a paying job there), he immersed himself in what would become his life’s work. He built his first wearable computer, the WearComp 0, out of components from the shop’s basement. It weighed about eighty pounds, and featured a thick helmet with antennae sticking out of the top, a camera viewfinder over the right eye, and several snaking cords attached to a base station worn on his waist. Inside was a battery pack and “image apparatus” to process and store the recordings. Mann admits it was cumbersome and unsightly. But it was portable, and it worked.
The next version of the WearComp, which Mann completed during his high school years – he spent much of his time alone, working on his projects, as a teen – had a joystick and graphical interface, and allowed him to link to other computers via a wireless data connection. It was attached to a computer he wired into a steel-frame backpack.
With these devices, he saw everything through a video lens, and, as with his early version of the Walkman, could block out external “pollution,” as he puts it. He could program the device to recognize and then either block out or alter recurring images – like advertisements – and HDR gave him a true picture of the outside world.
The device got smaller, lighter, and more effective over the years, until in the mid-90s it became the EyeTap, a precursor to Google Glass. Another version he sometimes wears looks like a pair of oversized sunglasses.
At about that time, Mann says another inventor and wearable computing pioneer named Mark Spitzer shared his version of an “electric eyeglass.” Mann told Spitzer that his invention was “like a wristwatch that could not tell time” because it was a display-only device, and didn’t have a camera the person wearing it could see through. Mann’s EyeTap, on the other hand, helped people see better, and looked and functioned like a glass eye. He noted that some people referred to it as an “eye glass,” and, sometimes, just “glass.”
Undeterred, Spitzer went on to found a company called MicroOptical. Its patents were later bought by Google, and Spitzer joined the company in 2012. Thanks in part to Google’s acquisition of those patents, Google Glass, a more refined, visually appealing version of the device, was released not long after, in 2013, two decades after Mann had invented his EyeTap.
***
In the early ’80s, as a student at Hamilton’s McMaster University, Mann persisted in using his WearComp, and continued to feel isolated from students and teachers. But soon, two life-changing events occurred: the 1984 Canadian release of “Liquid Sky” – a gender- and identity-bending science fiction film that made him more comfortable, almost confident, about living as a cyborg – and, that same year, the introduction to his future wife Betty Lo.
Mann eventually convinced Lo to use the WearComp and says she became as much of a cyborg as he is. (He now insists the term “cyborg,” coined in 1960 by Manfred Clynes, a friend of his, is too vague, positing that most of western society could be considered cyborgs based on the way we use technology to enhance our abilities.) He often shared his EyeTap feed with her live, transmitting it either to her computer screen or her own EyeTap version, while he shopped for groceries, just as someone might use FaceTime today.
“When I was growing up,” Mann says, “people thought [the EyeTap] was strange, and sometimes walked across the street to avoid me. But now people come running over to see it.”
He notes his children were never particularly troubled by him constantly wearing the EyeTap. When their friends at school asked about it, he says, he’d wow them with inventions like the “sequential wave imprinting machine,” or SWIM, which makes sound and radio waves visible, and the “surveilluminescent wand,” which illuminates a camera’s sight field to demonstrate how far surveillance cameras can reach. (He creates his own compound words like “inventrepreneur” and “authentegrity. He coined the term “sousveillance” – replacing the French sur for “above” with sous for “below” – to describe the recording of an activity by a participant involved in the activity, using a device like the EyeTap, rather than an eye in the sky.)
Steve doing neuroscience research on how to use the SWIM to overlay neuron action potentials directly upon the body. This research is ultimately directed toward augmented reality neurosurgery.
But being a cyborg dad did lead to some uncomfortable moments. He said in a recent interview that he once went to pick up his daughter’s passport and ran into trouble because “at the U.S. consulate [in Toronto], electronic devices are forbidden,” he said, “but I am an electronic device, by my mere being, so in some case I was existential contraband.”
Despite carrying a doctor’s certificate that permits him to wear his viewing device, Mann has had several interactions like this – some of them violent. In 2012, he and his wife took their children to Paris so they could learn Parisian French. Following a day of sightseeing, they stopped for lunch at a McDonald’s on the Champs Elysees. After they ordered their food, Mann was assaulted by several men. They tore up his doctor’s note, attempted to rip off his device, which required special tools to be removed, and pushed him out the door. It’s been referred to as the first cybernetic hate crime. Mann launched a public appeal, in which he looked to recoup the costs of damage to his equipment. McDonald’s denied its employees’ involvement in the altercation, despite much of it being recorded.
Steve examines a SWIM in the lab.
Other than these instances, the kids appear to have had a happy, though unconventional, childhood. He taught his youngest daughter, Stephanessie, to solder when she was a preschooler, and has consistently involved her in the testing of his new inventions. When kids at school asked his older daughter, Christina, about the device her dad wears, she was more than willing to offer an explanation.
“She’d respond by writing some equations,” Mann says, “and this was a fun way to get people interested in mathematics and science.”
***
“It seems that a lot of what I do is the future,” Mann says of his work via Skype. “Most of my ideas come to me in dreams. And somehow these things seem to come to the future.”
Along with his contributions to HDR and inventing the EyeTap, he created the first Linux-based watch – the smartwatch videophone – sixteen years before the Apple Watch.
He names thought leaders like legendary design critic Don Norman, author of “The Design of Everyday Things,” futurist Ray Kurzweil, and computing pioneer Mitch Kapor among his friends. He earned his PhD at MIT, where he was the founder of the Wearable Computer Project, and is the general chair of the Institute of Electrical and Electronics Engineers’ International Symposium on Technology and Society, a professional association with more than 400,000 members. He’s also an accomplished artist, having had his dystopian tech-related works displayed at MoMA.
Steve with the SWIM (Sequential Wave Imprinting Machine) that he invented in his childhood in the 1970s. Here it us used to visualize sound waves from a violin.
Aside from appearing in some TEDx talks and a few news stories, in contrast to people like Kurzweil, who have made the leap to mainstream, Mann still sits on the fringes. He has a sense though, as he gets older, and his kids grow up, that he’s leaving them a legacy, especially when it comes to making the world a better place. He’s been a vocal critic of surveillance, dedicating much of his time to exposing it via “metasensing” or, as he puts it, the “sensing of sensors and sensing their capacity to sense.” Devices like the surveilluminescent wand help people know when they’re being watched, and, potentially, take back some of their privacy.
A voracious reader of philosophy and science fiction, he is wary of the way that corporations and governments use technology. As security cameras became more ubiquitous, he grew uncomfortable with the idea that something was filming us, but we couldn’t film it. “One hundred years ago, the sheriff knew what everyone was up to, but people also knew what the sheriff was up to,” Mann said in a 2013 interview at the University of Toronto. “Now the police know what we’re up to, but we don’t know what they’re up to.”
These days, he spends most of his time in Palo Alto, California, where he’s a visiting professor at Stanford, and working as the chief scientist at Meta, a company whose main product, Metavision, uses a “natural user interface” – a term popularized by Mann to describe graphical interfaces that can be controlled with intuitive and natural gestures. (That term should become more popular with the adoption of augmented and virtual reality.)
Steve in his office at Meta with his “Augmented reality desk of the future” using his SWIM invention. The floating interface allows him to design a violin by making adjustments to the actual sound waveform it produces, and interacting with the system via PAR (Phenomenal Augmented Reality) and hand tracking.
Metavision is a natural next step for the man who created the EyeTap: Real life with a 3D digital overlay controlled by natural hand gestures. (Mann refers to the use of his device as “augmediated reality,” as it provides this digital overlay, but with the added benefit of better vision via HDR.) Two friends wearing headsets can have a virtual chess game that appears to play out in real life, with a chessboard resting on a table between them, and their hands moving the pieces as if they were physically in front of them.
It’s through the introduction of transformational devices like this that people are finally grasping what Mann envisioned decades ago. And, given his head start, there’s a lot he can tell us about where things are and should be going.
Unlike today’s most prominent tech evangelists, Mann hopes that with the rabid excitement about technology will come the requisite caution when it comes to its implications for the future. “I’ve seen some darker things of the dark future that we don’t want,” Mann says, “and I try to warn people of it.”
Hopefully, we’ll all start listening.
Stephen Baldwin is a Toronto-based journalist who writes about tech, business, and other things.