🍄 The Future Sense

Neuroscientists say that when it comes to our senses, there’s more than meets the eye. In a world “qualia,” we are resurrecting lost senses while discovering new ones.

Jan 19, 2024

Exploring every rabbit hole there is. For more wanderings, become an Alice in Futureland subscriber—it's free.

🍄 AudioDose: this is Alice on Sonic Mushrooms. Metasense here.
🎧 Alice podcasts
📘 Alice books: Tuning into Frequency: The invisible force that heals us and the planet

‘Not only do each of us perceive the world differently to anyone else that’s alive, but to anyone else that’s ever lived.’ —Biologist Ashley Ward, author of Sensational: A New Story of our Senses [2023], speaking on The Spectator podcast.

Hello, we’re Alice, and we are always in a state of wander. No tunnel vision present today. Whether “hearing” through touch or conjuring scent to prevent losing memory, scientists of senses such as sight and sound are using technology to augment, add to, and even swap our senses.

The way I see it …

Submerged in studies of the senses, neuroscientist David Eagleman had a hunch. The author and occasional scientific advisor for HBO’s Westworld, thinks that while humans haven't evolved to have certain senses – say, magnetoreception or night vision - doesn’t mean they can’t. “The interest in my lab for a long time has been what if we could actually push information into the brain via an unusual sensory channel and repair lost senses or build new ones?” Eagleman, who teaches courses on brain plasticity at Stanford University, tells CNN’s Chasing Life. “So we might actually be able to build new qualia and understand what that is like.”

In philosophy of mind, “qualia” is the subjective experience of a sense. “For example, when a neuroscientist looks at vision, it's all this activity in cells. And yet vision doesn't feel that way to you. It feels like, oh, look at that lovely orange and red sunset or, you know, the taste of cinnamon on your mouth or the smell of a campfire or something. These are qualia. It's your subjective experience, even though physically you just see spikes running around in the brain. And so the question is, could we have a new kind of qualia that is not something that we're used to getting through our eyes or ears or nose or mouth, things like that. In the same way that I can't imagine a new color, could I imagine or come to develop new qualia?”

Sensing something more?

Think of dinosaurs. They had feathers to keep them warm long before using them to fly, the late theoretical physicist John Wheeler, once told ALICE in a magical 2002 interview. “Do we have something, some faculty that we haven’t put to use the way the dinosaurs had put to use these feathers of theirs until later?” he asked. “It’s fantastic that evolution should have ended up with us. What other kind of creature could it have been?”

Eagleman concurs. “What is the brain capable of in terms of bringing in new senses and what could it do? Where could humans go? There's a sense in which this is the trajectory our species has been on from the beginning, which is, "Hey, let's build the Industrial Revolution and let's build the Internet and let's, you know, be tied into the entirety of humankind's knowledge with a rectangle in my pocket and so on," you know, we're constantly ratcheting things up,” he tells CNN. “And now I think this is the next step is understanding how do you want to experience the universe around you and what would it be like to tap into these other signals that are surrounding you but are invisible to you?”

Oh, make it make sense!

‘My own brain is to me the most unaccountable of machinery – always buzzing, humming, soaring roaring diving, and then buried in mud.’ – Virginia Woolf, in a letter to Ethel Smyth, 1932.

Biologist Ashley Ward says the field of the senses complex and ever changing. “We can detect gravitational waves in space, we can do all kinds of amazing things in the realm of the sciences but we don’t know how our sense of smell works,” Ward, author of Sensational: A New Story of our Senses [2023] tells The Spectator podcast.

“There’s no definitive answer for that. The same is true to a large extent about our sense of taste, indeed about any of our senses. We don’t have a full understanding of how basic sensations are translated into our perceptions, and that is going to be a fascinating challenge for the years going forward as we move from sensation to perception, and then ultimately, one hopes we get from perception to consciousness, the real lodestar at the centre of our existence.” So if consciousness, for now, is the part we can’t quite grasp, then human senses are certainly more tangible. Or are they?

We don’t passively take in the world around us; instead our mind is constantly making and refining predictions about what we expect to see,’ writes cognitive scientist and philosopher Andy Clark in The Experience Machine: How Our Minds Predict and Shape Reality [2023]. ‘This even applies to our bodies, as the way we experience pain, and other states, is shaped by our expectations, and this has broader implications for the understanding and treatment of conditions from PTSD to schizophrenia to medically unexplained symptoms.’

In an article titled, ‘The Mind-Expanding Ideas of Andy Clark,’ The New Yorker says ‘he dreams of a future in which his refrigerator will order milk, his shirt will monitor his mood and heart rate, and some kind of neurophone connected to his cochlear nerve and a microphone implanted in his jaw will make calling people as easy as saying hello.’ From the most mundane experiences to the most sublime, cognitive scientist and philosopher Andy Clark praises the power of our predictive brain to sculpt our experience.

What are the haptics? What’s the buzz?

Eagleman is the co-founder of Neosensory, a Cali-tech company building non-invasive brain-machine interfaces to create new senses. “About a decade ago, I got really interested in whether we can create new senses,” he told New Scientist in 2021. “You have your eyes, ears and nose, but when you look across the animal kingdom, you find animals with detectors that can pick up on things like magnetic fields, electrical fields or ultraviolet light. It just depends what sensors they have. I began to understand our sense organs as “plug and play” detectors. Nature doesn’t have to redesign the brain every time she makes a new detector. Instead, she tinkers with different ways of sensing energy. That opens up the idea of creating new kinds of detectors to plug in.”

First, they designed the VEST (Versatile Extra-Sensory Transducer) a sensory augmentation wearable made up of 32 vibrating motors. The VEST could translate any kind of real-time data into patterns of vibration on the skin— and even featured in an episode of Westworld. Then they shrank it, to make Buzz, a Fitbit-style wristband that captures sound and turns it into patterns of vibration on the skin. The device helps people with hearing impairments ‘hear’ by feeling vibrations. These follow the nerves up to the brain, which reaches an understanding of it.

Asked if he could go further, Eagleman responds: “Yes. I live in Silicon Valley and everything here is about hardware and software. But what’s happening in the brain suggests a completely different approach to building technology—call it live-ware. So I’m interested in building systems that aren’t just software but physically reconfigure themselves based on experiences like the brain does. In this way, it would become fast and efficient at the tasks that it does a lot. I feel like we are at the foot of the mountain looking up at it. At the moment, we have no idea how to build this kind of machinery. But I’m excited to see what will happen in the next few decades.”

Ready for your upgrade?

Science is discovering that we have more than 53 senses in which we experience the world—and what we individually know of as ‘reality.’ Physiologically, there are electrical signals generated by our sensory cells and afferent nerve fibers that inform our sensory mechanisms. And it turns out that our cells can sense—they can smell, taste, hear and feel/touch. We generally understand olfactory receptors to be located in the nose and to play the central role in our sense of smell, however, these receptors are also found in skin cells and can regulate cell functions. One in particular, OR2AT4, has been associated with mitigating keratinocyte proliferation, migration and regeneration. We also have taste receptors sprinkled around the body in places such as the liver and the brain.

What constitutes a sense is a still a matter of some debate. As Bruce Durie wrote in his article “Senses Special: Doors of Perception” for New Scientist, “When we talk of senses, what we really mean are feelings or perceptions.” The Federico and Elvia Faggin Foundation, a non-profit organization dedicated to the scientific study of consciousness, recognizes that “there is no known physical principle that can translate electrical activity in the brain or in a computer into sensations or feelings.” The foundation supports research to advance our understanding of consciousness under the assumption that it is an irreducible property of nature. But more to come on this, as ALICE visits with Federico Faggin, Italian physicist, engineer, inventor of the first commercial microprocessor, who talks more about the new science of consciousness, qualia (the inner reality of sensations and feelings) and future senses in our next podcast. Stay tuned.

What else we are wandering…

How little our senses tell us, so that we are constantly obliged to use apparatuses in order to analyze things.—19th-century French physiologist Étienne-Jules Marey

🔍 Sense sleuthing
The man behind the birth of big data and statistics used from Apple to Google is Gustav Fechner, a German medical doctor, philosopher and physicist, who went blind overnight in 1840. ‘Little did the scientist know, however, that his dire situation would eventually result in something remarkable,’ writes Chris Salter in Sensing Machines: How Sensors Shape Our everyday Life [MIT Press, 2022]. ‘A startling revelation that would forever change our understanding of the human senses and how they would come to interact with machines.’

Fechner took intensive short-term exposure to bright light everyday, quickly closing his eyes before it caused pain. ‘One day, he removed the thick bandages covering his eyes,’ writes Salter, Professor of Design and Computation Arts at Concordia University. ‘The light spilled in. As he glanced into his garden, the scientist experienced a miraculous sight. He saw the flowers “glowing.” They seemed to speak to him. In this ecstatic moment, Fechner came to an astonishing realization — plants must also have souls.’

Fast-forward 180 years, says Salter. ‘Dozens of jobs in new professions with strange sounding titles appear: vision engineer, applied perception scientist, visual experience researcher, color scientist, and neural interface engineer.’ Fechner founded all of this when he invented “psychophysics” — a “theory of the relations between body and mind” that aimed to establish a measurable connection between two spheres that had long remained separate: the material, physical universe and the mental, psychological one. ‘In Fechner’s formulation, psychophysics would be an “exact science, like physics” and “rest on experience and the mathematical connection of those empirical facts that demand a measure of what is experienced.” He asserted, in other words, that we can measure and calculate how we sense the world using mathematics, forever changing how we view sensing and perception in relationship to man-made machines.’

🔍 Let’s touch on touch
Our body has a language—one that includes the sense of touch, which is unique. In fact, people have a "touch language". A group of researchers from USC Viterbi School of Engineering and Stanford explored if this "touch language" can be sensed remotely. They wanted to see if two companions (platonic or romantic), could communicate and express care and emotion remotely. The baseline for their study found that people perceive a partner's true intentions through in-person touch an estimated 57 percent of the time. They found that when interacting with a device that simulated human touch, respondents were able to discern the touch's intention 45 percent of the time.

The challenge for the researchers was to create an algorithm that can be flexible enough to incorporate the many dimensions of touch. Overall, 661 touch gestures were recorded. About 45 percent of the time, individuals wearing the armband were able to understand the intended emotion communicated via remote touch. The study proves that the social meaning and expression conveyed through in-person touch can be communicated remotely.

🧠 Not feeling it
The “touch starvation” people experienced during the coronavirus pandemic turns out to be a real thing, says biologist Ashley Ward. Humans have two different types of touch that are neurologically different. “We have very fast, high-speed kind of internet cables in our bodies that register all sorts of important touch to our brains and that’s what we’ve become accustomed to think of when we think of touch, we register when we stub our toe or brush against a chair,” Ward tells The Spectator. “But there is another kind of touch, although relative to the high-speed nervous apparatus which translates the previously described form of touch to our brains, these kind of fibers operate like little country back roads and travel much more slowly to the brain.” And yet there are far, far more of them and they potentially are more important. “These are fibers which respond to us being touched by another person, we may be caressed in a slow, gentle and loving way and its incredibly important. It really opens up our brains to all kinds of reward and pleasure in a way that we really hadn’t appreciated—or snickered at it, and thought of it as not scientifically important—and yet recent research has suggested just how important these fibers are to our well being.”

🧠 I feel you
There are a wave of new Long Distance Touch Bracelets that mimic the sense of touch between two people virtually and connected to a mobile application via Bluetooth. When one person touches their bracelet, the application triggers the other bracelet to vibrate and light up or squeeze, in the case of the Hey Bracelet.

Craving more?

📘 Alice in Futureland books
🎧 Alice in Futureland podcasts