What if the touchscreen of your smartphone or tablet could touch you back? What if touch was as integrated into our ubiquitous technology as sight and sound?

Carnegie Mellon University and Northwestern University researchers have made a discovery that provides insight into how the brain makes sense of data from fingers.

In a study of people drawing their fingers over a flat surface that has two "virtual bumps," the research team is the first to find that, under certain circumstances, the subjects feel only one bump when there really are two. Better yet, the researchers can explain why the brain comes to this conclusion.

Their article, to be published the week of Feb. 9 in the Proceedings of the National Academy of Sciences (PNAS), describes a new mathematical model and experimental results on "haptic illusions" that could one day lead to flat-screen displays featuring active touchback technology, such as making your touchscreen's keyboard actually feel like a keyboard. Tactile information also could benefit the blind, users of dashboard technology in cars, players of video games and more.

"How does your body and mind interpret something flat and 'see' it as having shape and texture?" asked Roberta Klatzky, the Charles J. Queenan Jr. Professor of Psychology and Human-Computer Interaction at Carnegie Mellon and a world-renowned expert in cognitive psychology and haptic (touch-based) perception. "An important step toward effective surface haptics is to understand what kinds of stimulation might lead you to feel something other than uniform flatness when you touch the surface of your device. Our study contributes to this understanding."

Forces felt by the fingers as they travel along a flat surface can lead to the illusion that the surface actually contains bumps. This so-called "virtual bump illusion" is well known in the haptics field, and the research team was able to make use of it.

"Touch is so important in our real world, but it is neglected in the digital world," said J. Edward Colgate, the Allen and Johnnie Breed University Professor of Design at Northwestern's McCormick School of Engineering and Applied Science and an expert in touch-based systems. "We want to create something that will make touch a reality for people interacting with their screens, and this work is a step in that direction.

"By leveraging the virtual bump illusion, we were able to design a meaningful experiment that shed light on the way the brain integrates information from multiple fingers. Our big finding was 'collapse' — the idea that two virtual bumps are perceived as one if they are experienced nearly simultaneously," Colgate said.

The research team's experiment presented two virtual bumps, with the distance between them varying across trials, to subjects participating in the study. When bump and finger spacing were identical, subjects reported feeling two bumps as one. In this case, the brain thinks it is too coincidental that there should be two bumps at the same time, so it registers the bumps as one.

"Our findings will help us and other researchers figure out how to design haptic technology to produce certain tactile effects," said Michael A. Peshkin, a professor of mechanical engineering at the McCormick School. "Haptics — giving a feel to objects — just enhances the physicality of a person's experience."

Steven G. Manuel, the study's first author and a Northwestern alumnus, developed the model of where the "illusion of protrusion" comes from. It describes how the brain constructs a mental depiction of the surface using sensory signals from two fingers as they explore a surface over time and space.

A critical feature of the model, and one found in theories of perception more generally, is that it assumes the brain is biased toward inferring causes rather than registering coincidences. In essence, as the fingers encounter forces while they explore a flat surface, the brain creates virtual bumpiness that is most consistent with the physical bumps that would produce the same sensations.

The work was supported by a National Science Foundation Division of Information and Intelligent Systems grant, Surface Haptics via Tractive Forces.

Klatzky is a faculty member in the Center for the Neural Basis of Cognition (CNBC), a joint program between Carnegie Mellon and the University of Pittsburgh devoted to investigating the neural mechanisms that give rise to human cognitive abilities. The CNBC's legacy of work is one example of how Carnegie Mellon has been a longtime leader in the study of brain and behavior.

As the birthplace of artificial intelligence and cognitive psychology, Carnegie Mellon has created the first cognitive tutors, helped to develop the Jeopardy-winning Watson, founded a groundbreaking doctoral program in neural computation and completed cutting-edge work in understanding the genetics of autism. Building on its strengths in biology, computer science, psychology, statistics and engineering, CMU recently launched BrainHub, a global initiative that focuses on how the structure and activity of the brain give rise to complex behaviors.

For more information, visit http://www.psy.cmu.edu/people/klatzky.html.