Haptography And The Science of Touchscreen Texture (Interview)
Tuesday, September 18, 2012
Display Alliance in Engineers, Haptics, Haptography, Interviews, Interviews, Texture, Touch, Touchscreen, UPENN

Many thanks to Heather Culbertson from the Haptics Group at the University of Pennsylvania for talking to Display Alliance about haptics and her work on texture feedback for stylus-based tablets. You can read a little bit more about Heather at the end of the interview. To be interviewed for Display Alliance or to contribute your own content email me (Jason). I'm the managing editor of Display Alliance. I can also help you source display panels: jason@displayalliance.com

DA: What is haptics and what relevance does it have to our lives?

Haptics is the science of touch

Haptics is the science of understanding the sense of touch and augmenting a person's touch interactions with the physical world. The importance of the sense of touch to a person's daily life is often overlooked.

For example, try picking something up after your hand has fallen asleep. It's very difficult to do since you can't feel the shape of the object or determine if you are applying enough force to keep the object from slipping. Now try picking up the same object without looking. Even without sight you can use your sense of touch to help you adjust your hand until you have a stable grip on the object.

People are increasingly interacting with the world through technology such as a computer, smartphone, or tablet. But the only physical interactions that people have is with the keyboard or mouse, not with anything on the screen. Wouldn't it be great if we could add some touch sensations back to computing?

In the future, haptics could be applied to letting you feel a hug or a handshake from the person with whom you are video chatting. Or you could feel the tactile qualities of fabrics or clothing before you buy them online.

Haptics has been becoming more mainstream though. Many devices that people use everyday already use haptics. For example, the vibration in your phone or rumble packs in game controllers provide haptic cues to indicate that something has happened. But these haptic interactions do not feel realistic and leave a lot to be desired.

DA: What kind of problems do touchscreens pose for real-world applications?

The touch experience is distracting, unrealistic, and unreliable

One of the major problems with touch screens is that they require you to look at the screen in order to press any of the buttons.

Most likely you are able to dial the phone or change the radio in your car without looking if the device has traditional buttons, but this is very difficult to do with touch screens. There is no way to feel the position of buttons and the buttons are missing the distinct clicks that signify pressing and releasing. This causes touchscreens to require a lot more of the user's attention which can lead to distracted driving among other undesirable consequences. It also commonly leads to the incorrect pressing of buttons that has increased the necessity of autocorrect when texting or writing emails.

Furthermore, using touchscreens is often an unsatisfying experience because everything feels uncharacteristically smooth and slippery. For example, turning the page on an e-reader feels nothing like turning the page in a real book. If there was a way to make the objects on the screen feel more realistic, it would create a much more immersive experience for users.

DA: What are you researching at the moment? Is there a particular unsolved problem that you would love to solve?

Haptic texture feedback for stylus-based tablets

Currently I am working on providing realistic haptic texture feedback for stylus-based tablets.

If you pick up a tool such as a pen and gently drag its tip across the table, a rock, or the fabric of your shirt you are able to feel variations in the textures even though you are not directly touching them with your finger. This is because the contact between the tooltip and surface causes the tool to vibrate, which is in turn felt by your hand. These vibrations vary based on the texture, how hard you press with the tool, and how fast you move the tool.

My research focuses on ways to model and recreate these vibrations as a person drags a stylus across a virtual surface.

Haptography (haptic photography)

The modeling approach relies on data recorded from dragging a specially sensorized tool across real textured surfaces. The tool measures the induced vibrations as well as the speed and force the person used. From this data I am able to make mathematical models to represent the feel of the surface for reproduction on the tablet.

We've coined this method of data recording and modeling as haptography (haptic photography) because it allows a person to record the feel of an interesting interaction in much the same way that traditional photography allows a person to visually record an interesting scene or object.

Once the data is recorded and the model is made, I am able to recreate the feel of the surface on a tablet computer. The only additional hardware needed on the tablet side is a voice-coil actuator, which is attached to the stylus. The voice-coil works in much the same way as a speaker, but instead of outputting sound it outputs vibrations. These vibrations are much more controllable than the vibrations made by eccentric mass motors commonly found in cell phones and allow us to accurately reproduce the modeled vibrations to a user as he drags the stylus across a virtual texture. Thus we are able to create realistic textures on a completely smooth screen.

One unsolved problem I would love to solve is to apply my research to bare-finger touchscreens. I think everyone would agree that it would be really cool to be able to touch a screen with your finger and feel the buttons or different textured surfaces. But we are not yet to the point where we can display this tactile information without the additional voice-coil actuator hardware. This is something I would love to work on in the future though.

DA: As a young engineer what would you say about the world of technology and where it's headed?

Technology is for dreamers

I can honestly say that the world of technology never ceases to surprise me.

Looking back on the last 10-20 years I'm amazed at how much progress technology has made and how pivotal it has become to our everyday lives. I still remember when we first got internet in my house and now I can't imagine going more than a day without it.

Technology has invaded and enriched almost every aspect of our lives. I think that in the future it will become even more important to education and medicine. I'm really excited to see what new technologies emerge and what the creative minds of researchers can dream up.

Can you talk about the Haptics group at UPenn? Who is Charlie?

Toys and a parrot

The Haptics group at UPenn is directed by Dr. Katherine J. Kuchenbecker. Our group is very diverse with graduate and undergraduate students from mechanical engineering, robotics, electrical engineering, bioengineering, and cognitive science.

Current work in the lab includes haptic feedback for stroke rehabilitation, medical training, robot-assisted surgery, and teleoperation along with several other projects. Since haptic technology is by its definition hand-on experience, I like to say that we have some of the best toys in our lab.

We try to keep the lab a fun and exciting place to work because it really helps us stay creative and productive. One of the things that keeps it interesting is occasional visits by my parrot Charlie who will come to work with me and spend the day in the lab. He's always a big hit with the other students in the lab and even some of the professors at the school. Not to mention he certainly loves all of the extra attention he gets.

About Heather

Heather Culbertson is a third year PhD student in the Mechanical Engineering and Applied Mechanics (MEAM) department at the University of Pennsylvania. She works in the Haptics Group, a part of the GRASP Laboratory. Prior to joining UPenn, she earned her BS degree in mechanical engineering at the University of Nevada, Reno. You can email Heather for more information about her research.
Article originally appeared on Display Alliance (http://www.displayalliance.com/).
See website for complete article licensing information.