Pittsburgh, PA
October 4, 2022
    News           Sports           Lifestyle           Classifieds           About Us
Health & Science
Place an Ad
Running Calendar
Travel Getaways
Headlines by E-mail
Home >  Health & Science >  Science Printer-friendly versionE-mail this story
CMU work aims to change relationship between vehicle, driver

Monday, November 04, 2002

By Byron Spice, Post-Gazette Science Editor

Hand gestures have always been second nature to drivers, but researchers at Carnegie Mellon University hope to prove they are good for far more than thanking the guy who let you into his lane or flipping off the idiot who cut you off.

Marcel Just's studies at CMU's Center for Cognitive Brain Imaging have highlighted how even small distractions diminish a driver's concentration. (Lake Fong, Post-Gazette)

Pointing or waving also might be used to control the growing number of electronic systems in cars.

A few jabs with your index finger toward icons projected onto the windshield might let you adjust the radio, put a cell phone caller on hold or program the car's on-board navigation system without ever taking your eyes off the road.

"It's better than fumbling with the dials of a radio," contended Ed Schlesinger, co-director of the General Motors Collaborative Laboratory at Carnegie Mellon.

And this "gesture interface," being developed and tested at CMU on a specially outfitted Pontiac minivan, is just one example of how the relationship between car and driver could change. Not only might cars respond to voice commands, or demand a fingerprint verification before starting up, but the vehicle itself might keep an eye on the driver.

Cameras would watch for the gestures that would serve as the driver's commands. But they also could be used to verify the driver's identity, to look for signs of fatigue, or to just monitor what the driver is doing, so that the on-board systems know when the driver is too busy to be bothered by computerized navigation advice or an incoming cell phone call.

"What's changing, of course, is the nature of cars," said Marcel Just, a psychologist who directs CMU's Center for Cognitive Brain Imaging. An increasing number of car systems are being controlled electronically -- windshield wipers that automatically respond to rain, headlights that turn on when darkness falls, navigation systems that give directions and draw maps for the driver. And when people get into cars, they are often bringing along personal electronics, such as cell phones, personal digital assistants and laptop computers.

What isn't changing, Just added, is the capacity of the human brain. That is highlighted by some of his research, which uses functional magnetic resonance imaging to study brain activation in people in simulated driving situations. As he reported last year, these studies have shown that drivers who are talking on cell phones -- or even to a passenger -- are less able to concentrate on the road.

  More on the story:

No more steering wheel?

Online graphic:
Eyes on the road


Yet the distractions are only increasing. The GM-sponsored lab at CMU, in fact, may well accelerate this trend, given its mission to develop the next generation of vehicle information technology, a field known as telematics.

Wireless networks already link some vehicles with computer networks that provide information, summon roadside assistance, or allow occupants to read e-mail. GM's OnStar system, for instance, will dial phone calls in response to voice commands, as well as provide emergency and navigation assistance.

These networks, combined with Global Positioning Satellite technology, will grow more sophisticated. It may be possible, Schlesinger said, for vehicles to talk with each other over these networks, swapping information about location, speed and road conditions that could give drivers instant warnings about traffic accidents and other tie-ups.

The goal, Schlesinger said, is to present this information in a way that enhances the driving experience and increases safety, rather than simply adding to the burden on a driver's attention. Interacting with these systems, he added, should be no more distracting to the driver than talking with a passenger.

Sensitive to the driver's needs

One solution, suggested Asim Smailagic, director of CMU's Laboratory for Interactive Computer Systems, is to design the car's computer systems so that they are "context aware." That is, the computer system knows enough about the driver and about the vehicle's surroundings that it anticipates when the driver needs certain information, when the driver needs a reminder and when the driver just needs to be left alone.

To see what technology might be used to make that a reality, students in Smailagic's rapid prototyping class in spring 2001 outfitted a Pontiac minivan with a number of special sensors and tested them in real driving situations.

When senior Mike Beattie slips behind the wheel, for instance, a camera attached above the windshield records his face and a computer identifies him. The message, "Welcome, Mike," flashes on a small computer monitor mounted to his right. Likewise, a fingerprint reader mounted left of the steering column recognizes his thumbprint.

The system might limit use of a vehicle to authorized users, as well as automatically recall an individual's preferred seat position and mirror adjustments, and access personal information, such as the driver's daily schedule.

A microphone mounted on the shoulder belt picks up verbal commands from the driver, to place a phone calls, for example, or a request for information to be displayed on the computer monitor. Information such as vehicle speed, a fuel tank gauge and driver alerts is projected onto a heads-up display, a transparent screen mounted on the windshield.

Not visible is an "intelligent agent," software that tries to anticipate the driver's needs. When the on-board GPS unit shows the vehicle approaching a dry cleaner, for instance, the system may remind the driver to stop and pick up his laundry.

The driver can end cell phone calls by waving his right hand, activating a "gesture cam" mounted on the floor between the driver and passenger seats.

GM engineers were particularly intrigued by the gesture interface, Smailagic said. This past summer, students in CMU's Human-Computer Interaction Institute further developed the interface, designing a system in which computer menus for operating the radio and navigation systems were projected on the heads-up display. Drivers could change menus or make selections by hand movements.

The interface wasn't fully functional, operating in what student Matt Hornyak described as "Wizard of Oz mode." That is, humans interpreted the hand gestures, rather than a camera/computer system. But the experiments suggested that drivers found the gesture system easy to use without taking their eyes off the road.

It's hard to say how many of these ideas will eventually become reality.

"There's bound to be parts of it that will solve problems as we go along," said Dick Johnson, a researcher at GM's Electrical and Controls Integration Lab in Warren, Mich. But the system the students put together was intended as a testbed and likely won't be used in its entirety in any production vehicle. For now, GM is interested in seeing what kind of ideas emerge from the CMU lab.

"They have a lot of really bright students," he added.

Is the driver drowsy?

One aspect that might be expanded, Smailagic suggested, is the use of cameras to monitor driver conditions. Monitoring the driver's blink rate, for instance, might detect a drowsy driver. Pupil dilation could be an indicator of driver attention and, if it could be monitored adequately by the camera, could provide hints about when a driver is becoming overloaded with tasks.

This problem, called workload management, is one that already worries automakers.

"They're groping for some way to measure how hard your mind is working," said Just, noting that researchers from more than one auto company have paid visits to his brain imaging lab.

Judging mental workload based on fMRI scans, as Just does in his lab, is not practical in an automobile. But visually monitoring a driver's eyes, as Smailagic suggests, may be an acceptable substitute.

Researchers at the University of New Hampshire, for instance, have done eye-tracking experiments that show fewer eye movements to the side of the road when a driver is talking to a passenger. Likewise, eye-tracking studies at the University of Madrid show drivers glance in the rearview mirror less often when engaged in conversation.

Those findings are consistent with Just's fMRI studies. "Logically, it fits together very well right now," he added. Just will soon be able to test this relationship directly, with a new apparatus that will track eye movements simultaneously with the fMRI brain activation scans.

Monitoring vehicle speed and direction, monitoring road conditions, or even just analyzing the driver's schedule might also provide clues about the driver's workload.

With costs of most technologies declining over time, it's almost always possible to add new technical capabilities to a vehicle, Smailagic said, but the human attention span remains constant.

"We have to be very careful how we spend that precious resource," he added.

Byron Spice can be reached at bspice@post-gazette.com or 412-263-1578.

Search | Contact Us |  Site Map | Terms of Use |  Privacy Policy |  Advertise | Help |  Corrections