By Paul Sterman '84

If eyes are indeed the windows of the soul, then for Martin Hackl, assistant professor of linguistics and cognitive science, they are also a microscope that permits a closer study of the mind.

Eyes—in particular, eye movements—are at the root of current research being done by the Pomona College linguist, who hopes to use his findings to shed new light on an age-old problem: the relationship between language and thought.

Once common wisdom, the notion that the ability to think is dependent upon the ability to speak—that is, that we think mostly or entirely in words—has long since been debunked. In recent years, debate among scholars has raged over whether or not language plays any sort of role in our ability to think. Theories abound, but due to the subjective nature of the phenomenon, evidence for either view has been lacking. By tracking the movements of his subjects’ eyes, however, Hackl hopes to create a new body of objective data that may point the way to new understandings of how the mind works.

Specifically, by tracing where a person moves his eyes when figuring out a problem on a computer screen under time pressure, Hackl investigates to what extent linguistic form determines problem-solving strategies. For instance he has demonstrated that people consistently have a more difficult time proving the concept “B has fewer items than A’’ than they do “A is has more items than B’’—even though the two statements differ only in form. That conclusion in and of itself—that people have a tougher time with the “fewer than’’ concept—is not a new one; academic researchers have seen it many times before. But though his research is still ongoing, Hackl’s innovative experiments are the first ever to do it with eye-tracking data—and the first to offer a tantalizing clue as to why it might be so.

“That’s entirely new,’’ says Hackl. “Nobody has seen that before.’’

And if Hackl is correct in his interpretation of his early data, he may have found, for the first time, a real, demonstrable link between non-verbal reasoning and the language faculty—that innate knowledge of the structure of language that, according to linguist Noam Chomsky, humans are born with.

“Now, nobody has actually ever shown anything like that,’’ he adds with a smile, “so if this really comes out right, this would be a spectacular finding.”

While the research involved with Hackl’s project is long and arduous, the linguist says it will be worth it when he can write up the research and inspire people to talk about its implications. “As the data has been coming in, it’s been looking good, so I just keep plugging on,” he says. “I’ve shown the basic stuff and pilot data at about three or four places already—at research colloquia—because I have enough to show that I don’t mind doing it,” he adds. “And so far the reaction has been very, very positive.”

The ambitious nature of the research isn’t surprising, note Hackl’s colleagues, given his reputation as a maverick in his field. He’s one of the few able to deftly navigate the academic waters of both linguistics and psychology, capable of spinning out nimble works of logic in his published papers on formal semantics, as well as being right at home doing empirical research as a psychologist.

“It’s very difficult to find people doing both linguistics and cognitive science,” says Professor Deborah Burke, chair of the Linguistics and Cognitive Sciences Department. “There are very few people doing the types of things he’s doing in the lab. He’s really doing some very brave things.”

Adds Professor William Banks, a professor of psychology who teaches in the same department as Burke and Hackl: “(Former President) Peter Stanley said when he created the Linguistics and Cognitive Sciences Department that this may possibly be the only liberal arts college in the country that could field a complete department of linguists and cognitive scientists—it’s really unusual, and I’d certainly say Martin is one of the legs we’re standing on.”

Thanks to a gift from Trustee Emerita Joan Hanley, Hackl was given the funds to set up the Eye Tracker Lab at Pomona—a facility that Hackl says is one of the reasons he loves being at the college. Eye tracking, in its various forms, has been around for many years, but eye-tracking labs—with their high-tech equipment that enables researchers to record a person’s eye movements onto a computer screen in front of them—have opened up rich new research opportunities.

“It’s wonderful to have it here,” says Hackl. “I’m envied—and justifiably so.”

“A lot of major research universities have eye-tracking labs, but very few universities in the liberal arts do,” adds Burke.

Arriving at Pomona in 2002, Hackl designed and eventually began running experiments to test some ideas that had grown out of his Ph.D. dissertation at M.I.T., based in part on the work of his new colleague, Banks. Back in the 1970s, Banks had also shown this kind of “less than” problem-solving was more difficult for people to do, but he had used only reaction times as a factor. Hackl wanted to add one important new dependent variable: eye movements.
To give a thumbnail picture of the procedure involved in Hackl’s experiment is to simplify many of its subtle machinations, but the process works something like this:

In his lab on the first floor of Mason Hall, Hackl—or one of his student researchers—positions a subject in front of a computer and places on the person’s head a visor-like piece of equipment holding two miniscule cameras and small infrared light. One of the cameras records the movements of the subject’s eyes as they dart about the computer screen.
On the screen in front of the subject appears a series of juxtaposed images to be used in answering questions. The researcher then asks something like: “True or false—the pig has more apples than the cow.” A few moments later, with a new version of the image on the screen, the question becomes: “True or false—the cow has less apples than the pig.”
Each time, the camera traces how the subject’s eyes move.

And how they move, according to Hackl, reflects a consistent pattern. Coming up with the answer to the first question, in the majority of cases, requires three clear eye movements across the screen—to the pig, to the cow, to the answer prompt. The second question, however, usually requires four movements—pig, cow, back to the pig, and finally to the prompt.

The questions are logically equivalent, and yet, for most people, the “less-than” query seemingly requires one more problem-solving step. The subject appears to work just a bit harder, seems a tad less certain of the result. The eye wanders back for one last look.
Why is it harder for people to do these problems?

Hackl has a hypothesis that seems to explain what he sees in his data:
The mind isn’t tackling the second question, he theorizes, in the same way as the first. “Because if you did, you would just be doing the same pattern as when you’re asked, ‘Is A more than B?’ You’d just go ‘boom, boom,’” Hackl explains, gesturing to indicate a simple eye movement from one spot to the next. “It would be the same thing. There would be nothing to it.”

Instead, he posits, the brain may have to translate the question back into a “more-than” question in order to answer it—hence that final tell-tale eye movement. “So you look from A to B and you ask, ‘Is B more than A?’ rather than ‘Is B fewer than A?’ Since the answer is no, you have to look back and ask, ‘But is A more than B?’ You see, the eyes are weird creatures. They only seem to ask more questions.”

That translation of the question into its opposite, he notes, is the kind of thing the language faculty does all the time. “The ‘fewer than’ thought might not exist outside of the language faculty,’’ Hackl posits. “It might not exist for systems that gather information from
the world. It is something that’s a core part of the language faculty, but it might
not exist outside of it. So that’s my hypothesis.’’

“Now, of course, if this is right,’’ he adds, “that would mean that certain thoughts—like a ‘fewer than’ thought—you can only formulate if you have a language faculty. If you don’t have a language faculty, you can’t have them. You can have equivalent thoughts—like a ‘more than’ thought—but you can’t actually have a ‘fewer than’ thought. There are certain thoughts you can only have if you have a language faculty.”

Of course, there’s still much more research needed to prove such a hypothesis. Hackl says he wants to make sure he’s got more than enough empirical data to blunt the skepticism he knows other cognitive scientists will aim his way.

In the meantime, the linguist says there is much about this eye-tracking work he has found captivating. Just designing the computer exercises, the true-or-false problems, is quite a challenge, he notes. For example, the little scenes—what he calls “toy worlds” can’t be too distracting—but they can’t be too boring, either, because you’ve got to keep the subject’s eyes ready to give their best effort, time after time.

“The way we do it is we show a little story, like a slide show,” he says. “We design all this and the timing is very delicate, because the subjects lose their attention very quickly if you don’t get the timing right. It’s basically like editing a movie. It’s really difficult ... and it’s a lot of man hours.”

He adds, however, that he continues to enjoy the work in the lab. “If I didn’t, I wouldn’t spend so much of my time and effort on it.”

Paul Sterman ’84 is a freelance writer living in Orange, California.