Professor Martin Hackl hopes to trace the link between language and
thought by looking through ...
The Windows of the Mind
If eyes are indeed the windows of the soul, then for Martin Hackl,
assistant professor of linguistics and cognitive science, they are also
a microscope that permits a closer study of the mind.
Eyes—in particular, eye movements—are at the root of current research
being done by the Pomona College linguist, who hopes to use his findings
to shed new light on an age-old problem: the relationship between
language and thought.
Once common wisdom, the notion that the ability to think is dependent
upon the ability to speak—that is, that we think mostly or entirely in
words—has long since been debunked. In recent years, debate among
scholars has raged over whether or not language plays any sort of role
in our ability to think. Theories abound, but due to the subjective
nature of the phenomenon, evidence for either view has been lacking. By
tracking the movements of his subjects’ eyes, however, Hackl hopes to
create a new body of objective data that may point the way to new
understandings of how the mind works.
Specifically, by tracing where a person moves his eyes when figuring out
a problem on a computer screen under time pressure, Hackl investigates
to what extent linguistic form determines problem-solving strategies.
For instance he has demonstrated that people consistently have a more
difficult time proving the concept “B has fewer items than A’’ than they
do “A is has more items than B’’—even though the two statements differ
only in form. That conclusion in and of itself—that people have a
tougher time with the “fewer than’’ concept—is not a new one; academic
researchers have seen it many times before. But though his research is
still ongoing, Hackl’s innovative experiments are the first ever to do
it with eye-tracking data—and the first to offer a tantalizing clue as
to why it might be so.
“That’s entirely new,’’ says Hackl. “Nobody has seen that before.’’
And if Hackl is correct in his interpretation of his early data, he may
have found, for the first time, a real, demonstrable link between
non-verbal reasoning and the language faculty—that innate knowledge of
the structure of language that, according to linguist Noam Chomsky,
humans are born with.
“Now, nobody has actually ever shown anything like that,’’ he adds with
a smile, “so if this really comes out right, this would be a spectacular
While the research involved with Hackl’s project is long and arduous,
the linguist says it will be worth it when he can write up the research
and inspire people to talk about its implications. “As the data has been
coming in, it’s been looking good, so I just keep plugging on,” he says.
“I’ve shown the basic stuff and pilot data at about three or four places
already—at research colloquia—because I have enough to show that I don’t
mind doing it,” he adds. “And so far the reaction has been very, very
The ambitious nature of the research isn’t surprising, note Hackl’s
colleagues, given his reputation as a maverick in his field. He’s one of
the few able to deftly navigate the academic waters of both linguistics
and psychology, capable of spinning out nimble works of logic in his
published papers on formal semantics, as well as being right at home
doing empirical research as a psychologist.
“It’s very difficult to find people doing both linguistics and cognitive
science,” says Professor Deborah Burke, chair of the Linguistics and
Cognitive Sciences Department. “There are very few people doing the
types of things he’s doing in the lab. He’s really doing some very brave
Adds Professor William Banks, a professor of psychology who teaches in
the same department as Burke and Hackl: “(Former President) Peter
Stanley said when he created the Linguistics and Cognitive Sciences
Department that this may possibly be the only liberal arts college in
the country that could field a complete department of linguists and
cognitive scientists—it’s really unusual, and I’d certainly say Martin
is one of the legs we’re standing on.”
Thanks to a gift from Trustee Emerita Joan Hanley, Hackl was given the
funds to set up the Eye Tracker Lab at Pomona—a facility that Hackl says
is one of the reasons he loves being at the college. Eye tracking, in
its various forms, has been around for many years, but eye-tracking
labs—with their high-tech equipment that enables researchers to record a
person’s eye movements onto a computer screen in front of them—have
opened up rich new research opportunities.
“It’s wonderful to have it here,” says Hackl. “I’m envied—and
“A lot of major research universities have eye-tracking labs, but very
few universities in the liberal arts do,” adds Burke.
Arriving at Pomona in 2002, Hackl designed and eventually began running
experiments to test some ideas that had grown out of his Ph.D.
dissertation at M.I.T., based in part on the work of his new colleague,
Banks. Back in the 1970s, Banks had also shown this kind of “less than”
problem-solving was more difficult for people to do, but he had used
only reaction times as a factor. Hackl wanted to add one important new
dependent variable: eye movements.
To give a thumbnail picture of the procedure involved in Hackl’s
experiment is to simplify many of its subtle machinations, but the
process works something like this:
In his lab on the first floor of Mason Hall, Hackl—or one of his student
researchers—positions a subject in front of a computer and places on the
person’s head a visor-like piece of equipment holding two miniscule
cameras and small infrared light. One of the cameras records the
movements of the subject’s eyes as they dart about the computer screen.
On the screen in front of the subject appears a series of juxtaposed
images to be used in answering questions. The researcher then asks
something like: “True or false—the pig has more apples than the cow.” A
few moments later, with a new version of the image on the screen, the
question becomes: “True or false—the cow has less apples than the pig.”
Each time, the camera traces how the subject’s eyes move.
And how they move, according to Hackl, reflects a consistent pattern.
Coming up with the answer to the first question, in the majority of
cases, requires three clear eye movements across the screen—to the pig,
to the cow, to the answer prompt. The second question, however, usually
requires four movements—pig, cow, back to the pig, and finally to the
The questions are logically equivalent, and yet, for most people, the
“less-than” query seemingly requires one more problem-solving step. The
subject appears to work just a bit harder, seems a tad less certain of
the result. The eye wanders back for one last look.
Why is it harder for people to do these problems?
Hackl has a hypothesis that seems to explain what he sees in his data:
The mind isn’t tackling the second question, he theorizes, in the same
way as the first. “Because if you did, you would just be doing the same
pattern as when you’re asked, ‘Is A more than B?’ You’d just go ‘boom,
boom,’” Hackl explains, gesturing to indicate a simple eye movement from
one spot to the next. “It would be the same thing. There would be
nothing to it.”
Instead, he posits, the brain may have to translate the question back
into a “more-than” question in order to answer it—hence that final
tell-tale eye movement. “So you look from A to B and you ask, ‘Is B more
than A?’ rather than ‘Is B fewer than A?’ Since the answer is no, you
have to look back and ask, ‘But is A more than B?’ You see, the eyes are
weird creatures. They only seem to ask more questions.”
That translation of the question into its opposite, he notes, is the
kind of thing the language faculty does all the time. “The ‘fewer than’
thought might not exist outside of the language faculty,’’ Hackl posits.
“It might not exist for systems that gather information from
the world. It is something that’s a core part of the language faculty,
but it might
not exist outside of it. So that’s my hypothesis.’’
“Now, of course, if this is right,’’ he adds, “that would mean that
certain thoughts—like a ‘fewer than’ thought—you can only formulate if
you have a language faculty. If you don’t have a language faculty, you
can’t have them. You can have equivalent thoughts—like a ‘more than’
thought—but you can’t actually have a ‘fewer than’ thought. There are
certain thoughts you can only have if you have a language faculty.”
Of course, there’s still much more research needed to prove such a
hypothesis. Hackl says he wants to make sure he’s got more than enough
empirical data to blunt the skepticism he knows other cognitive
scientists will aim his way.
In the meantime, the linguist says there is much about this eye-tracking
work he has found captivating. Just designing the computer exercises,
the true-or-false problems, is quite a challenge, he notes. For example,
the little scenes—what he calls “toy worlds” can’t be too
distracting—but they can’t be too boring, either, because you’ve got to
keep the subject’s eyes ready to give their best effort, time after
“The way we do it is we show a little story, like a slide show,” he
says. “We design all this and the timing is very delicate, because the
subjects lose their attention very quickly if you don’t get the timing
right. It’s basically like editing a movie. It’s really difficult ...
and it’s a lot of man hours.”
He adds, however, that he continues to enjoy the work in the lab. “If I
didn’t, I wouldn’t spend so much of my time and effort on it.”
Paul Sterman ’84 is a freelance writer living in Orange, California.