Home » News » Computers Learn to Assess Emotions – Sometimes Better than Humans

Computers Learn to Assess Emotions – Sometimes Better than Humans

Computers Learn to Assess Emotions - Sometimes Better than HumansIn what sounds like science fiction, MIT researchers have determined that computers can distinguish if a person is smiling from delight, or smiling because they are frustrated.

Moreover, researchers say that computers programmed with the latest information from this research do a better job of differentiating smiles of delight and frustration than human observers do.

Researchers believe the findings could pave the way for computers to better assess the emotional states of their users and respond accordingly. Software could be developed to help train those who have difficulty interpreting expressions, such as people with autism, to more accurately gauge the expressions they see.

“The goal is to help people with face-to-face communication,” said doctoral student Ehsan Hoque, lead author of a paper just published in the IEEE Transactions on Affective Computing.

In experiments conducted at the MIT Media Lab, people were first asked to act out expressions of delight or frustration, as webcams recorded their expressions. Then, they were either asked to fill out an online form designed to cause frustration or invited to watch a video designed to elicit a delighted response — also while being recorded.

Researchers learned that when an individual was asked to pretend that they were frustrated, 90 percent of subjects did not smile. But when presented with a task that caused genuine frustration — filling out a detailed online form, only to then find the information deleted after pressing the “submit” button — 90 percent of them did smile, says Hogue.

Still images showed little difference between these frustrated smiles and the delighted smiles elicited by a video of a cute baby, but video analysis showed that the progression of the two kinds of smiles was quite different: Often, the happy smiles built up gradually, while frustrated smiles appeared quickly but faded fast.

In such experiments, researchers usually rely on acted expressions of emotion, Hoque says, which may provide misleading results.

“The acted data was much easier to classify accurately” than the real responses, he said. But when trying to interpret images of real responses, people performed no better than chance, assessing these correctly only about 50 percent of the time.

Researchers say that understanding the subtleties that motivate emotions is a major goal of this research. “People with autism are taught that a smile means someone is happy,” he said, but research shows that it’s not that simple.

While people may not know exactly what cues they are responding to, timing does have a lot to do with how people interpret expressions, he says, For example, former British prime minister Gordon Brown was widely seen as having a phony smile, largely because of the unnatural timing of his grin, Hoque said.

Similarly, a campaign commercial for former presidential candidate Herman Cain featured a smile that developed so slowly — it took nine seconds to appear — that it was widely parodied, including a spoof by comedian Stephen Colbert. “Getting the timing right is very crucial if you want to be perceived as sincere and genuine with your smiles,” Hoque said.

Dr. Jeffrey Cohn, a professor of psychology at the University of Pittsburgh who was not involved in this research, said this work “breaks new ground with its focus on frustration, a fundamental human experience. While pain researchers have identified smiling in the context of expressions of pain, the MIT group may be the first to implicate smiles in expressions of negative emotion.”

Cohn said, “This is very exciting work in computational behavioral science that integrates psychology, computer vision, speech processing and machine learning to generate new knowledge … with clinical implications.” He said this “is an important reminder that not all smiles are positive. There has been a tendency to ‘read’ enjoyment whenever smiles are found. For human-computer interaction, among other fields and applications, a more nuanced view is needed.”

In addition to providing training for people who have difficulty with expressions, the findings may be of interest to marketers, Hoque said. “Just because a customer is smiling, that doesn’t necessarily mean they’re satisfied,” he said. And knowing the difference could be important in gauging how best to respond to the customer, he said: “The underlying meaning behind the smile is crucial.”

Researchers believe the analysis may help to create computers that respond in ways appropriate to the moods of their users. One goal of the research of Affective Computing Group is to “make a computer that’s more intelligent and respectful,” Hoque said.

Source: MIT

Woman smiling at her computer photo by shutterstock.

Computers Learn to Assess Emotions – Sometimes Better than Humans

Rick Nauert PhD

Rick Nauert, PhDDr. Rick Nauert has over 25 years experience in clinical, administrative and academic healthcare. He is currently an associate professor for Rocky Mountain University of Health Professionals doctoral program in health promotion and wellness. Dr. Nauert began his career as a clinical physical therapist and served as a regional manager for a publicly traded multidisciplinary rehabilitation agency for 12 years. He has masters degrees in health-fitness management and healthcare administration and a doctoral degree from The University of Texas at Austin focused on health care informatics, health administration, health education and health policy. His research efforts included the area of telehealth with a specialty in disease management.

APA Reference
Nauert PhD, R. (2018). Computers Learn to Assess Emotions – Sometimes Better than Humans. Psych Central. Retrieved on December 5, 2020, from
Scientifically Reviewed
Last updated: 8 Aug 2018 (Originally: 30 May 2012)
Last reviewed: By a member of our scientific advisory board on 8 Aug 2018
Published on Psych All rights reserved.