Artificial intelligence no match for 4-year-old

Robert Sloan: computer scored poorly on “why questions.” Photo: Roberta Dupuis-Devlin/UIC Photo Services (click on image for larger file size)

Artificial and natural knowledge researchers at UIC gave IQ tests to one of the best available artificial intelligence systems to see how intelligent it really is.

Turns out, it’s about as smart as the average 4-year-old, they reported July 17 at the U.S. Artificial Intelligence Conference in Bellevue, Wash.

The UIC team put ConceptNet 4, an artificial intelligence system developed at Massachusetts Institute of Technology, through the verbal portions of the Wechsler Preschool and Primary Scale of Intelligence Test, a standard IQ assessment for young children.

They found that ConceptNet 4 has the average IQ of a young child. But, unlike most children, the machine’s scores were uneven across different portions of the test.

“If a child had scores that varied this much, it might be a symptom that something was wrong,” said Robert Sloan, professor and head of computer science and one of the study’s authors.

Sloan said ConceptNet 4 scored well on tests of vocabulary and ability to recognize similarities.

“But ConceptNet 4 did dramatically worse than average on comprehension — the ‘why’ questions,” he said.

One of the hardest problems in building an artificial intelligence, Sloan said, is devising a computer program that can make sound and prudent judgments based on a simple perception of the situation or facts — the dictionary definition of common sense.

Common sense has eluded artificial intelligence engineers because it requires both a large collection of facts and what Sloan calls implicit facts — things so obvious that we don’t know we know them.

A computer may know the temperature at which water freezes, but we know that ice is cold.

“All of us know a huge number of things,” said Sloan. “As babies, we crawled around and yanked on things and learned that things fall. We yanked on other things and learned that dogs and cats don’t appreciate having their tails pulled. Life is a rich learning environment.

“We’re still very far from programs with common sense — AI that can answer comprehension questions with the skill of a child of 8.”

Sloan and his colleagues hope the study will help to focus attention on the “hard spots” in artificial intelligence research.

Study coauthors are UIC professors Stellan Ohlsson, psychology, and Gyorgy Turan, mathematics, statistics and computer science, and undergraduate Aaron Urasky.