This AI program interprets the SAT at the same level as the average American

For the first time, an artificial intelligence software program capable of seeing and reading, has answered questions from the SAT at the level of an average 11^th grade student.

Developed from the Allen Institute for Artificial Intelligence (AI2), the GeoS combines optical character recognition (OCR) algorithms to read the text, and custom language processing to comprehend what it reads.

Under this capacity, the AI program was able to score 49% accuracy on official SAT geometry questions, which is almost identical to the results of real human test-takers, and  61% in practice questions. GeoS was not given digitized or specially labeled versions of the test, but rather the exact layout and questions students receive. It read the text, interpreted the diagrams, analyzed the question and then came up with a solution.

As you can plainly see, the AI is not entirely without fault, producing an error when unable to identify specifics words or decipher the grammar used in phrasing the question. But if these factors prove none-issues, GeoS will always select the correct answer, whereas a high school may have no trouble interpreting the question, but may apply the incorrect formula.

The Allen Institute’s AI approach relies on logic and reason, as geometry questions are dense, difficult to tear apart, and very uniform in subject and structure. GeoS’s programmers planned for the design principles that go into writing the questions. A successful robot is easy to adapt to the distinguishing rules of the SAT.

One intriguing development that may come from this AI program someday is use for algorithms to quality-check SAT questions. There could be different AI programs intended to achieve different levels of success on average questions.

But need not worry: there is not a sign of an AI-pocolyse coming. The geometry questions may be dense, but they are not as complex as something like conversational speech.

“Believing that writing these types of programs will bring us closer to real artificial intelligence is like believing that someone climbing a tree is making progress toward reaching the moon,” said Hubert Dreyfus, a philosophy professor at the University of California, Berkeley.

Source: ExtremeTech