Do Androids dream of handwriting recognition?

Do Androids dream of handwriting recognition?

Why, in 20 years of mobile, is this groundbreaking handwriting recognition feature just now being introduced? The answer is, it isn’t

BY KATIA SHABANOVA
Paragon Software Group
www.paragon-software.com

Engineers and programmers have been trying for decades to teach computers and other electronics to recognize handwritten text. Only in the last few years have the world’s largest software companies made significant progress teaching smartphones and tablets to adequately recognize handwriting and translate it into typed text on the screen.

This August, Samsung launched its Galaxy Note 10.1 tablet. As one of the key competitive advantages of the device, the manufacturer cited the S Pen recognition system, created in partnership with Wacom. A similar feature exists in the Galaxy Note II device, which Samsung recently presented at the Berlin Consumer Electronics Show IFA. The handwriting recognition function included in the device makes it just as comfortable to work on with your fingers as with the electronic S Pen. In the latest devices, the S Pen allows the transmission of up to 1,024 degrees of clicking on the screen — achieving essentially the same precise screen recognition as if writing with an actual pen and paper. The Galaxy Note also supports the Shape Match and Formula Match features, allowing graphs, figures, charts and mathematical formulas to be drawn or recorded using either the S Pen’s recognition and subsequent conversion to text, or the graphical format.

But why, in 20 years of mobile, is this groundbreaking handwriting recognition feature just now being introduced? The answer is, it isn’t.

The first attempts to teach computers to understand handwritten text began in the Soviet Union in the 1960s, when the emergence of personal computers and smartphones was, it seemed, in an uncertain distant future. At the time, the space industry was driving the development of handwriting recognition, explains the head of Paragon Software’s mobile development division, Alexander Zudin, as a way to avoid sending pencils and paper into space to cut down on costs. Each extra gram sent into space amounted to huge additional costs. Commercial use of handwriting recognition had not been identified at the time, but by the mid-1990s, with the appearance of the first handheld computers known as PDAs, keen interest was shown by various device manufacturers.

Some form of handwriting recognition was installed on all the early devices from Palm, Apple (which entered the market with one of the first prototypes of the modern smartphone, Newton) and PDA manufacturers working under the first version of Windows Mobile. Interestingly, handwriting applications did not become the “killer apps” programmers thought they would be, simply because their development didn’t meet consumer expectations. They just didn’t work well.

“‘Teaching’ PDAs every letter and number written by hand was too complicated,” says Zudin, “and the recognition accuracy was very low.”

The qualitative leap in the handwriting recognition market didn’t happen until about 20 years later, due to the massive proliferation of tablets and smartphones. Entering information by hand was a natural development in the new mobile world, with increased opportunities for handwriting recognition on the sensitive touch screens that could be found in nearly every purse, pocket or briefcase.

Paragon Software began development of its own PenReader handwriting recognition software in 1997, says Zudin, for early PDA devices. Over the next few years, specialized operations, such as orthographical correction of recognition results and support for more than 30 world languages, were added to the PenReader engine, increasing the complexity, exposure and reliability of its handwriting identification. A full consumer version of PenReader with cursive text recognition was released in 2007. Unlike other input methods, PenReader adjusts to the user’s natural handwriting style rather than requiring any handwriting modifications or use of complex symbols like those used in early handwriting recognition software.

With increased precision in the technology, handwriting recognition has gained traction. The most popular devices and platforms all have handwriting recognition services available, especially as mobile applications. In the App Store, a number of programs can be found that allow graphical information to be drawn on the screen and saved as pictures — and some programs even have the ability to recognize handwriting, including the popular note-taking app Evernote, which not only supports handwritten text notes, but also audio, speech-to-text and the ability to scan documents with cloud archiving capabilities. The iPad tablet, because of its size and shape, naturally lends itself to being scribbled on by hand, but a special stylus for this task costs about $80.

Handwriting apps aren’t just for iOS. In Google Play there are a number of applications based on handwriting recognition technology. Handwriting Dato and Handwrite Note Free, for example, are designed to store and catalog handwritten notes. More “advanced” applications, such as MyScript Calculator, are designed for writing complex mathematical calculations by hand.

A new handwriting recognition feature is also expected to be supported in the latest Microsoft Windows 8. The handwriting recognition function was already available to Windows 7 users, but only enabled by expensive electronic pens and limited to just a few Windows 7 tablets. Microsoft has made a significant step forward with Windows 8, according to one of the beta testers of the system, making writing on the screen almost as easy as writing with a pen on paper.

It is not only gadget manufacturers and their software developers that have shown interest in handwriting recognition, but also other industry players. The German automotive manufacturer Audi equipped their 2011 A8 and A6 models with on-board computers that support handwriting functions capable of entering information into the Multi Media Interface, which includes the car’s media player and navigation system. Audi’s rationale was that some users find it easier to manually enter information rather than pressing buttons.

Paragon Software has seen increased awareness from manufacturers looking to pre-integrate handwriting recognition into personal computing devices and electronics of all types. PenReader, utilized in place of the default keyboard in any application, can be easily configured to integrate completely with any operating system, which is why the company recently released the PenReader API as part of a software development kit aimed specifically at device manufacturers. Zudin says that success of its consumer version has drawn interest from a spectrum of electronics makers for a range of end uses that the company never envisioned when it began tinkering with handwriting recognition 15 years ago.

Google, also recognizing the attraction of alternate input methods, launched its Google Handwrite feature in July to facilitate handwritten Google web searches on touchscreen smartphones and tablets. Available in 27 languages for iOS5 and higher, and Android smartphones running 2.3 and higher (4.0 and higher on tablets), Google Handwrite looks to be a natural enemy of anti-keyboard pecking input apps like Swype, Swiftkey or PenReader, and even the company’s own Google Voice Search speech recognition input method.

Not so fast. Literally.

Writing English characters by hand is still viewed as more cumbersome than typing, many believe. The path of adaptation seems to be to satisfy the ability to input information quickly — whether by typing with your fingers or through voice recognition. There has been significant progress in these areas, with pre-integrated alternate input methods in high demand. But, for whatever reason, handwriting recognition still remains a niche. ■

About Paragon Software Group

Paragon Software Group’s lineup of effective, quality reference applications enrich mobile learning for users of 30 global languages with more than 350 electronic dictionaries, encyclopedias and phrase books developed in conjunction with the world’s leading publishing houses, such as Duden, Berlitz, Langenscheidt, Merriam-Webster, Oxford, PONS, Le Robert, VOX, and others. For up-to-the-minute information on Paragon Software Group happenings, including news and information on new products, please connect with the mobile division on Facebook or Twitter @Paragon_Mobile.

About the author

Katia Shabanova is director of public relations at Paragon Software Group where she handles editorial exposure for the company’s mobility and disk utilities divisions. Ms. Shabanova studied linguistics at Moscow State Linguistic University and University of Texas at Austin; English and German philology at Santa Clara University, California; and earned Master of Arts degrees in English and German Philology at Georg-August University of Göttingen, Germany. You may contact her at kshabanova@paragon-software.com or connect with her on LinkedIn http://tinyurl.com/8nxzeou