Real-time recognition captures mobile screens on the fly

ABBYY is enabling ‘instant’ text extraction from the preview screen of mobile devices via a newly introduced Real-Time Recognition SDK (RTR SDK). Realtime recognition enables quicker, simpler extraction of text from documents and objects placed in the live video stream from smartphone and mobile cameras.

The extracted data can be used in a variety of mobile applications ranging from highly responsive customer services to automated enterprise processes and consumer apps.

Applications based on ABBYY Real-Time Recognition SDK can pull text from on-screen objects and automatically convert it to digital data. On-screen images are processed in live video stream, eliminating the need to first take a photo and save it. This makes the data entry process simpler and more intuitive. The conversion takes place directly on the device and within the mobile app, no Internet connection is needed. Once digital, data can be easily processed by the application or even sent to cloud-based systems for further handling.

Enterprises can develop real-time recognition-enabled customer services and products to speed up data acquisition and processing times. As part of customer onboarding processes such as account opening or loan application, information from ID cards, pay slips and other documents can be captured within seconds and then transferred directly into online forms or apps for new customer sign-ups.

For customer self-services provided by banks or insurance companies, mobile apps can quickly extract and use transactional information such as bank transfer details (BPay, SWIFT, IBAN, etc.) or policy numbers in claim forms to make data entry processes more efficient. As no photos are stored on the device, the technology is well-suited for use in processes that must comply with data security and data privacy standards.

Software vendors developing consumer apps can easily add new functionality and value for their users. Travel apps can offer fast translation assistance when images captured on preview screens can be quickly translated using mobile dictionaries.

Text reading apps for persons with reading and learning disabilities can be enhanced by ‘live text input’ into their text-to-speech components.

“Real time recognition can help transform data into outstanding customer experiences,” explains Henry Patishman, ABBYY Director of Sales Australia and New Zealand.

“Simply point your device’s camera at the text and ‘lift out’ what you can see live on the display. This provides an extremely intuitive and accurate way to capture information printed on virtually any background. We see great promise with this new ABBYY technology and strongly believe that it can be a real value-add to a vast range of mobile apps”.

The new Real-Time Recognition SDK complements ABBYY’s portfolio of tools for capture, digitisation, and extraction of data. ABBYY’s offering ranges from mobile-based information capture and OCR, to enterprise-level automated document processing and data extraction. 

Specifications and Availability

ABBYY Real-Time Recognition SDK is available for integration into new or existing applications for iOS and Android. The ABBYY developer toolkit supports easy integration and offers code samples and ‘quick start’ guides to help developers get started. The SDK supports capture of texts in 63 languages.

A video illustration of the solution is available at

For more information on the ABBYY Real-Time Recognition SDK or to request a trial for developers, please visit the ABBYY website (