Apple has announced a new feature on the iPhone, iPad and Mac called Live Text, which automatically digitizes text in photos. Integrating the feature into Spotlight Search on iOS will allow you to search for images based on recognized text.
Text recognition in photos was previously introduced by Samsung and Google, but Apple’s implementation is deeply integrated into the operating system, allowing you to interact with text in images: copy and paste it, search the Internet, and, if it’s a phone number, call that number. You can use Live Text to turn handwritten notes into emails and search for receipts or recipes. Live Text is powered by neural networks running on the device itself. This approach, according to Apple, allows you to significantly increase the level of privacy, since the data is not sent to the cloud, but remains on the device. Text recognition will be available in seven languages: English, Chinese (simplified and Traditional), French, Italian, German, Spanish and Portuguese.
In addition to extracting text from photos, iOS 15 will also allow users to perform visual searches using the Visual Look Up function, which is similar to Google Lens and recognizes pets based on breeds, types of flowers, works of art, books, geographical objects and attractions. iOS 15 will be available for installation in the fall of 2021.