Later this year, Apple hopes to offer system-wide real-time audio transcription and summarization across many of its devices, as the iPhone maker looks to harness the power of AI in increasing efficiency for some of its core applications.
People familiar with the matter have told us that Apple is working on an AI-powered summing feature and significantly enhanced audio transcription capabilities for some of its next-generation operating systems. . The new features are expected to bring significant efficiency improvements to users of Notes, Voice Memos, and other apps.
Apple is currently testing the possibilities of adding features to several app updates expected to launch with the release of iOS 18 in late 2024. They are also expected to move forward with apps respectively in macOS 15 and iPadOS 18.
The default Voice Memos app that Apple brings to its device portfolio will be one of the first to receive upgraded features. Early versions of the app provided a running transcript of each recording, working similarly to the company’s recent Live Voicemail feature.
The transcription takes up the central area of the app window, replacing the larger graphical representation of the recorded audio found in the existing version of the app.
Transcription is also being included in the next version of the Notes. Pre-release versions of both apps feature a dedicated transcription button in the form of speech bubbles, according to people familiar with the software. Tapping the new speech bubble will display the audio transcript recorded in the app.
The transcription tool will go hand in hand with – and provide new context for – the upcoming recording features in Notes, which AppleInsider first detailed in April. Specifically, the update will add AI-generated recorded audio summary option to instantly provide a basic text summary of key highlights and action items.
The AI summarizing feature, combined with new real-time transcription and in-app recording options, is expected to make Apple’s built-in Notes app a real powerhouse. This trio of features will benefit many real-world applications, taking on the heavy lifting of processing large amounts of data to key targets. All this brings convenience and clarity to users at a glance.
Students can easily record lectures and classes without relying on third-party tools. If recording from the new Notes app, there will be the option to include transcripts and summaries in notes, along with other media like images, links, and data structures like tables.
These features will also benefit professionals who regularly participate in conference calls, virtual business meetings, or seminars as part of their work. Such events often reveal large amounts of information, various statistics, detailed business plans, dates and schedules that Apple’s AI technology analyzes and organizes into summaries. has a reasonable structure.
The same applies to more advanced classes or lectures that often include many types of information, such as definitions, explanations of complex ideas or theoretical principles, illustrative examples , etc.
Meanwhile, journalists will get an extremely effective way to record and summarize long interviews. Creative people like authors and screenwriters can easily jot down key ideas and look over them later without having to play back and listen to large portions of the recording just to isolate key data points.
Although Apple has made every effort to ensure that its transcription and summarization features produce accurate results, errors are inevitable. Therefore, maintaining the original audio along with AI-generated transcripts and summaries ensures that no source information is lost during transcription or summarization.
Summary is just one part of Apple’s larger AI effort
The new transcription and summarization features will be part of Apple’s broader AI push this year. Similar briefing features are also expected to come to Safari 18 through Smart Browsing and the built-in Messages app – through integration with Apple’s on-device AI software.
The use cases and overall purposes of AI-powered summary features in Safari and Messages are completely different. While Notes will give users the option to summarize meetings, conference calls, and lectures, Safari will allow web summaries, while Messages will provide condensed versions of message content. message.
Apple’s AI software could also be used to protect user privacy, as certain AI features are expected to operate entirely on the device. However, in the case of audio transcription and advanced AI summarization, server-side processing may be required for the time being.
By incorporating audio summarization and transcription into its system applications, the company aims to demonstrate some of the best-in-class use case advantages of deploying AI to solve industrial situations. the real world. The goal of Apple’s AI efforts is to deliver developer features that promise to help its customers be more productive and successful in their everyday tasks.
At the same time, the company is hoping to be better positioned against the rise of competing third-party apps that are now using AI technology, some of which already have good adoption rates. as consumers incorporate them into their digital lives.
For example, the Otter app is another recipient of Apple’s Editor’s Choice Award. It provides functionality similar to the features discussed in this article. With it, users can record, transcribe, and summarize meetings through synthetic AI, all in one app.
Microsoft’s OneNote also offers recording support in the form of voice notes, serving as another potential competitor to Apple’s Notes and Voice Memos apps.
However, it should be emphasized that not all software features that Apple tests in pre-release software builds are included in the existing release cycle. Apple is known to cancel projects or delay features to subsequent operating system and app releases at the last minute, so ultimately there are no guarantees regarding timing and availability. use.
That said, the new real-time transcription and AI summary features still appear to be on track for their expected launch alongside Apple’s next-generation operating systems at its Developers Conference. (WWDC) company in June. They are expected to be joined by improved Calendar and Calculator apps, among others.
