Apple Intelligence Expands With Live Translation, Smarter AI, and Developer Access

Apple has announced an expansion of its Apple Intelligence features, bringing new capabilities to iPhone, iPad, Mac, Apple Watch, and Vision Pro. The update, revealed at the company's Worldwide Developers Conference, introduces live translation, more powerful visual intelligence, and—in a major move—opens up Apple's on-device foundation model to third-party developers.



A key new feature is Live Translation, which will be integrated directly into Messages, FaceTime, and the Phone app. Apple says this will allow users to communicate across languages in real-time, with all processing handled on-device for privacy. The feature will provide translated live captions during FaceTime calls and spoken translations during phone calls. This builds on earlier rumors that Apple was planning to bring live translation to its AirPods.




For developers, Apple is opening access to its on-device large language model through a new Foundation Models framework. This will allow developers to build AI-powered features into their apps that work offline and maintain user privacy, without incurring cloud API costs. Apple highlighted that the framework includes native support for Swift, making it easy to integrate.



Visual intelligence is also getting an upgrade, now allowing users to search for or take action on anything visible on their screen. The feature can recognize an event in a photo or webpage and suggest adding it to the Calendar, or let users ask ChatGPT questions about what they're viewing. Genmoji and Image Playground are also being enhanced, with the ability to mix emoji and text descriptions to create new images.




A new fitness experience called Workout Buddy is coming to the Apple Watch. It uses Apple Intelligence to analyze a user's workout data and fitness history to provide personalized, spoken motivation during a session. This aligns with earlier reports that Apple was planning to overhaul its Health app with AI-driven coaching features.



Other updates include more intelligent Shortcuts that can tap directly into Apple's AI models, AI-categorized Reminders, and order tracking summaries in Apple Wallet. These new features will be available for testing by developers starting today, with a public beta rolling out next month. The final release is scheduled for this fall on supported devices, which include all iPhone 16 models, the iPhone 15 Pro, iPad mini (A17 Pro), and iPad and Mac models with an M1 chip or later.



Please download the iClarified app or follow iClarified on Twitter, Facebook, YouTube, and RSS for more WWDC updates.