Apple's new “Visual Intelligence” feature was one of the most impressive things shown off at the iPhone 16 event on Monday. The tool lets users scan the world around them through the iPhone's camera to identify a dog breed, copy event details from a poster, or look up virtually anything around them.
It's a handy-looking feature that fits perfectly with the iPhone's new camera button. But it could also be the foundation for bigger products in the future: It's exactly the kind of feature Apple will need for future technologies like AR glasses.
It's not hard to imagine how visual intelligence could help you on a device that sees everything you see. Take, for example, the idea of learning more about a restaurant, as Apple demonstrated with visual intelligence on an iPhone: instead of taking your phone out of your pocket to look up information about a new place, with glasses you could simply look at the restaurant, ask a question, and let the glasses tell you more.
Meta has already proven that computer glasses can be good
Meta has already proven that computer glasses with an AI assistant can be a good and useful tool for identifying things. It's not far-fetched to imagine Apple doing something similar with a very high level of fit and finish for theoretical glasses. Apple would almost certainly also make glasses connect to all your apps and personal context on your iPhone, which would make visual intelligence even more practical.
Of course, Apple already has a headset with cameras: the Vision Pro. But most people don't walk around with their headset outside their home and probably already know what they have at home. It has long been reported that Apple wants to develop true AR glasses, and that seems to be the ultimate goal for this type of technology.
The thing is, AR glasses made by Apple could still be a long way off. BloombergApple's Mark Gurman reported in June that a 2027 launch date was “under discussion” for the glasses in development, but noted that “nobody at Apple that I've spoken to believes the glasses will be ready in a few years.”
But whenever those glasses come to market, they'll need software—and here's how Apple is building the groundwork for it. Visual Intelligence could be Apple's first step toward a killer app for computer glasses, and if Apple starts now, the company may have years to refine the feature before it appears in glasses.
It wouldn't be unprecedented for Apple to take this approach. The company worked on AR technologies in the iPhone for years before launching the Vision Pro. Yes, the Vision Pro is arguably more of a VR headset than an AR device, but it's clearly a first step toward something that could evolve into AR glasses. While Apple improves that hardware, it can also work on software features like Visual Intelligence on the iPhone, and in due course pack all of the best ideas into a glasses-like product..