From Depth Scanner to Around-the-Corner Vision
The LiDAR sensor on recent iPhones and iPad Pro models was marketed mainly for faster focusing and better augmented reality capability. Researchers at the MIT Media Lab have now revealed a far more dramatic trick: using the same consumer-grade iPhone LiDAR sensor to see around corners. Their work shows that the device can detect and track objects completely outside the camera’s field of view, a technique known as non-line-of-sight imaging. Previously, this kind of object detection needed bulky, expensive lab-grade lasers, limiting it to research environments. By contrast, the MIT team achieves similar concepts with the low-power sensor already sitting in many pockets. This doesn’t produce sharp photos of hidden scenes, but it does let the phone infer that something is there, how it is moving, and its rough shape, effectively turning ordinary hardware into a surprisingly powerful perception tool.
How an iPhone LiDAR Sensor Can See What You Can’t
The magic lies less in the sensor and more in how it is used. The MIT team relies on motion and what they call an aperture sampling model. As you move your iPhone, its LiDAR sends out pulses of light that bounce off visible surfaces and hidden objects around a corner. The system collects many noisy, imperfect reflections as the camera changes position, then stitches them together over time. This process tracks three things at once: the hidden object’s shape, the object’s position, and the camera’s own position. The result is a gradually improving estimate of where objects are and how they move, even when the camera never directly sees them. Think of it like echolocation with light instead of sound, transforming simple depth scans into a dynamic, around-the-corner sensing system.
What This Non-Line-of-Sight Trick Can Actually Do
In demonstrations, the system went beyond just detecting a single hidden object. First, it could track one object moving out of sight and reconstruct its basic shape. Then it handled multiple objects at once, distinguishing between them as they moved. Perhaps most impressively, it used hidden landmarks for camera self-localization: the phone could orient itself based on objects it could not directly see. That capability is especially promising for robots, delivery drones, and future self-driving systems, which could navigate more safely by anticipating obstacles around blind corners. While the output looks more like a ghostly outline than a high-resolution image, it gives machines critical awareness of their surroundings. In everyday terms, your phone’s LiDAR sensor becomes less of a simple depth tool and more of an early warning system for what lies just out of view.
Why This Matters for Apps, AR, and Navigation
Because this technique works with the same consumer-grade iPhone LiDAR sensor already in current devices, its potential applications are broad. Navigation aids could warn you about a person or obstacle approaching from around a corner, improving accessibility and safety. Augmented reality capability could become more immersive and responsive, with apps reacting to people or objects before they enter the camera frame. Games might use hidden movement for richer, more dynamic environments. For robotics developers, the ability to localize a camera using hidden landmarks opens new paths for mapping and exploration. What makes this especially intriguing is that it repurposes hardware Apple already ships but has not marketed for non-line-of-sight imaging. The researchers even note that the underlying sensor setup can be replicated for under USD 50 (approx. RM235), and their code is publicly available, lowering the barrier for experimentation.
When Can You Try Seeing Around Corners Yourself?
Despite relying on everyday hardware, this around-the-corner vision is not yet a tap-to-enable feature on your iPhone. The main obstacle is data access: current mobile platforms usually do not expose the raw LiDAR measurements that the MIT algorithm needs. According to the researchers, enabling this on commercial phones would require companies to open up deeper access to sensor data. For now, the team has shared its code publicly so others can study and build on the approach, and they highlight that the custom sensor rig they used can be assembled for under USD 50 (approx. RM235). That means hobbyists and researchers can start exploring similar non-line-of-sight imaging techniques today. If phone makers eventually expose the necessary data, future apps could turn what is currently a research demo into an everyday feature hiding in your pocket.