From Portraits to Peeking Around Corners
The iPhone LiDAR sensor is already known for powering better portraits and smoother augmented reality iPhone apps. Now, researchers at the MIT Media Lab have unlocked a far more surprising ability: using this tiny depth sensor to detect and track objects that are completely outside the camera’s field of view. This approach, known as non-line-of-sight imaging, has traditionally depended on bulky, high-powered lab lasers, making it impractical for everyday devices. MIT’s work shows that the consumer-grade LiDAR already in many smartphones and tablets can handle the job instead. In practical terms, it means your phone’s LiDAR is evolving from a tool that understands only what it can directly “see” to one that can infer what is happening just out of sight. It is a major step forward in how smartphone sensors are used and what they can reveal about the world around you.
How Corner Detection Technology Actually Works
This corner detection technology relies less on raw power and more on clever math and motion. As you move your device, the LiDAR sensor fires pulses of light that bounce off visible surfaces, scatter around a corner, hit hidden objects, and then return. Each individual reading is noisy and incomplete, but MIT’s system uses an aperture sampling model to stitch thousands of these weak signals together over time. Instead of producing a sharp image, the algorithm builds a progressively better estimate of what is hidden: detecting that something is there, inferring how it moves, and roughly reconstructing its shape. You can think of it as echolocation with light, where motion of the phone is part of the sensing process. This is a new way to understand smartphone sensors explained: it is not just about clearer pictures, but about extracting meaning from very indirect measurements.
Lab Breakthrough, Consumer-Grade Parts
What makes this breakthrough stand out is its use of accessible hardware. Instead of custom, ultra-expensive equipment, the MIT team showed that similar sensing performance can be achieved with sensor hardware assembled for under USD 50 (approx. RM230). That cost profile puts the approach squarely in reach of mainstream devices that already ship with a LiDAR module, like recent iPhone and iPad Pro models. While you cannot install this exact system on your phone yet—current platforms do not expose all the raw LiDAR data that the method needs—the researchers have released their code publicly. This means developers and hardware makers can start experimenting immediately. As future smartphones open up lower-level sensor access, the same ideas could be integrated into camera apps, navigation tools, and AR platforms, making around-the-corner awareness a standard feature rather than a lab demo.
What It Can See: Tracking, Shapes, and Self-Localization
The current system does not create photorealistic pictures of hidden scenes, but it accomplishes several powerful tasks. First, it can track a single moving object around a corner, estimating how it is shifting over time. Second, it can infer the rough shape of that object, giving more context than a simple motion alert. Third, it can track multiple hidden objects at once, which is crucial in busy environments such as hallways or intersections. Finally, and perhaps most intriguingly for robotics and future phones, it can use hidden objects as landmarks to localize the camera itself. In other words, a device can orient and position itself using things it cannot directly see. For smartphones, that opens the door to navigation systems and AR experiences that remain stable and aware, even when the most useful reference points are just out of sight.
Future Uses for iPhone Users: Safety, Navigation, and AR
Once platforms expose the right LiDAR data, app developers can tap into this research quickly thanks to the public code. For everyday iPhone users, that could lead to safety apps that warn you about a fast-approaching person, cyclist, or vehicle before it appears in front of you. Navigation tools might gain a new layer of awareness in tight indoor spaces, using corner detection technology to anticipate obstacles or crowds around bends. In augmented reality iPhone apps, virtual objects could react to hidden motion—imagine a game where enemies advance from behind walls based on real-world movement, or AR guides that stay aligned even as you turn into new corridors. This work pushes smartphone sensors beyond their current limitations, turning LiDAR from a depth add-on into a foundation for richer, more anticipatory experiences that understand not only what you see, but what lies just beyond your view.