Depth Sensors’ Impact Goes Deeper than iPhone X

　　The FaceID system in the iPhone X has demonstrated how depth sensing can enable facial detection, recognition, and authentication, but potential applications for depth sensors extend beyond those use cases as well as the iOS platform. Qualcomm, for one, has taken its Spectra image signal processor (ISP) technology to the next level with a 3-D depth-sensing camera module for Android developed in collaboration with Apple supplier Himax Technologies.

　　Next year could see the emergence of a depth-sensor ecosystem, including firmware and apps, as more smartphone and wearable-device vendors incorporate third-party modules in their designs.

　　Qualcomm combined the Spectra imaging technology with Himax’s expertise in wafer optics, sensing, drivers, and module integration to create the SLiM depth sensor for mobile, augmented-reality (AR), virtual-reality (VR), automotive, and surveillance applications. Himax CEO Jordan Wu said his company had been working with Qualcomm for more than four years to develop the 3-D sensing solution.

　　Qualcomm’s active depth-sensing module (right) features a depth map with more than 10,000 points of depth and can detect changes in depth as small as 0.1 mm apart. Source: Qualcomm

　　The turnkey camera module delivers real-time depth sensing and 3-D point-cloud generation for both indoor and outdoor environments. The computer vision camera module is expected to appear in a number of products in the first quarter of 2018.

　　Qualcomm’s integration of low-power Spectra ISPs into its popular mobile processors bolsters the capability of mobile devices to embrace the emerging use cases.

　　A depth sensor uses a time-of-flight (TOF) technique to resolve the distance from an object based on the known speed of light. Infrared dots are projected onto an object as a point cloud, and the sensor then reads distortions in the field and gathers depth information.

　　Depth-sensor-based approaches have gradually moved toward mobile power requirements for handsets and head-mounted displays. “The technical problem for room-scale 3-D sensing and beyond is power and performance,” said Simon Solotko, senior analyst at Tirias Research. That means managing the power for the sensor and the image signal processor, along with the complex software required to translate point clouds into useful, interactive input.

　　The target power for mobile is always as low as possible. As Solotko pointed out, however, active laser illumination in time-of-flight and structured-light solutions requires high power if you want to move beyond facial- and gesture-recognition applications with up to a 2-meter range to serve room-scale and longer-distance applications with a sensing range from 2 to 10 meters.

　　Today’s sensor packages are providing high-quality point clouds in the sub-half-watt range for short distances and in the 5-W range for longer distances. Short-range depth sensors have thus fulfilled the requirements for mainstream smartphones, and as the iPhone X demonstrates, targeted and optimized applications serve everyday use cases.

　　“But long-range power requirements are too high and have forced designers to adopt purely camera-based approaches” for longer-distance applications, Solotko said.

　　Over the past several months, monoscopic multiview technology for consumer AR — in the form of the ARKit and ARCore developer platforms for iOS and Android, respectively — has created excitement among indie developers as well as major players like Amazon. “AR experiences are appealing and useful, and are already expanding the vocabulary of experiences that can be delivered by mobile developers,” said Solotko.

　　Microsoft has been active in acquiring and developing intellectual property for AR/VR, and it was early to market with a mobile depth-sensor solution in the HoloLens mixed-reality smart glasses and a camera-based solution in Microsoft Windows Mixed Reality.

　　Intel provides the RealSense development platform for gesture-based interfaces and has amassed rich software stacks and design IP for depth-sensing technology. However, according to Solotko, it remains unclear where this technology will ultimately reside — in notebooks, integrated head-mounted displays, or Intel-powered mobile devices.