iPhone X’s TrueDepth Module Dissected

　　Although experts in the imaging industry are aware of a complex “TrueDepth” module that Apple has devised for its iPhone X, most other details inside the device’s 3D system — chips, components, and all the way down to substrates — remain a deep, dark secret.

　　EE Times talked to Yole Développement, which completed this week a teardown of Apple iPhone X TrueDepth module in collaboration with its partner, System Plus Consulting. They deduced that silicon-on-insulator (SOI) wafers are being used in near-infrared (NIR) imaging sensors. They noted that SOI has played a key role in improving the sensitivity of NIR sensors — developed by STMicroelectronics — to meet Apple’s stringent demands.

　　Pierre Cambou, activity leader for imaging and sensors at Yole Développement, called the SOI-based NIR image sensors “a very interesting milestone for SOI.”

　　Many companies located in France’s so-called Imaging Valley, near Grenoble, have used SOI wafers, developed by Soitec — initially for backside illumination (BSI) sensors. Meanwhile, research on SOI for NIR sensors dates back to 2005, according to Cambou.

　　But Apple’s adoption of ST’s NIR sensors marks the debut of SOI in mass production for image sensors, noted Cambou. “Image sensors are characterized by large surface due to the physical size of light. Therefore, this is a great market to be in for a substrate supplier” like Soitec, he added.

　　Meanwhile, Jean-Christophe Eloy, Yole's president and CEO, told EE Times that, in designing its TrueDepth module, “Apple took the best of both worlds — STMicroelectronics and Ams.” Apple adopted leading-edge NIR imagers from STMicroelectronics, while it deployed dot illuminators from Ams (Premstaetten, Austria). Eloy noted that Ams is “extremely good at its complex optical module.” Earlier this year, Ams acquired Heptagon, known for its Time-of-Flight (ToF) technology stack.

　　Recap on how it works

　　Apple put a 3D camera on the front of the iPhone X to identify its owner’s face and unlock the phone.

　　As Yole previously explained, to make this possible, Apple combined a ToF proximity detector with an infrared “structured light” camera that can either use a uniform “flood” or “dot-pattern” illumination.

　　The way that the 3D system works is very different from a regular CMOS imager taking a photo. First, the iPhone X combines an infrared camera with a flood illuminator that projects uniform infrared light in front of the phone. It then takes images, which, in turn, trigger a face-detection algorithm.

　　This face-recognition function, however, isn’t meant to run all the time. The infrared camera linked to the ToF proximity sensor signals the camera to take a picture when it detects a face. The iPhone X then activates its dot pattern projector to take an image.

　　Both the regular and dot-pattern images are then sent to the application processing unit (APU), which puts them through a neural network trained to recognize the owner and unlock the phone.

　　Yole’s Cambou noted that no 3D image is computed at this point. The 3D information is contained in the dot-pattern image. “To run 3D applications, the same APU can use another algorithm [that] computes the depth map of the image.” He added, “The iPhone X takes advantage of the massive processing power available in the A11 chip, as structured light approaches are known to be computationally intensive. The use of a neural network is the key technology that made it possible.”

　　Five sub-modules

　　The teardown by Yole and System Plus Consulting has found a “complex assembly of five sub-modules” in Apple’s optical hub. They are: near-infrared camera, proximity detector (Time-of-Flight) + IR flood illuminator, RGB camera, dot-pattern illuminator, and color/ambient light sensor.

　　As shown below, the IR camera, RGB camera, and dot projector are all aligned.

　　NIR image sensors

　　At the heart of Apple’s iPhone X’s optical hub, there is STMicroelectronics’ NIR sensor. Yole and System Plus Consulting found inside ST’s NIR sensor “the use of silicon-on-insulator (SOI) on top of deep-trench isolation (DTI).”

　　The idea of DTI technology is well-known. In general, the issue with the high sensor resolutions required in today’s cameras is that pixels are forced inside the same space, creating noise, discoloration, or pixelization of neighboring sensors when capturing a photo. DTI is deployed to prevent leakage between photodiodes. Apple reportedly etched literal trenches between each one, then filled the trenches with insulating material that stops electric current.

　　So, on top of DTI, why did Apple want to use SOI wafers for NIR image sensors?

　　Optically speaking, Cambou explained that SOI wafers are advantageous because the insulator layer functions like a mirror. “Infrared light penetrates deeper, and it reflects back to the active layer,” he noted.

　　Electrically speaking, Cambou noted, SOI improves NIR’s sensitivity largely because it’s good at minimizing leakage within the pixel. The improved sensitivity provides good image contrast.

　　Contrast is important because “the structured light operation is disturbed by sunlight,” explained Cambou.

　　Of course, regular CMOS image sensors or NIR sensors are “happy to have extra light if the goal is to have a better image,” said Cambou. However, light is a problem when a user tries to unlock an iPhone X under the bright sun.

　　“The problem is the contrast of the projected dots of NIR light versus ambient light from the sun or any other light source,” said Cambou. “But the sun is usually the biggest problem.” Hence, it was paramount for Apple to improve NIR’s contrast by using SOI wafers.

　　Asked if ST’s NIR sensors are using FD-SOI or SOI wafers, Cambou said that the research firms couldn’t tell.

　　As for NIR sensors, do we know if Apple is using 850-nm or 940-nm wavelength NIR? Cambou noted, “We couldn’t determine which.” However, he speculated, “Apple most likely used 850 nm like everyone else (i.e., Intel’s RealSense, Facebook, HTC, and others), but STMicroelectronics is known for developing 940-nm SPAD proximity ranger, so it is a possibility that they intend to move to this wavelength in the future.”

　　Asked about surprises unearthed by the teardown, Cambou cited the size of ST’s NIR sensor chip. It measures 25mm2, and has only 1.4 megapixels due to the large 2.8-μm pixel size. Cambou noted, “Nevertheless, in this category, this pixel is considered as ‘a small one’ compared to competitions [that] typically use 3.0 μm to 5 μm.”

　　Beginning of the new era

　　Yole positions the iPhone X as the beginning of a new era for 3D imaging.

　　Cambou also believes that Apple is forging the future for NIR sensors. Pointing out last week’s announced acquisition of InVisage Technologies, he noted, “In my point of view, Apple wanted InVisage for NIR sensor capabilities, although there may be several ways to interpret this acquisition.”

　　Cambou doesn’t believe that InVisage could match STMicroelectronics’ product in terms of performance, but it could provide a solution for miniaturization, he noted. “Face ID technology could, therefore, be scaled down for other products such as AR headsets.”

　　Business implications

　　On one hand, Apple’s iPhone X is creating big opportunities for SOI wafer makers like Soitec. Equally significant is that it has already triggered a meaningful comeback for STMicroelectronics. Cambou is confident that ST will be a player in the emerging ToF camera market.

　　Of course, the semiconductor business is often affected by a short cycle of boom-and-bust. But ST, whose business was decimated after it lost Nokia in the mobile phone market, has “made a very elegant transition,” observed Cambou.

　　ST has created different types of image sensor applications — moving from CMOS image sensors to NIR and SPAD sensors in the future — all the while leveraging its assets and home-grown underlying technologies.