Miniature infrared sensor technology to drive new applications
27 June 2016
The now widespread use of accelerometers and gyroscopes has demonstrated how well the introduction of sensors can enhance applications and even drive new modes of engagement with computer systems.
(Click here to view article in digi-issue)
They have enabled new types of product to emerge, as Nintendo demonstrated with the Wii gaming platform a decade ago.
A key feature in the rapid adoption of accelerometers and gyroscopes in mobile systems, among others, was their use of semiconductor integration. The same techniques can now enable the use of another type of sensor that has so far been restricted to use in specialised, high-end equipment. Providing the ability to see heat, infrared sensors are now available in a form that supports a much wider range of applications than has previously been possible.
Historically, longwave infrared (LWIR) sensors have been comparatively bulky and power-hungry devices, but semiconductor integration techniques developed for visible-light sensor arrays can be leveraged to create much smaller and more cost-effective sensors.
The wavelength of LWIR places restrictions on how small a sensor can become. CMOS visible-light imagers generally have pixel sensors that measure approximately 1µm across; suitable for the photons they need to record, which lie in the 0.4 to 0.8µm wavelength range. LWIR photons have wavelengths more than an order of magnitude longer, in the 8 to 14µm range. However, this still allows for sensors that have thousands of individual pixel sensors that can fit into the space of a mobile phone’s CMOS imager.
A low-cost, low-space IR sensor makes it possible to create new, easily accessible devices that use heat either as their main input or perform sensor fusion that augments the inputs from other imagers and transducers. Often the algorithms needed to make sense of noisy image data can be greatly helped by adding thermal imagery as an input.
In surveillance and safety applications, it can be important to know how many people are on a train platform, congregating in a space, or joining the queues at a supermarket or airport. Based on the count, a supermarket might open additional checkouts and at a station, staff may intervene to direct people to less-crowded areas.
Although image-processing algorithms exist to perform people counting, they can easily be confused by changes in ambient-light conditions. Thermal imaging makes it much easier to distinguish individuals by either replacing image processing or augmenting it. Unlike visible-light data, the thermal information is unaffected by the changes in lighting that can disrupt image-processing algorithms.
Traditionally, thermal-imaging people-counting systems have relied on comparatively low-resolution sensors, with a pixel area based on thermopiles of just 8 x 8 or 16 x 16. With these systems, people show up as undifferentiated blobs, making it harder for the algorithm to tell them apart if they are standing close together. Increasing the thermal-sensor resolution to more than 50 x 50 pixels provides a much better ability to tell people apart using the thermal data alone. It also allows a larger area to be covered by one imager which reduces overall system cost.
Thermal imaging provides an effective diagnostic tool for a variety of situations. In veterinary healthcare, unusually hot locations can indicate an injury. Higher temperatures often accompany inflammation. Alternatively, reduced heat compared to normal can indicate areas where blood flow is restricted and the presence of cancerous growths. Being able to detect these problems non-invasively is important. Animals cannot often communicate these problems. Not only that, they will often try to mask the discomfort because of their own survival instincts.
Zoos and larger veterinary practices have, for a number of years, used traditional, expensive thermal imagers to detect problems in valuable livestock and cherished pets. The ability to build low-cost systems using a high-integration sensor brings the ability to employ heat for diagnosing injuries to a much wider range of users. For example, stable owners could buy a standalone imager to find tendon, hoof and saddle-related injuries or even fit an LWIR add-on to a mobile phone to scan the suspect area instead of calling out the vet simply to see if there is a problem.
The thermal imager can be combined with a visible-light imager to help improve usability. The visible-light image provides an easy way for the user to see precisely which part of the body has the injury by overlaying one picture over the other. This is an approach that can also be applied to engineering diagnostics. For example, in electronics, thermal imaging can show components that are producing higher heat levels than expected. Components running hot can signal an imminent failure. The overlay of a visible-light image helps the engineer determine precisely which component is affected.
Similarly, the combination of images can show the effects of pipe damage. For example, leaking water behind a wall can show up as a cold wall when a thermal image is taken. The visible-light image will show the position of the cold zone relative to objects that the user can see on or around the wall itself.
In some cases, thermal imaging provides one of the only means of determining visually what is happening in a room. Firefighters today still rely on feel to navigate around a smoke-filled room. Although fire stations may own a thermal camera, their budget likely only allows the use of one per team at most. These are often comparatively bulky handheld devices. They have power requirements of approximately 1W, requiring a large battery pack to provide sufficient capacity. As a result, one firefighter needs to be dedicated to navigation while others focus on firefighting or rescue.
A small, low-power thermal imager would make it possible to integrate thermal imaging in a headset that can be integrated with the breathing apparatus, removing the need to carry the device by hand. By reducing power consumption to the order of 200mW, the battery can be made much smaller, improving overall portability.
Another application where both size and power are vital is in the development of unmanned aerial vehicles (UAVs). These devices are now being employed by the rescue and security services to improve situational awareness as well as for environmental and agricultural surveys. Thermal imagers help improve the range of situations they can monitor and improve the ability of operators to land a UAV under low-visibility conditions.
As with accelerometers and gyroscopes, LWIR sensing in a compact form provides the ability to create better user interfaces for a wide range of electronics systems, including ebook readers and gaming systems, through the use of gestural control. Many of the optical systems used for gesture recognition today are, as with the people-counting application, affected by ambient-light conditions.
A resolution in excess of 50 x 50 pixels makes it possible to distinguish limbs from the body and track them as they move, using just the heat information or combining that input data with other cameras and detectors for increased accuracy. At closer range on an ebook reader, for example, a LWIR sensor can watch hand and finger gestures close to the screen, letting the user turn a page by waving their hand over the display without having to touch it.
Suitable CMOS imagers sensitive to LWIR are now appearing on the market. With a pixel that measures 17µm across, FLIR Systems’ Lepton makes it possible to put an 80 x 60 pixel sensor onto an IC that fits into the same 32-pin Molex connector as that used for many of today’s tiny mobile-phone cameras (Figure 2). Power consumption of just 200mW provides the ability to operate the sensor for long periods without significant impact on a battery.
To enable the Lepton, FLIR further leveraged other mobile-phone imager technologies, such as very low-cost lenses where many are built simultaneously using a high-volume wafer-level process. Similarly, wafer-level packaging of the sensors not only reduces camera size but the number of manufacturing steps needed, all of which are highly automated. Further, the use of single-chip camera electronics, incorporating a digital signal processor (DSP), reduces size and the number of interconnects and dispenses with the need for the integrator to use a field-programmable gate array (FPGA) to perform those functions (Figure 3). All that is needed to carry the thermal image to the applications processor is a simple SPI serial interface.
The availability of different field-of-view options – 25° or 50° – provides the ability to tune the imager for different applications. The wider field of view allows the surveying of a complete scene, while the 25° option is highly suitable for longer-range use.
Thanks to a combination of higher resolution and lower cost made possible by high-integration IC and wafer-manufacturing techniques, LWIR sensing can now support a growing range of applications. Those identified here are just the tip of the iceberg.
Contact Details and Archive...