Sensor fusion for precision location tracking of first responder & military personnel
02 July 2019
ADI sensor fusions approach to precision location tracking for first responder & military personnel
Precision location of first responders or military personnel deep within GPS-denied infrastructure has long been an elusive goal of the fire safety, emergency personnel & military communities. The objective is to pinpoint location of personnel to within a few metres, over the course of tens of minutes.
This article was originally featured in the July 2019 issue of EPDT magazine [read the digital issue]. Sign up to receive your own copy each month.
These coincidentally are similar goals as found in guidance systems on tactical missiles, for which solutions of choice today can cost $10,000 minimum, with prohibitive size, weight and power characteristics. Those same solutions were used in early proof-of-concept demonstrators for first responders, but cost and size were proven to be barriers to actual deployment.
First responder or military personnel location determination, therefore, remains one of the most complex location applications in existence today. There is no one ‘silver bullet’ sensor that can achieve the desired goals, and multiple technology nodes are necessary, each of which are at the leading edge of capability. Furthermore, the solution involves a large scale sensor fusion and system integration approach. Cost-effective, high performance MEMS (microelectromechanical systems) inertial sensors can now provide the seed for a potential solution.
Table 1. Full systems approach, matched to key goals
In this article, Bob Scannell, Business Development Manager for Analog Devices’ MEMS inertial sensor products, envisions a complete sensor-to-cloud sensor-fused system, including highly sophisticated algorithms.
The major approaches and enabling technologies are described in Table 1.
The major challenges presented to the system developer can be summarised into the following three broad categories: procedural, environmental and sensor fusion. The highly complex nature of first responder or military personnel missions, coupled with the challenges posed by the varied and extreme environment, must be comprehended without compromise in the course of designing a multisensor solution.
Figure 1. Key operational requirements defining the first responder design problem
Fire safety search and rescue missions follow a highly disciplined process, which at the same time must adapt to fully non-deterministic real life scenarios. A deployable precision location system must adapt to existing processes and equipment, to the greatest extent possible. One requirement resulting from this is to be operational without any fixed or ad-hoc infrastructure, because first responders are typically burdened with significant equipment (weight and cost) already. Any system development should be guided from the early stages by the goals of achieving miniaturised embedded equipment and per responder costs in the order of smartphones. It is useful to recognise here that existing smartphone location performance is highly inadequate, thus the challenge. Figure 1 outlines the most relevant primary and secondary operational requirements of the desired system.
While outdoor positioning has become ubiquitous with GPS coverage, a fully indoor or mixed (indoor/challenged outdoor) environment is far less supported. Some indoor positioning situations (such as a shopping mall) can be realised with installed infrastructure – however, these are neither precision, nor practical for the first responder goal. For the system designer of a tracking system, the following considerations drive design definition, component choices and risk mitigation approaches:
• RF propagation paths.
Table 2. Benefits & limitations of candidate sensors
• Temperature/shock effects on sensors.
• Potential for damaged/altered infrastructure.
The challenges previously noted in process and environment are the basis for the central design approach for this problem, sensor fusion. Relevant primary sensing modes are selected to provide uncompromised performance in critical operational modes, while at the same time, complementary sensors are matched to the key obstacles for each phase of the application, as illustrated in Table 2.
Figure 2. Industrial targeted MEMS devices are capable of low noise & stable operation, even under extreme motion dynamics
Because of MEMS ability to operate free of external infrastructure and to provide precision in a dynamic environment, it is expected to play a primary role in the overall solution – if capable of operation in extreme environments and if coupled with the appropriate secondary sensors.
Progress in MEMS
While consumer inertial MEMS devices have raced toward commoditisation (with limited focus on performance specifications) and military MEMS have remained prohibitively expensive, industrial and automotive MEMS have aimed toward an enabling level of both performance and cost.
Industrial and automotive sectors require accurate sensing in relatively complex and extreme environments, compared to the consumer sector, and suppliers to these sectors have incorporated architectural features that are specifically tuned to reject performance detractors, such as off-axis motion, vibration and shock events, and errors that are induced due to time and temperature. While such design features are often most easily accommodated via larger sensors or more costly processes, the economic pressures of both automotive and an increasingly important industrial market force a more critical approach to designing for both performance and cost-effectiveness. The result is a highly attractive performance/price positioning for MEMS components, which are specifically developed for industrial applications, as shown in Table 3, where the percentage of error relative to distance travelled is compared for three major classes of components. Industrial grade MEMS can provide nearly as good navigation capability as high-end military devices, while at a reasonable price delta to the commoditised consumer MEMS components.
Table 3 & Table 4
The reason for this advantage requires a deeper look at the critical specifications of a MEMS component relative to the targeted application. In the case of the first responder goal, one critical task of MEMS sensors is to discern the type of movement being experienced, and measure the steps and stride. As opposed to a pedestrian motion model, the first responder movement will be more random, dynamic and difficult to discern. Furthermore, because of the accuracy goals, the sensor must be able to reject false motion such as vibration, shock and side-to-side rock/sway of the foot or body. Rather than a simple accuracy analysis based around the noise of the sensor, which may be sufficient for a pedestrian model, the first responder model must also include key specifications such as linear-g rejection and cross axis sensitivity. Table 4 provides a side-by-side comparison of an industrial and low end MEMS device, looking at the RSS error combination of three notable specifications. It can be readily seen that noise is not the detrimental factor, but rather the linear-g and cross axis performance, which many low end devices do not even specify, are the overriding concerns.
Though only a short number of years ago, high performance inertial sensors were primarily only achieved from approaches such as fibre optics, industrial MEMS processes have now clearly proved they are up to the task, with a relative comparison of key navigational metrics noted in Table 5 below.
An industrial MEMS IMU example is the ADIS16488A, as illustrated in Figure 2, which incorporates ten degree of freedom high performance sensing, and has also been qualified for among the most demanding of applications, commercial avionics (as indicated in Table 6), demonstrating its readiness for the extreme application demands of first responders.
Table 5 & Table 6
Advances in inertial MEMS performance, with continued proof of quality and ruggedness, are now being combined with significant strides in integration. This last hurdle is particularly challenging, as sensor size can be inversely proportional to both performance and ruggedness, if not carefully managed otherwise. A highly strategic, coordinated and challenging series of process advances must be proven and merged to enable the level of performance density required of this application, as illustrated in Figure 3.
The selection of appropriate sensors for a given application is followed by deep analysis to understand their weighting (relevance) during different phases of the overall mission. In the case of pedestrian dead reckoning, the solution is dictated primarily by available equipment (such as embedded sensors in a smartphone) rather than by designing for performance. As such, there is a heavy reliance on GPS, with the other available sensors, such as embedded inertial and magnetic, offering only a small percentage contribution to the task of determining useful position information. This works reasonably well outside, but in a challenged urban environment or indoors, GPS is not available, and the quality of the other available sensors is poor, leaving a large gap (in other words, uncertainty) in the quality of the position information. Though advanced filters and algorithms are typically employed to merge these sensors, without either additional sensors or better quality sensors, the software does little to actually close the uncertainty gap, which ultimately significantly lowers the confidence in the reported position. This is conceptually illustrated in Figure 4.
Figure 3. Industrial MEMS IMUs are advancing in performance, size, cost & integration (no compromise) to uniquely enable critical applications, such as first responders or military personnel
By contrast, the industrial dead reckoning scenario, such as first responder, is designed for performance with system definition and component selection guided by specific accuracy requirements. Significantly better quality inertial sensors allow them to take the primary role, with other sensors carefully leveraged to reduce the uncertainty gap. Algorithms are conceptually more focused on optimal weighting, handoff and cross correlation between the sensors, along with an awareness of environment and real-time motion dynamics, than they are on extrapolating/estimating position between reliable sensor readings (see Figure 5).
Accuracy in either case above can be improved via improved quality sensors, and while the sensor filtering and algorithms are a critical part of the solution, they do not by themselves eliminate the gap in coverage from limited quality sensors.
Precision location & mapping (PLM) system
For the specific case of first responder tracking, the mission has been partitioned into the following stages, in order to best assess sensor processing requirements: arrival at scene; deployment; inside the building; and rescue (see Table 7). It is envisioned that the fire truck is equipped with a high-end GPS/INS system, which is capable of geofixing the position of the vehicle upon arrival at the scene, as a known reference point. From this point, and until the firefighter enters the building, there is an indeterminate and random sequence of movement for which the PLM system relies on an ultrawideband ranging implementation to maintain an accurate fix of the firefighter position and orientation. Upon entry into the structure, the inertial sensors become the primary tracking sensor, with the goal of providing location accuracy of a few metres.
Figure 4 & Figure 5
The system is designed to rely solely on inertial sensors if need be, but also be able to take advantage of other signals of opportunity when available and reliable, such as UWB ranging signals, magnetometer corrections and barometric pressure measurements. As discussed earlier, the implemented algorithms not only track location, but generate a real-time path map of the search pattern. If a firefighter goes down or is in distress, the map generated from the initial path is a supplemental sensor input to the rescue firefighter, who is also guided by inertial sensing.
While high performance sensors are certainly at the heart of the PLM system, the following are critical enablers of the system as well:
• Deep understanding of the component sensors, and their drift characteristics/limitations under stress.
• Extensive knowledge of the human body movement model.
Figure 6. The PLM system is a complete sensor-fused solution, based on high performance sensors, complementary sensor filtering & processing, + cloud-based databases & analysis. Outputs are precise location & search path map
• Detailed application level insights and operational modes definition.
These provide the definition, guidance and boundaries for the implementation of the sensor fusion processing (see Figure 6). The core of the processing is a particle filter, which tracks multiple possible movements over time, eliminating errant paths as the filter distinguishes them. The sensors themselves are distributed on the firefighter for optimal performance and a wireless body network, as well as a rugged backhaul communications network seamlessly connecting firefighter, rescuer, command and control, and cloud-based maps and coordination, where possible and useful.
The PLM system provides an infrastructure-free approach to detecting position, leveraging high performance sensors and advanced algorithms to optimally merge all signals of opportunity. System goals are metre-level accuracy and real-time path map generation. Advances in industrial grade MEMS inertial sensors have enabled PLM and a full systems development approach allows addressing technical hurdles, while also achieving commercial metrics.
Table 7. Sensor requirements for distinct phases of first responder mission
Continuing work is focused on integrating the latest generation sensor advancements and matching these to new insights in the first responder operational scenario definition. Final integration will include optimised form factors and body placement, as well as more complete implementation of the required communications links and final system qualifications.
Contact Details and Archive...