Mixed reality cockpit: pilots' breakthrough HMI
26 September 2017
This ‘wearable cockpit’ technology is designed to revolutionise the training and combat jet cockpits of the future – offering much higher human-to-machine intuitiveness than traditional aircraft controls.
A ground-breaking ‘wearable cockpit’ technology concept, designed to revolutionise the training and combat jet cockpit environments of the future, has been unveiled for the first time by engineers at BAE Systems’ Military Air & Information business.
At its core, the next-generation demonstrator leverages the adaptability of the Striker II helmet-mounted display (HMD) (see below picture) from BAE Systems’ Electronic Systems business in Rochester, Kent. The full-colour HMD is used to project augmented and virtual reality interactive cockpit displays and controls directly in front of the pilot’s eyes, replacing current physical cockpit layouts.
The technology allows pilots to customise the cockpit display and the way they interact with it based on their own personal preferences and mission objectives. By replacing the limitations of current physical cockpits with a projection, which is capable of being infinitely reconfigured, the technology is designed to provide major improvements in situational awareness, speed of decision-making and the ability to affordably and rapidly upgrade the cockpit in line with aircraft enhancements.
Engineers at BAE Systems’ site in Warton are now exploring technologies that build on the current capabilities of the Striker II HMD. These include voice control, use of gestures and touch to allow the pilot to interact with these virtual cockpit controls by, for example, selecting a cockpit control on the helmet display and then using gesture to reposition it.
The aim is to develop a capability to meet the changing and increasingly complex demands that future developments in combat aircrafts will bring. The concept will be usable in multiple aircrafts, reducing development time, cost and training requirements.
Said Jean Page, Lead Technologist in Human Factors at BAE Systems’ Military Air & Information business: “Using the colour display on the Striker II helmet as the building block, our concept envisages a wearable, software-reconfigurable cockpit for future combat aircrafts with limited or no hardwired controls or displays.
“Pilots will be able to reconfigure the mixed reality set-up being projected in front of their eyes however they want, and the capability of the concept will significantly improve their situational and spatial awareness. Flexible control technologies, such as voice control, coupled with virtual controls and gestures, will provide the pilot with the ability to react to the requirements of a mission much faster than the current technology allows – and with reduced training.”
The concept would introduce several potential benefits to the pilot, including the ability to change the number and sizes of displays to the pilot’s precise preferences, without being restricted by physical space; and the ability to lay concepts on to the display such as ground terrain overlays, pop-up tracking windows, restricted airspace and 3D situational displays.
Steve Formoso, Chief Test Pilot at BAE Systems’ Military Air & Information business, added: “We believe these kinds of new technologies will provide the pilot with a vital advantage as future aircrafts continue to develop. These concepts, particularly if used in conjunction with the Striker II HMD, offer the potential for pilots to do things much more quickly. In a combat aircraft, if you can do things quickly and intuitively, it provides a vital edge over whoever you are flying against.”
Contact Details and Archive...