The EU’s TULIPP project enhances embedded vision applications
30 October 2018
TULIPP: Towards Ubiquitous Low-power Image Processing Platforms, an EU initiative on high-efficiency embedded systems for image processing applications, have delivered their first three use cases – medical x-ray imaging, automotive ADAS and UAVs.
The new use cases, coupled with the TULIPP embedded computing reference platform, deliver outstanding results for embedded vision applications.
The medical x-ray imaging use case combines an embedded computing board with a medical x-ray imaging sensor to eliminate the noise on images when radiation doses are reduced. The ADAS use case enables the implementation of pedestrian detection algorithms running in real time on a small, energy-efficient, embedded platform. The UAVs (unmanned arial use case equips a UAV with real-time obstacle detection and avoidance capabilities based on a lightweight and low-cost stereo camera setup.
Said Philippe Millet of Thales and TULIPP’s project co-ordinator: “At first glance, medical x-ray imaging, ADAS and UAVs would appear to have very little in common.
“But that’s only true when viewed from the perspective of the final application, as they all have a requirement for high-performance image processing and they also suffer from the so-called SWaP (size, weight and power) computing constraints typical of embedded systems.
“TULIPP has addressed these challenges by taking a diverse range of application domains as the basis for defining a common reference processing platform – comprising the hardware, the operating system and its programming environment that captures the commonality of real-time, high-performance image processing and vision applications.”
TULIPP’s medical x-ray imaging use case demonstrates advanced image enhancement algorithms for x-ray images that run at high frame rates. It focuses on improving the performance of x-ray imaging Mobile C-Arms, which provide an internal view of a patient’s body in real time during the course of an operation; it increases surgeon efficiency and accuracy with minimal incision sizes; and it aids faster patient recovery and lowers nosocomial disease risks.
Using TULIPP’s embedded hardware reference platform (which is roughly the size of a smartphone), the said medical use case demonstrates how radiation doses – to which patients and staff are exposed, and which are typically 30 times ambient radiation levels – can be reduced by 75% at the same time as maintaining the clarity of the real-time x-ray images. These would otherwise be rendered useless by the increases in the noise level on the images, which a reduced radiation dose can cause.
ADAS adoption is dependent on the implementation of vision systems, or on combinations of vision and radar, and the algorithms must be capable of integration into a small, energy-efficient electronic control unit (ECU). An ADAS algorithm should be able to process a video image stream with a frame size of 640 x 480 at a full 30Hz (or at least at the half rate).
Moreover, the ADAS use case demonstrates pedestrian recognition in real time based on the Viola & Jones algorithm. Using the TULIPP reference platform, the ADAS use case achieves a processing time per frame of 66ms, which means that the algorithm reaches the target of running on every second image when the camera runs at 30Hz.
TULIPP’s UAV use case demonstrates a real time obstacle avoidance system for UAVs, based on a stereo camera setup with cameras orientated in the direction of flight. As popular as the name ‘autonomous drones’ is, most current systems are in fact still remotely piloted by humans. The use case uses disparity maps, which are computed from the camera images, to locate obstacles in the flight path and to automatically steer the UAV around them. The use case in question is the necessary key towards totally autonomous drones.
To quote Philippe Millet, TULIPP’s project co-ordinator – he concludes: “The TULIPP use cases, coupled with the development kit, comprising hardware platform, multi-core operating system, development tool chain and guidelines, have demonstrated that the computational demands of complex image processing can be delivered in a diverse range of embedded applications – within the context of challenging size, weight and power constraints.”
The above uses cases will be demonstrated for the first time at Vision 2018 in Stuttgart, Germany from 6th to 8th November 2018. TULIPP will also hold a practical workshop on the Project at the HiPeac 2019 Conference in Valencia, Spain on 22nd January 2019.