FPGAs give sensors the right image

30 November 2009

MSB Screen Shot
MSB Screen Shot

Image acquisition is the most critical component of security and surveillance video chains, since the ability of image signal processing to extract useful information out of the video stream is only as good as the quality of image capture – in reality the combination of sensor quality and the quality of sensor output processing.

A growing trend in security and surveillance systems has been the migration of image processing functions to the network edge – i.e. within the camera itself, close to the sensor.

Flexibility and efficiency are among the highest objectives in the product development cycle. This article examines the integration of microcontrollers and signal processing elements for CMOS image sensors within an FPGA architecture. By their very nature, FPGAs and microcontrollers are flexible, but the question remains, does the use of these devices always lead to an efficient design?

A simple rule is that replacing a standard microcontroller with an FPGA cannot, on average, produce a cost effective substitute. The FPGA solution becomes more attractive only when the boundary conditions of the design cannot be accommodated by the integrated peripherals within the microcontroller; for example, the number of PWM channels, the number of counter/timers or the number of I/O ports.

Similarly, the transfer of DSP algorithms into an FPGA can significantly increase the data throughput and the execution speed of the control function. However, there may be some loss of flexibility. Nevertheless, DSP developers can normally take advantage of an FPGA as the functions can be described in VHDL or Verilog-HDL and directly instantiated in hardware. Even when some modules are available as prefabricated IP blocks, the task still remains to integrate these blocks into a complete design.

Helion is a company that works in the field of image signal processing utilising FPGAs and DSPs. Their core knowledge lies in the area of the drive and interfaces to the CMOS image sensors, dynamic expansion and signal pre-processing up to the TFT. Using the Helion system means that many of the functions that determine image quality can be realised directly within the camera itself. Depending on system requirements, these solutions are based on DSP, FPGAs or a combination of both.

Helion has chosen as an FPGA technology the low-cost families from Lattice Semiconductor. The MachXO and LatticeXP2 are non-volatile devices which as single chip solutions enable a particularly compact camera design. As the LatticeXP2 includes Block RAM and DSP blocks, image processing calculations can be optimally implemented. MachXO is the most cost effective solution if only a simple sensor and control function are required. Also available from Lattice are SRAM FPGA families, the LatticeECP2, ECP2M and ECP3. These also include DSP Blocks and the “M” variant includes 1.5MBit to 5.3MBit internal memory.

In the majority of projects Helion also uses the 32-bit microcontroller, the LatticeMico32, which is available free of charge and comes with an open-source license model inclusive of a development environment. This soft core operates mainly on the less time critical control paths where the data through the FPGA is directed through separate routings.

The project specific Mico32 configuration is constructed in the Mico System-Builder (MSB) Software and assembled from a variety of available components including customer specific IP blocks. For the interconnection of these IP blocks and the customer specific peripherals, the Mico32 utilises a bus interface which operates in accordance with the Wishbone specification. The MSB automatically interconnects the internal components of the design, and when all of the required IP blocks are available within the MSB the FPGA design is completed without a single line of VHDL or Verilog VHDL being written.

A simple example can clarify the advantage of the Mico32-based approach. A display controller in the FPGA is routed by an external microprocessor over an I2C bus for control of the picture position. In total, 256 control- and parameter-registers are scheduled. In order that no TV interference appears, the parameters written by the I2C must be processed in the blanking period. The processor releases the control register when the data set input is consistent and can be activated. From a further register the processor obtains the current line number of the display controller.

These functions can be solved directly within an FPGA design. One needs an I2C slave, I/O ports, state machines for the supervision of the IC2 register bank, and for copying the parameters and providing the arbitration for access to the internal memory interfaces of the I2C slaves. The time expended for the corresponding FPGA development in VHDL or VerilogHDL is relatively high and the reusability of the code is confined to HDL encoded controllers.

The alternative Helion Mico32 solution utilises components that are already provided by Lattice in the MSB: I2C slave, GPIO, memory, DMA and user-specific components such as port monitoring to activate memory transfers. The MSB provides the ability to configure the device for “Shared bus” or “Slave Side-Arbitration;” this means that the Arbiter is generated automatically in the correct form. Using this method the FPGA code for combining the components in the MSB is generated within minutes. The C code can then be written for the Mico32 and checked by means of the Debugger provided. All the components are easily reused.

Using this example, many more IP blocks that are intended for the data path can be combined in the MSB; for example, a colour pipeline or the interface to an image sensor. For image transfer, Slave Side-Arbitration is optimal as the MSB generates its own Wishbone Buses. In this way the data flow is not limited through the microcontroller and the arbitration logic and the connection of all components is automated under a single entity – the ideal case for System-on-Chip designs. The more complex design in the MSB clarifies the view that the system description would be substantially more demanding than in the first example in VHDL or VerilogHDL.

On the strength of the elegant and time saving support provided through the MSB, Helion has integrated a large portion of know-how in the area of image sensor drive circuitry and data pre-processing in the form of its own components, which are available as IP cores. Systems can be conceived and realised in a modular form and function proofed within the MSB. An example is shown as a block diagram in Figure 3.

In addition to the MSB, Lattice’s ispLEVER FPGA design software is required for the complete system development environment. The Verilog HDL code produced by the MSB is synthesised and implemented in ispLEVER. Included in the tool is a Reveal logic analyser, the user after debugging the µC code, has the option of checking for fault analysis on the data or control path in real time. Reveal provides extensive trigger and trace options. The processing of the complete picture is not wholly realistic, the debugging of the level of µC source code can not include all events or data sequences in the FPGA. Reveal can however, with a skilful use of triggering signals and logic combinations record sequences in real time and transfer the results to a host computer. For example if one considers setting a Port bit of the Mico32 as a Reveal Trigger and combine this with a specified condition of control and data signals, it is possible to sharply switch the logic analyser with µC-software, this also applies to the debugger.

The MICO System Builder permits classic job sharing. An FPGA designer specifies his own IP blocks and integrates these components into the MSB. The software developer connects the components together on a project basis and develops the C/C++ programs for the Mico32. By skilful division and distribution of the data stream onto one’s own bus structure a high data throughput is achieved. The microcontroller therefore brings, on one hand a high efficiency in the FPGA in addition to the flexibility during the development stage. On the other hand the efficiency of the control and pre-processing allows in many cases further DSP processing following the signal chain, or the option of using a smaller more cost effective DSP.

Dr. Ing Arndt Bussmann works for Helion and Joachim Müller is Graduate Engineer with Lattice Semiconductor

Contact Details and Archive...

Print this page | E-mail this page

This website uses cookies primarily for visitor analytics. Certain pages will ask you to fill in contact details to receive additional information. On these pages you have the option of having the site log your details for future visits. Indicating you want the site to remember your details will place a cookie on your device. To view our full cookie policy, please click here. You can also view it at any time by going to our Contact Us page.