Design of real-time panoramic video system based on FPGA platform

This paper proposes and implements a design scheme of panoramic video processing system based on FPGA. The system is implemented in Xilinx Spartan-6 FPGA full hardware monolithic mode, including image acquisition, image pre-processing, image correction, panoramic stitching, video compression, Ethernet transmission, HDMI display and other modules of 5 cameras, reaching a 360-degree panoramic view Video quality, high resolution, real-time requirements.

Overview

The current panoramic video stitching schemes are endless, but they are basically processed by software. The disadvantage of this method is that the resolution is lower and the processing speed is very slow. It is difficult to achieve real-time effects. In order to improve the resolution and processing speed of the video system and meet the higher needs of users, this paper proposes and designs a full hardware processing method based on FPGA to achieve a high-resolution, high-quality, real-time 360-degree panoramic video system. The new shows that this achievement is not only blank in the country, but also has a leading position in the international performance index. The system has wide application prospects in military, high-end guard prevention, new-generation video remote network conference and other places.

The video data acquisition end of the system is where five Macron high-resolution CMOS image sensors are placed horizontally, and its position is accurately pre-calibrated. At the same time, it is equipped with a high-quality large-angle optical lens to synchronize and collect video at high speed, and send the collected video to the image pre-processing part. Image pre-processing mainly performs image processing such as color generation and color correction to generate normal color video streams for subsequent operations. Image correction performs real-time correction processing on problems such as lens distortion due to the use of large-angle lenses and parallax distortion due to lens position distribution. Panoramic stitching mainly completes real-time operations such as cylindrical projection model matching, image stitching, and boundary processing to generate a perfect panoramic video stream. This video compression module uses the standard JPEG standard for compression. In order to deal with the real-time compression of ultra-large panoramic video and speed up the compression process, this compression module adopts a customized dual-core parallel compression method. The dual-core parallel compression method not only speeds up the compression process, but also solves the problem of requiring large buffer resources when super-large-sized images are compressed. At the same time, the video compression ratio of this compression module can be adjusted and controlled by Ethernet in the remote back-end part.

The UDP part uses hardware to implement the UDP protocol to increase the speed of data transmission, and at the same time provides a human-computer interaction platform. The HDMI display part has completed the video display control process. The PCIE capture card is connected to the PCIE interface on the PC side, and receives the compressed video data sent from the Ethernet. On the PC side, through software graphic design, real-time display of panoramic video, partial zoom-in, and control of FPGA module processing and other functional windows are realized.

The above is the sharing of the real-time panoramic video system design of the FPGA platform, I hope it is helpful to you.