Design of lane departure warning system based on FPGA

Abstract

This paper introduces an implementation scheme of lane departure warning system based on FPGA chip and based on digital image processing technology and SOPC technology. The system uses the CCD camera to collect the image in front of the vehicle, uses Hough transform to realize lane detection, and uses the edge detection function to complete the departure warning function. The system has good portability, flexibility and versatility. The hardware structure and software design ideas of the system are discussed in detail, and the advantages of the system are analyzed.

Introduction

With the development of China's national economy, the number of cars has increased dramatically, and the rapid development of highway construction has caused the current road traffic to show a trend of high-speed driving, intensive traffic flow, and driver non-professionalism. Traffic accidents caused by car collisions Endanger the safety of people's lives and property. According to the statistics of the National Highway Traffic Safety Administration (NHTSA) in 2009 where passenger cars collide in traffic accidents, in all kinds of traffic accidents, the frontal collision of vehicles accounts for 62% of the total accidents, and most of them are Due to traffic accidents caused by lane departures, by designing an early warning system for vehicle lane departures, the probability of a forward collision can be effectively reduced, and the loss of highway traffic accidents can be effectively reduced.

This article takes advantage of the SoPC (programmable system on chip) technology design to be flexible, scalable, scalable, and short in design cycle, and designs a lane departure warning system that can be flexibly configured and easily upgraded and maintained.

1 Overall technical route of the system

According to the system functional requirements, the system's processing flow is divided into three stages: image preprocessing, feature extraction, and feature discrimination.

(1) Image preprocessing

The system obtains the image of the lane ahead of the vehicle through the camera, and the digital image processing technology completes the collection, graying and filtering enhancement of the digital image;

(2) Feature extraction

Use Sobel edge detection algorithm to complete the image edge detection, and on this basis, use Hough transform to complete the lane line inspection;

(3) Feature discrimination

According to the detected lane marking information, the edge detection function EDF is used to make the relevant departure judgment.

2 System hardware design

The system uses a CCD camera to collect lane images, decodes the acquired video through the video analog-to-digital conversion chip ADV7181, obtains a digital video message, and stores it in SDRAM for LCD display, and uses it as original information for subsequent image signal processing The unit completes the image processing task and extracts lane information from it.

The soft-core processor Nios II embedded in the FPGA is the control core of the entire system, completing the overall software process execution control, Hough transformation, lane detection and collision warning tasks of the system; the Nios II processor is completed through the Avalon switching architecture and other components of the SOPC system Data exchange and control. The image signal processing unit of the internal components of FPGA completes the tasks of digital image signal preprocessing, median filter denoising, Sobel edge detection and other tasks, and its processing results are used as the basis for Nios II to detect lane departure.

Peripheral storage devices SDRAM, FLASH, SRAM, SDRAM is used to store image information, FLASH is used to store programs and system parameter configuration, SRAMN is used to store software temporary storage data: the system uses buttons and LCD as the human-machine interface. Additional logic is used to complete other auxiliary tasks of the system.

2.1 Design of SOPC system

The core of the SOPC system is the Nios II CPU. In this paper, the standard Nios II IP core is used to achieve a balance between hardware scale and performance; the JTAG UART is used to complete FPGA configuration and system debugging; the display interface IP includes SDRAM and SRAM The IP core of FLASH completes the connection and communication between the SOPC system and the external storage device of the FPGA; the interface of the human-machine interface device completes the connection of the LED, LCD, keys and the SOPC system.

2.2 Design of image signal processing unit

The image signal processing unit uses Verilog HDL to implement FPGA-based image preprocessing, median filtering, and Sobel edge detection functions. During the design process, TFPGA is fully utilized for parallel processing and pipeline processing.

2.2.1 Fast median filter unit

The fast median filter unit is mainly composed of a 3×3 template generation module, a median filter algorithm module, and a row and column counting module.

2.2.2 Sobel edge detection unit

The Sobel edge detection unit consists of three parts: a 3×3 template generation module, a gradient operation module, and a gradient comparison output module. The three parts work in a pipelined way, through the movement of the window template, to complete the edge detection task of the entire image.

3 Software flow

3.1 System software flow

The software execution flowchart of the lane departure detection system is shown in the figure below. After power-on, complete the initialization, configure the relevant configuration information of FPGA in EPCS and the program in Flash into FPGA, and complete the initialization of system components such as cameras.

After the system is initialized, enter the lane departure detection and warning process. Start the camera to start image acquisition, and call the FPGA internal image processing module to perform color space conversion, grayscale, median filtering, edge detection and other operations on the image.

After the program loads the edge-detected image, the image is divided into left and right parts for straight line detection. Taking into account the fact that in practical applications, the probability that the lane is close to horizontal or vertical is extremely small, and also in order to filter out interference (such as the horizon, roadside light poles, vehicle edges in front, etc.), in the process of straight line detection using Hough transform The following strategy is adopted: in the left half of the image, the direction angle is between 95° and 175° for straight line detection; in the right half of the image, the direction angle is between 5° and 85° for straight line detection. The search process traverses the entire image, the search calculation is completed, and the local maximum value is found in the accumulator A(ρ, θ), thereby determining the position and parameters of the lane marking line.

After calling the Hough transform function to identify the straight line, if there is available lane information after image processing, enter the lane departure warning and judgment process. The lane departure warning also uses a two-level warning mechanism. When the departure angle is greater than the warning value, it emits sound and light Warning; when the deviation angle is less than the warning value but greater than the reminder value, a sound and light reminder is issued. If there is no relevant lane information after image processing, it returns to the obstacle detection and collision warning process.

4 Conclusion

The system makes full use of the FPGA's programmable and SoPC system's reconfigurable features. It is very convenient to upgrade and maintain the system, which can greatly extend the life cycle of the system. At the same time, a single-chip solution with FPGA as the core is adopted. The peripheral circuit is simple and can make The volume of the system is made very small; in addition, the system can also be accelerated through custom modules, custom instructions, C2H, etc. The idea is to sacrifice hardware resources in exchange for the increase in computing speed. Through acceleration, the real-time requirements of image processing can be achieved, thereby further improving the real-time performance of the lane departure detection system and improving the practicality of the system.