Design of Machine—Vision—Based Statistical System of Drivers’ Safety Awareness

Yuan Yuan Li Cangyuan Wen Shuran Chen Weiqi Xu Kemeng

1. Executive Summary

February 21， 2016 The State Council of the CPC Central Committee issued the “Opinions of the Central Committee of the Communist Party of China on Further Strengthening the Management of Urban Planning and Construction”， of which Article 16 mentions that “the completed residential quarters and units have to be gradually opened and the internal Public roads， to solve the problem of traffic network layout and promote the conservation of land use. “However， there are many pedestrians in communities， many of them are elders and children. If the road within to community open to the public， can pedestrian safety be guaranteed？ The project hopes to put forward constructive suggestions on the feasibility of this policy based on the data of the speed of vehicles in the community and the number of accidents near junctions.

The project uses existing mature radar technology and artificial intelligence recognition technology， to achieve the rate of speed measurement and recording up to 1-200km/h， and the rate of license plate recognition success equals to or more than 80%， and to support recognizing more license plate in one vision field. Based on embedded system design， our equipment is easy to carry， have a certain endurance， and is waterproof. Recorded data can be stored offline. If it connects the public Wi-Fi， it can support real-time data transmission.

The results of the analysis of the final data obtained from the project can provide suggestions for “public the roads within the community” and alert the speeding vehicles. At the same time， because of the small videos storage capacity and low maintenance cost， the device can be widely used in residential or public safety system.

Nevertheless， this project still has a lot of space for improvement. Because the current design is only for slow vehicles， the performance of camera equipment is poor， the device can not accurately identify the high-speed vehicles， so the camera performance can be further enhanced. In addition， the battery power increase allows long-term unmanned working which saves human resources. For systems， it can be further enhanced by integrating the device into a small radar so that being more portable. For the exterior aspect， due to the lack of time， the appearance is not designed and shabby， but we can have a perfect device in the future.

2. Background

Due to the serious traffic congestion in urban areas， it is very difficult for vehicles to travel during the congestion period. To reduce traffic congestion and improve the driving efficiency of vehicles， the State Council issued the “Opinions of the Central Committee of the Communist Party of China on Further Strengthening Urban Planning and Construction Management”， proposed the advocacy of opening the community fence. However， in our daily life， we often see the phenomenon of overtaking， speeding at intersections and curves， as well as conflicts between vehicles and pedestrians. If we open the community fence， these inappropriate behaviors may pose a threat to pedestrian safety. Based on this， we decided to design an evaluation system.

The old speed acquisition system has been used in very limited area such as traffic violation detection but not this application. We determine to make a low-cost device that can quantize drivers safety awareness through computer vision with Doppler radar. The device will be placed in the community intersections and crossroads for a long time， recording the proportion of speeding， the size of the traffic flow， the number of accidents and so on. Based on these data， we can analyze and evaluate the feasibility of this strategy.

As we all know， nowadays， most of the computer vision systems are based on neuron network dealing with rich data， which need high speed computing. However， to an embedded system， its hard to make a balance among efficiency， cost and stability. The GPU manufacturer， NVIDIA， gives a solution called ‘Tegra Series by integrating a GPU with high performance and a CPU with low power cost on a single chip. It is possible to achieve high computing performance in small size using Tegra Chip. Our system is exactly built on it. Besides， to make a thorough inquiry on the improvement of adding a GPU to processor， we will also make one more version of the software system for other two platforms， one is for PC， another is for a board called Lattepanda， which has more memory than Tegra K1 and equipped with Intels low-end processor Atom.

Based on the high-performance platform and various sensors， the project will focus on the traffic data collection and analysis. The Character Recognition in Vehicle License Plate System will be implemented to the project and further study on the algorithm about data store and analysis will go on. Finally， according to the data and the result of the analyze， judge the feasibility of the policy.

3. Objectives

Up to now， our product can detect traffic flow， the speed of each car， license plate and time that each car go through. In addition， we have prepared a protective device for it.

The detection of traffic flow and license plate can be accomplished by importing the video in the program， which can achieve high-frequency image acquisition performance through camera drive. The statistics of traffic flow is provided by importing the measured data into Excel. License plate detection can get its color， the provinces （Chinese） and the license plate number （letters and numbers）. In particular， the identification accuracy can reach more than 80% with high efficiency. It is also completely based on OpenCV implementations， thus the portability of transplanting is ensured， so the same code can run on both Windows and Linux platforms. Speed detection can measure the direction of the vehicle （forward or backward） and its speed using Doppler micro-blog speed measuring radar.

We have also prepared a protection device for the electrical systems. A proper shelf will be used to equip the sensors and the core part will be put into a locked iron box with lock. In addition， we have designed a long-term， low-cost outdoor power supply system. The system has large battery capacity， which can output various types of voltage to satisfy the core board and sensors. Through the multi-stage stabilizer circuit， we can get a stable system current output. The system also has an outdoor wireless network to connect the system to the Internet， as a result， even if the whole device is placed in a weak-signal area， it still can complete the task， and thus provide more possibilities for the project.

4. Methodology

We divide the development process of the project into three parts.

The first part of the systematic design of the system is discussed in the laboratory through panelists. The first is what functions our system should have and the discussions on the technical details after the necessary functions have been discussed. This aspect includes the implementation of hardware circuits， and the design of software system solutions.

After discussion， we think that drivers safety awareness can be indirectly reflected by their driving speed in the community. Through long-term statistics on the driving speed of specific driver groups in the community， drivers driving habits and driving safety awareness can be reflected. So， what we must do is to identify a driver with a license plate and identify the drivers driving safety awareness with a statistic of speed. We chose the Nvidia TK1 Evaluation Board for high-performance， low-power scenarios as a central part of the hardware solution. After consulting the information vendors and other means， we determined the speed measurement program using microwave radar， license plate sampling using high-speed video camera. Hardware system， we designed a multi-level power supply voltage regulator circuit solution to the system for a variety of high-quality power requirements. Considering the future of system debugging convenience and scalability， we have joined the system of high-power WIFI components.

In terms of software， we designed our own vehicle and license plate recognition solution in the first half of the year， but the result was not good enough as the sample library was not ideal. Later， we adopted the open source project EasyPR， which provides more than 80% recognition rate and supports a variety of license plates. It is based entirely on OpenCV development which makes it easy for us to port it to our embedded system. The systems main application uses Python， which is simple to develop and easy for us to maintain.

The second part is the test session， we will test the link in two steps， the first step is to use the radar and the camera in real time on the road in a period to collect data on the PC， the second step is to return to the laboratory. The data collected outside will be simulated as real-time data to input to the main program on TK1， to further debug the main program and design. This test method on the one hand can reduce the cost of outdoor testing to reduce the possibility of unknown errors in the outdoors. On the other hand， we can have enough time to back-end the main program and hardware debugging and further improvements.

The third part is machine debugging. After the last part is completed， we bring the whole machine to the outdoors and carry out real-time tests on all aspects including the hardware and software systems. To carry out such a test is to confirm the system can work in real time and to collect data for further analysis and optimization.

5. Key results and achievements

5.1 Key results and discussions

The bar chart above is a map of the speed of the vehicles measured at the certain part of a road during the evening peak. This section is in a traffic hub， so there are lots of pedestrians during the rush hour. What need to be noted is that since these data are collected during the evening peak period， the number of vehicles is slightly higher than the number in normal time.

In the figure， the speed of vehicles that below 10km/h is 85%. This is a critical value， the vehicles driving under this speed can circumvent the pedestrians successfully. However， if the speed is higher than this value， considering the reaction time， it is very uncertain that whether the vehicle can circumvent the pedestrian at once. We can see that about 15 percent of vehicles run at dangerous speeds， so if our country wants to successfully implement the call， this kind of drivers need to be reminded.

The components of the system and their functions are as following：

（1）Use high-power WIFI to connect with nearby public network for easy debugging and system scalability.

（2）NVIDIA TK1 development board has a high energy efficiency ratio. It is suitable for performance requirements of embedded systems.

（3）Use Doppler microwave radar. This speed measuring device has no interference to drivers and the environment， and requires less equipment.

（4）Camera for license plate recognition.

5.2 The achievement against activities and milestones

From February to April， the camera driver has been completed， which can achieve high-frequency image acquisition performance； the core vision library ‘OpenCV has been compiled for the Tegra board and several optimizations have been done； a long-term and low-cost outdoor power supply system with a battery of large capacity which can export various types of voltage to satisfy the core board and sensors has been made， and the output current pattern of the system is smooth.

In May， the placement and protection of the electrical system has been determined. A proper shelf has been used to equip the sensors and the core part will be put into an iron box with lock.

From June to July， an outdoor wireless network has been implemented to the project， which makes the system connected to the internet even if being placed to a weak signal area. Thus， make more possibility for the project. Besides， the Character Recognition in Vehicle License Plate System can achieve over 90% accuracy with high efficiency and the portability of the system was kept， and consequently， the same code can run on both windows and Linux.

From August to November， we have optimized the hardware and software aspects of the system， including the optimization of the circuit power and real-time software. At the same time， we have a new design for the system that makes it waterproof while taking up only a small amount of space.

6. Impacts

6.1 SCIENTIFIC IMPACT

We are the first to port EasyPR to the embedded system， to make it possible to identify real-time license plates based on the TK1 embedded system.

6.2 COMMUNITY IMPACT

The feasible premise of the policy， “Public the Community Roads”， is that most drivers value pedestrian safety. When derivers meet the pedestrians， drivers should let the pedestrians go first， no accelerating. The system can record traffic conditions， speed， number of traffic accidents and other information at intersections， providing accurate data， sending results to government and providing data for managers to analyze the feasibility of the policy of “Public the Community Roads”， which can ensure the safety of pedestrians. Meanwhile， the device can record vehicle violations， assisting managers in enforcing the law， so that making the roads near the junctions more orderly， the society more stable and provide more security to pedestrians. The device can also warn vehicles over-speeding and other illegals at the crossing， according to the data， increase the drivers safety awareness and pay more attention to the pedestrians. The safety awareness of drivers is enhanced， and the safety of pedestrians can be guaranteed. And then the feasibility of this policy will be enhanced， which can ease the pressure on roads. At the same time， road safety will be more orderly and social safety will be guaranteed.

7. Conclusions and recommendations

Through this project， we have learned a lot about the development of embedded systems， image processing， hardware circuit design. On the hardware side， we have grasped the design and optimization of the hardware circuit by referring to the data and actual hands-on operation. For example， in power optimization， considering the power requirements of the radar on the one hand， the voltage is accurate and stable， on the other hand， the current is small， so we designed a separate boost circuit to power the radar. In software， we conducted a series of software development work. The first is the porting of drivers and libraries， which gave us a lot of knowledge of embedded Linux development. Second， since our project is to be designed， we focus on software development to the entire system level. This allows us to face the project from a completely new perspective.

In the future， we hope to use a wider variety of visual applications to capture and count drivers detailed driving habits. At the same time， we want to make the system smaller and consume less power. Besides， we hope to improve the usability of the system so that it can be put into real life in the future.

Acknowledgments

First， we would like to extend our sincere gratitude to our tutor， Prof. Duan Lijuan， for her instructive advice and useful suggestions about image processing on our thesis. We are deeply grateful to her help in the completion of this thesis.

We are also deeply indebted to all the other tutors and teachers in Fan Gongxiu Honors College， Bjut and School of Instrumentation Science and Opto-electronics Engineering， Beihang University for their direct and indirect help to me.

Special thanks should go to our friends who have put considerable time and effort into their comments on the draft.

Appendixes

Figure 6 is the scene of our outdoor real-time testing. Pedestrians and vehicles are numerous during the late peak of the road. Close to the situation after opening the cell wall， so as we are the test site.

Figure 7 In terms of budget， we spent a total of 7，500 yuan， of which radar and TK1 development board accounted for 84% of the total budget， while the other components accounted for 16% of the budget.

References：

[1]Cortes，C.Vapnik，V.Support-vector networks[J].Machine Learning， 1995，20（3）：273–297.

[2]Pulli，Kari，Baksheev，Anatoly，Kornyakov，Kirill，Eruhimov，Victor（1 April 2012）.Realtime Computer Vision with OpenCV[J].Queue，2012： 40：40–40：56.

[3]Possel，Markus.Waves，motion and frequency：the Doppler effect.Einstein Online，Vol.5.Max Planck Institute for Gravitational Physics， Potsdam，Germany. Retrieved September，2017，4.

【作者簡介】Yuan Yuan， Li Cangyuan， Wen Shuran， Chen Weiqi， Fan Gongxiu Honors College， Beijing University of Technology； Xu Kemeng， School of Instrumentation Science and Opto-electronics Engineering， Beihang University.