Search Results

The around view monitor (AVM) system for vehicles usually suffers from the distortion of surrounding objects caused by incomplete rectification and stitching, which seriously affects the driver's judgment of the surrounding environment during the reversing process. In response to solve this problem, an augmented around view monitor (AAVM) system fusing image and depth information is proposed, which highlights the point clouds of persons or vehicles at the rear of the vehicle. First, an around view image is generated from four fisheye cameras. Then, the calibration of multi TOF cameras is conducted to improve their accuracy of depth estimation and obtain extrinsic camera positions. Next, the 2D-driven object point cloud detection method is proposed to localize and segment object point clouds like vehicles or persons.

The driver's face detection and tracking method important for Advanced Driver Assistance Systems (ADAS) and autonomous driving in various situations. The deep SORT algorithm has integrated appearance information, the motion model and the intersection-over-union (IOU) distance methods, and has been applied to face tracking, but it depends on detection information in every frame. Once the detection information lacks, the deep SORT algorithm will wait until the target detects bounding boxes appear again, even if the target didn’t disappear or shield. Hence, we propose to use a new tracker that not completely depend on the detection algorithm to cascade with the deep SORT algorithm to realize stable driver's face tracking. At first, the driver's face detection and tracking will be accomplished by the MTCNN-deep-SORT algorithm.

Highly autonomous vehicles have drawn the interests of many researchers in recent years. For highly autonomous vehicles, a high-definition (HD) map is crucial since it provides accurate information for autonomous driving. However, due to the possible fast-changing environment, the performance of HD maps will deteriorate over time if timely updates are not ensured. Therefore, this paper studies the updating of lightweight HD maps in closed areas. Firstly, a novel two-layer map model called a lightweight HD map is introduced to support autonomous driving in a flexible and efficient way. Secondly, typical updating of scenarios in closed areas with non-paved roads is abstracted into operations including area border expansion, road addition, and road deletion. Meanwhile, a map updating framework is proposed to address the issue of map updating in closed areas. Finally, an experiment is conducted to demonstrate the feasibility and effectiveness of the proposed map updating approach.

The around view monitor (AVM) system for the long-body road vehicle with multiple articulated carriages usually suffers from the incomplete distortion rectification of fisheye cameras and the irregular image stitching area caused by the change of relative position of the cameras on different carriages while the vehicle is in motion. In response to these problems, a set of calibration and stitching methods of AVM are proposed. First, a radial-distortion-based rectification method is adopted and improved. This method establishes two lost functions and solves the model parameters with the two-step optimization method. Then, AVM system calibration is conducted, and the perspective transformation matrix is calculated. After that, a static basic look-up table is generated based on the distortion rectification model and perspective transformation matrix.

Panoramic images can provide critical information for Advanced Driving Assistance Systems (ADAS), such as parking spaces and surrounding vehicles. However, the vehicle in the bird's-eye view image is severely distorted and incomplete, and the visual information becomes very blurred in some illumination insufficient environments. If the driver cannot see the surrounding environment information, the risk of collision will increase, especially during parking. To better percept the local environment with the help of panoramic images, we use panoramic image segmentation results to construct a virtual surround view monitoring system to provide drivers with clearer perception information. Firstly, a lightweight segmentation network is redesigned based on SegNet, which will improve the accuracy of the segmentation without increasing the model’s inference time. Secondly, we build an augment visualization around view monitor (AV-AVM) system with regards to the segmentation results.

As one of the most challenging driving tasks, parking is a common but particularly troublesome problem in large cities. Recently, an excellent solution-automated valet parking (AVP) has become a hot research topic, which allows the driver to leave the vehicle in a drop-off area, while the vehicle driving into the parking slot by itself. For AVP, the precise localization is an indispensable module. However, the global positioning system (GPS) cannot be used in the underground parking lot and the localization method based on lidar is too expensive. In response to solve this problem, we propose a simultaneous localization and mapping system with the semantic information of parking slots (PS-SLAM), which is based on visual-inertial and around view images. First, the calibration of multi-sensors is conducted to obtain their intrinsic and extrinsic parameters. In this way, the around view image and transformation matrices between sensors can be acquired.

The driving risk field model offers a feasible approach for assessing driving risks and planning safe trajectory in complex traffic scenarios. However, the conventional risk field fails to account for the vehicle size and acceleration, results in the same trajectories are generated when facing different vehicle types and unable to make safe decisions in emergency situations. Therefore, this paper firstly introduces the acceleration and vehicle size of surrounding vehicles for improving the driving risk model. Then, an integrated decision-making and planning model is proposed based on the combination of the novelty risk field and model predictive control (MPC), in which driving risk and vehicle dynamics constraints are taken into consideration. Finally, the multiple driving scenarios are designed and analyzed for validate the proposed model.