With the rapid development of driverless cars, the environment perception solution relying on single sensor cannot meet the demands of vehicles target detection in complex traffic scenarios. Fusion of multiple sensors has become a mainstream perception solution for driverless vehicles. In this study, a vehicle detection method in traffic environment based on the fusion laser point cloud and image information is proposed. Firstly, the deep learning method is used to detect the object data collected by the lidar and the camera sensor. Secondly, the Hungarian algorithm is utilized to track the target detection results in realtime. Then, the characteristics of the detection and tracking results from the two sensors are optimally matched with each other. Finally, the matched and unmatched targets are picked and outputted as the final perception results. The proposed algorithm is evaluated in some traffic environment tracking sequence of the public dataset KITTI and real road testing. Experimental results show that the realtime vehicle detection accuracy of the proposed fusion method increases more than 12% and the number of false detection decreases more than 50%.