With the development of deep neural networks and artificial intelligence, Autonomous Driving Systems (ADS) are developing rapidly. According to the commercialization of Autonomous Vehicles (AVs), non-AVs and AVs will drive simultaneously on the road. The stability of autonomous vehicles can significantly affect the entire road condition. In this study, we use a Deep Reinforcement Learning (DRL) approach to making an AV learn a reasonable lane-changing and the acceleration control to keep the desired velocity. For the learning efficiency of the AV, it provides minimal state information and replaces the lane-changing action space with a lower level. Therefore, we modified the action selection method of TD3 and used it. Finally, the driving performance of the TD3-based AV and the LC2013-based vehicle is compared in various environments. The TD3-based AV performed better than the LC 2013.