The NVH performance is one of the most important concerns in vehicle development. For all-wheel drive (AWD) vehicles and rear-wheel (RWD) drive vehicles, prop shaft is a major transmission component which may cause various NVH problems. This paper focuses on the vehicle NVH problems caused by the second order excitation force of prop shaft. In order to control the NVH performance of the prop shaft efficiently and fundamentally, this work first studied the rotation kinematical characteristics of prop shaft. Then a rigid-elastic coupling model of vehicle driveline was built with the theory of multi-body dynamics. With this model, the sensitive factors that may affect the second order excitation force were investigated. This paper also describes a case study to verify the conclusions which are revealed from the theoretical calculation and the simulation. In this case, a vehicle vibration problem caused by the second order excitation force of prop shaft was successfully solved by following the indicative conclusions mentioned above. The main significance of this study is to establish an effective approach for analyzing and controlling the NVH issues related to vehicle driveline system. The indicative conclusions of this work could be referred by NVH engineers in the early phase of vehicle development.