During the engaging process of sleeve and teeth ring in mechanical transmissions, their rotational speed and position differences cause multiple engaging ways and trajectories, and casual impacts between them will delay the engaging process and cause a long power off time for a gear shift. In order to reveal the engaging mechanism of the sleeve and the teeth ring, it is essential to build a high-fidelity model to cover all of their engaging ways and capture their speed changes for an impact. In this work, our contribution is that their impact process is modeled as a precise, continuous and nonlinear damping model, and then a hybrid automaton model is built to connect the system dynamics in different mechanical coupling relationships. First, in order to solve the vertical impact force on splined chamfers of sleeve and teeth ring, a numerical solving method for nonlinear damping model is proposed through finding a precise relationship of restitution coefficient, stiffness coefficient and damping coefficient. Second, in order to solve the frictional force on the tangential direction of splined chamfers, the switching condition from static friction to kinetic friction and the frictional force direction with relation to the relative tangential speed of sleeve and teeth ring are determined formally. Third, five motion modes are included in a hybrid automaton model to connect different mechanical coupling relationships in an engaging process: impacting on the up chamfer, impacting on the down chamfer, moving forward, meshing and being engaged. Finally, simulations are carried out to describe the trajectory of the sleeve relative to the teeth ring and the continuous changing of the interactions between the two elements during the engaging process.