Dual Mass Flywheel (DMF) has better damping capacity than the conventional Clutch Torsional Damper (CTD), and is more suitable for diesel engine, Dual Clutch Transmission (DCT) and hybrid vehicles. Dual Mass Flywheel-Radial Spring (DMF-RS) is a DMF that has a specific structure. In the light of working principal and static analysis, the hard nonlinear torsional stiffness of DMF-RS is derived in this paper, which is very important to a driveline damper. On this basis, a simulation model is developed to analyze the dynamic response of DMF and CTD excited by idle engine; the comparison of the two dampers reveals that the DMF has better damping capacity, high-frequency filter ability and can reduce crankshaft load. Further, the nonlinear vibration transfer behavior of DMF-RS and its influencing factors are investigated using the method of harmonic balance; it not only indicates that appropriate damping ratio will avoid unstable jump and bifurcation, but also illustrates that resonance frequency of DMF-RS is lower than that of CTD. Besides, the virtual prototype is optimized by Generalized Reduced Gradient (GRG) method, and the results show the influence of variables on damping capacity. The use of the model can be applied as theoretical foundation and can reduce the time to design a DMF-RS since the optimal approach is more efficient, which will be utilized to guide the future experimental studies.