Two experiments were carried out to clarify the characteristics of manual driving when the task of vehicle control is transferred from an autonomous driving system at SAE levels 3 and 5 to manual driving.The first experiment involved another vehicle merging into the lane of the host vehicle from the left side of a highway. This experiment simulated the functional limit of a level 3 system with the driver in a situation of low alertness. When the other vehicle changed lane in front of the host vehicle, the driving task was transferred from the system to the driver. The second experiment simulated a driver travelling along a city road with manual driving after the driver used the system in a situation of sleeping on a highway. In this experiment, a pedestrian emerges from a blind spot along a city road, and the driver needs to brake having recently awaken.In the first experiment, the driver with low alertness could not control the vehicle when manually driving. In the second experiment, the driver took a long time to deploy the brake and could not properly operate the brake pedal.There are two main results of this research. (1) The autonomous driving system has to detect the driver’s situation and activate a warning that notifies the driver of the change in the driving task (level 3). (2) If the driver uses the system in a situation of sleep, manual driving is possible only after the driver wakes completely (level 5).