With the rapid development of automobile industry, the intelligent vehicles that can be viewed as the integrated carrier of advanced technology of automobile are paid much attention to by society. It is imperative to study the motion control of the intelligent vehicles due to the nature of their nonholonomic operation constraint system whose dynamic characteristics are highly nonlinear with the uncertainty of parameters. In this paper, utilizing the vision system of intelligent vehicles, a vehicle lateral control strategy based on image segmentation is established to enhance the vehicle's capability to predict future behavior and deal with unexpected situations. Applying the image recognition and tracking results of the visual system, the breadth and depth of the vision are divided into three-dimensional segmentation where each block gives different weights. When the vehicle's current trajectory meets obstacles, according to the location of obstacles in the visual image, the vehicle will be reprogrammed utilizing the arcing pattern of the optimal strategy in order to bypass the obstacles and continue to follow the desired trajectory under the macro path. At the same time, to ensure the stable operation of vehicles and prevent the possible vehicle sliding and roll-over caused by a sudden involvement of the lateral acceleration, a model predictive vehicle lateral control method under the dynamic constraints is proposed. The simulation analysis is conducted to validate the effectiveness of the control strategy and the proposed model predictive controller under different vehicle operating conditions.