Semantic segmentation has become a fundamental topic in the field of the computer vision, whose goal is to assign each pixel in the image to the corresponding category label. This topic is of broad interest for potential applications of automatic driving. Recently, modern frameworks of semantic segmentation are mostly based on the deep convolutional neural networks. And the general trend focus on increasing the accuracy of the framework, but at the cost of bringing extra parameters and making the network more complicated, which makes the network hard to implement on the vehicle mobile and embedded devices with limited computational resources. In this paper, a novel architecture is developed based on Inverted Residual and Atrous Convolution, in the sense that not only computation cost can be drastically reduced, but also high accuracy can still be maintained. In addition, two simple global hyper-parameters for seeking a tradeoff between the accuracy and latency are introduced to build a model with appropriate size, which can operate in a computational limited platform. The experiments are performed on challenging CityScapes dataset and CamVid dataset. And the results are presented to demonstrate the good performance of the proposed architecture, in comparison with existing state-of-the-art methods. Furthermore, extensive experiments on the tradeoff between the resource and accuracy are also carried out. The results indicate that the model with appropriate size can be obtained by the choice of the two global hyper-parameters, which can be easily matched to the design requirements for mobile vision applications.