Learning only from a small set of examples remains a huge challenge in machine learning. Despite recent breakthroughs in the applications of neural networks, the applicability of these techniques has been limited by the requirement for large amounts of training data. What’s more, the standard supervised machine learning method does not provide a satisfactory solution for learning new concepts from little data. However, the ability to learn enough information from few samples has been demonstrated in humans. This suggests that humans may make use of prior knowledge of a previously learned model when learning new ones on a small amount of training examples. In the area of autonomous driving, the model learns to drive the vehicle with training data from humans, and most machine learning based control algorithms require training on very large datasets. Collecting and constructing training data set takes a huge amount of time and needs specific knowledge to gather relevant information. This paper aims to learn control parameters from only a few training images. We build a simple control system which can use prior knowledge to correct parameters when the vehicle deviates from the training route, and allows for learning on a few training examples. The system introduces a new architecture that when combined with neural networks, significantly lowers the amount of data required to make meaningful predictions and improves the ability to learn meaningful information on the surrounding environment in never before seen scenarios. We test a simple implementation of our algorithm on a 1/10-scale autonomous driving vehicle. The proposed models produce effective control commands in some untrained lane deviation scenarios, when the number of training examples is normally too small for traditional machine learning methods to work, and allows the vehicle to find the correct track in the lane.