The use of smart portable devices in vehicles creates the possibility to record useful data and helps develop a better understanding of driving behavior. In the past few years the UTDrive mobile App (a.k.a MobileUTDrive) has been developed with the goal of improving driver/passenger safety, while simultaneously maintaining the ability to establish monitoring techniques that can be used on mobile devices on various vehicles. In this study, we extend the ability of MobileUTDrive to understand the impact on driver performance on public roads in the presence of distraction from speech/voice based tasks versus tactile/hands-on tasks. Drivers are asked to interact with the device in both voice-based and hands-on modalities and their reaction time and comfort level are logged. To evaluate the driving patterns while handling the device by speech/hand, the signals from device inertial sensors are retrieved and used to construct Gaussian Mixture Models (GMM). The GMM variations indicate the differences between driving with and without operating devices. A probe experimental sample set (5 drivers) shows a significantly reduced degradation in driving performance during speech/voice interactions versus tactile/hands-on tasks. With the proliferation of autonomous systems and artificial intelligent systems in the automotive space, this study provides an insight into driver/passenger comfort and confidence in adjusting to these changes.