In recent years, driving distractions have emerged as a leading cause of vehicular accidents. This study aims to develop a deep learning framework that utilizes both physiological and visual signals to detect and predict drivers' distraction. We combine Convolutional Neural Networks (CNN) for extracting features from visual data such as facial expressions, eye movement, and head posture, with MobileNet, a lightweight yet effective model, to efficiently process data in real-time. Furthermore, physiological signals like heart rate and galvanic skin response are incorporated to provide a comprehensive assessment of the driver's state. Our dataset comprises synchronized visual and physiological data recorded from actual driving sessions. Results indicate a significant improvement in distraction detection accuracy over existing methods, particularly in challenging scenarios where visual cues alone are insufficient. This integrated approach holds great promise for the development of robust in-car safety systems that can alert drivers in real-time and potentially prevent countless accidents caused by distractions.