Hybrid Deep Learning Framework for Multi-Cancer Classification Using CNN and ACO-Optimized LSTM Networks

Abstract

Early and accurate detection of multiple cancer types plays a vital role in improving patient outcomes and enabling personalized treatment strategies. Traditional diagnostic approaches often depend on manual analysis of medical images, which can be both time-consuming and prone to inconsistencies. To overcome these limitations, this study introduces an adaptive hybrid deep learning framework that combines Convolutional Neural Networks (CNNs) with Long Short-Term Memory (LSTM) networks optimized using Ant Colony Optimization (ACO). In this approach, CNNs are used to automatically learn spatial features from histopathological and radiological images, capturing subtle patterns associated with various cancer types. ACO is then employed to optimize the hyperparameters of the LSTM model and select the most relevant features, enhancing learning efficiency and reducing overfitting. The optimized LSTM performs multi-class classification by modeling complex dependencies within the extracted features. The proposed framework is evaluated on benchmark datasets containing lung, breast, and colorectal cancer images. Experimental results demonstrate that this hybrid CNN–ACO–LSTM model significantly outperforms traditional CNN, LSTM, and CNN–LSTM models across multiple evaluation metrics, including accuracy, precision, recall, F1-score, and AUC. Overall, the integration of spatial learning, adaptive optimization, and sequential modeling makes the framework robust, scalable, and suitable for clinical decision support systems, contributing to improved diagnostic accuracy and personalized oncology care.