Medical image fusion is an important task in medical diagnosis that aims to provide a complete representation of medical images by combining multiple imaging modalities. The fused image is then fed into deep learning algorithms for tumor classification. In this paper, we propose an approach for medical image fusion of CT and MRI scan images for brain tumor detection. This work proposes an algorithm for the fusion of several imaging modalities, such as MRI and CT, based on a classifier with a fusion rule. Several qualitative and quantitative evaluation metrics have been used to assess the performance of the proposed method and compare it to cutting-edge image fusion techniques. On the basis of metrics like standard deviation, entropy, mutual information, etc., the experimental findings are assessed. In terms of accuracy and training loss metrics, the experimental results show that the proposed approach outperforms the individual modalities. As a result, the suggested technique can be employed as an effective and accurate instrument for the detection of brain cancers. The method can be used to increase diagnosis precision and decrease the false-negative rate, which will ultimately improve patient outcomes