With the continuous integration of computer technology into agricultural production, it also reduces personnel costs while improving agricultural production efficiency and quality. Crop disease control is an important part of agricultural production, and the use of computer vision technology to quickly and accurately identify crop diseases is an important means of ensuring a good harvest of agricultural products and promoting agricultural modernization. In this paper, a recognition method based on deep learning is proposed based on soybean brown spot. The method is divided into image pre- treatment and disease identification. Based on traditional threshold segmentation, the pre-processing process first uses the HSI colour space to filter the information of the normal area of the leaf, adopts OTSU to set the threshold to segment the original image under the Lab colour space, and then merges the segmented images. The final spot segmentation image is obtained. Compared with the renderings of several other commonly used methods of segmentation, this method can better separate the lesions from the leaves. In terms of disease identification, in order to adapt to the working conditions of large samples of farmland operations, a convolutional neural network (CNN) of continuous convolutional layers was constructed with the help of Caffe to extract more advanced features of the image. In the selection of activation functions, this paper selects the Max out unit with stronger fitting ability, and in order to reduce the parameters in the network and prevent the network from overfitting, the sparse Max out unit is used, which effectively improves the performance of the Max out convolutional neural network. The experimental results show that the algorithm is superior to the algorithm based on ordinary convolutional neural network in identifying large sample crop diseases.