To realize intelligent cancer diagnosis for pancreatic images returned by endoscopic ultrasonography, the classic classification and recognition network AlexNet and SE attention mechanism are combined with in this paper. The proposed SE-AlexNet network can accurately detect cancerous images and discriminate the pancreatic parts to which the normal images belong. In order to verify the reliability and superiority of the algorithm, a series of comparison experiments are carried out under the self-made dataset in the paper, including the model comparison experiments between the basic network AlexNet and other classical classification networks, and the comparison experiments of various models improved and obtained by inserting different attention mechanism in different positions. The results show that the overall precision and recall rate of SE-AlexNet model can reach 99.56% and 98.69%. For cancerous images, the detection precision and recall rate are 100%, which can achieve an effective auxiliary diagnosis for practical endoscopic ultrasonography of pancreatic cancer.