Dermoscopy is an effective way to assist dermatologists' diagnosis, but artificial classification is highly dependent on the clinical experience of the doctor, and the complexity of the dermoscopy image itself poses a huge challenge to classification. In order to solve the problem of dermoscopy image classification, a dermoscopy image classification method based on voting block integration is proposed based on ensemble learning. First, the dermoscopy images provided by ISIC 2019 were pre-processed. In order to alleviate the problem of less data sets and uneven distribution, the data set is augmented by generating adversarial networks and rotating images. Then, based on the idea of transfer learning, multiple convolutional neural networks are trained, and multiple convolutional neural networks with better classification results are selected to form voting blocks. Through voting block integration, the dermoscopy image is classified. The experimental results show that the accuracy, sensitivity, and specificity of the method can reach 0.925, 0.563, and 0.927, respectively. Compared with a single convolutional neural network model, each evaluation criterion is improved, which provides an effective solution for dermoscopy image classification.