[Abstract] To address the high computational cost of existing coal–gangue segmentation models for intelligent sorting systems, as well as their insufficient acquisition of multi-scale contextual information for targets with different particle sizes and inadequate utilization of multi-resolution features, a lightweight coal–gangue segmentation method termed MPF-DeepLabV3+ is proposed. Based on the DeepLabV3+ encoder–decoder architecture, the proposed method replaces the original encoder backbone with the lightweight MobileNetV4 to reduce the parameter count and computational complexity while maintaining feature extraction capability. A Parallel Aggregation Pyramid Pooling Module (PAPPM) is introduced, in which parallel branches with different pooling scales are used to enlarge the receptive field of deep features and accommodate variations in the particle sizes of coal–gangue targets. In the decoder, a Multi-scale Feature Fusion Module (MFF) is constructed to adaptively weight and fuse the multi-resolution features produced by different stages of the backbone network. A Convolutional Block Attention Module (CBAM) is subsequently incorporated to further improve segmentation accuracy. Experimental results show that MPF-DeepLabV3+ achieves a mean Intersection over Union (mIoU) of 91.70%, a mean F1 score (mFscore) of 95.67%, and a mean accuracy (mAcc) of 95.12% on the test set with a parameter count of 8.04 M, outperforming mainstream semantic segmentation models, including U-Net, PSPNet, DDRNet, and SegFormer, in segmentation accuracy. Compared with the baseline DeepLabV3+ model, MPF-DeepLabV3+ increases the mIoU by 1.30 percentage points, while reducing the parameter count and Floating-Point Operations (FLOPs) by 34.74% and 85.74%, respectively, and improving the inference speed by 36.86%. Tests conducted under conventional and dense coal–gangue distribution conditions show that the model maintains stable segmentation performance under both conditions, providing a basis for its future deployment on intelligent coal–gangue sorting terminals.