双分支原型学习与自适应特征筛选的文本行人搜索方法
DOI:
CSTR:
作者:
作者单位:

中国人民公安大学信息网络安全学院

作者简介:

通讯作者:

中图分类号:

TP391

基金项目:

中央高校基本科研业务费专项资金项目(2024JKF12),国家自然科学基金(62076246)


A Dual-Branch Prototype Learning and Adaptive Feature Screening Method for Text-Based Person Search
Author:
Affiliation:

School of Information and Network Security,People’s Public Security University of China

Fund Project:

  • 摘要
  • |
  • 图/表
  • |
  • 访问统计
  • |
  • 参考文献
  • |
  • 相似文献
  • |
  • 引证文献
  • |
  • 资源附件
  • |
  • 文章评论
    摘要:

    基于文本的行人搜索旨在根据文本描述从全景图像中定位目标行人,这要求模型兼具全局语义理解与局部细节匹配。现有方法多采用CLIP单一视觉编码器实现跨模态语义对齐,其自注意力机制具有强大的语义对齐性能,但多层连续交互倾向于将局部判别性信号逐步平滑化。为此,提出一种双分支原型学习与自适应特征筛选方法(Dual-Branch Prototype Learning and Adaptive Feature Screening,DPLAS),其中全局分支继承ViT的跨模态语义对齐能力,维持图文全局匹配;局部分支引入ResNet-50,其逐阶段下采样生成的结构化特征图为部件聚类提供了确切的空间定位,残差恒等映射则在深层网络中持续保存未被全局分支编码的底层细节信息,最终输出具有空间位置信息的像素特征序列。然后通过空间软约束与多样性正则化将其自适应聚类为部件级语义原型,并以文本查询为条件对各原型进行动态权重筛选,抑制聚类过程中的语义坍塌与背景噪声。在三个基准数据集上,所提方法在CUHK-PEDES取得75.63%的最高Rank-1准确率,较基线方法分别提升2.25%、2.52%和1.90%,验证了所提方法对提升细粒度跨模态对齐精度的有效性。

    Abstract:

    Text-based person search aims to localize target pedestrians from full-scene images based on natural language descriptions, which requires models to simultaneously possess global semantic understanding and local detail matching capabilities. Existing methods predominantly employ a single visual encoder, such as CLIP, to achieve cross-modal semantic alignment. While its self-attention mechanism exhibits powerful semantic alignment performance, the continuous multi-layer interactions tend to progressively smooth out local discriminative signals. To address this issue, a Dual-Branch Prototype Learning and Adaptive Feature Screening (DPLAS) method is proposed. Specifically, the global branch inherits the cross-modal semantic alignment capability of the Vision Transformer (ViT) to maintain robust global image-text matching. Concurrently, the local branch introduces ResNet-50, where the structured feature maps generated by its stage-wise downsampling provide precise spatial localization for part clustering. Meanwhile, the residual identity mapping continuously preserves the underlying detailed information unencoded by the global branch in the deep network, ultimately outputting a sequence of pixel features embedded with explicit spatial position information. These discrete features are then adaptively clustered into part-level semantic prototypes under the guidance of spatial soft constraints and diversity regularization. Furthermore, dynamic weight screening is applied to each prototype conditioned on the text query, effectively suppressing semantic collapse during the clustering process and filtering out background noise. Experimental results on three benchmark datasets demonstrate that the proposed method achieves a top Rank-1 accuracy of 75.63% on CUHK-PEDES, with improvements of 2.25%, 2.52%, and 1.90% over the baseline method, respectively, thereby validating its effectiveness in enhancing the accuracy of fine-grained cross-modal alignment.

    参考文献
    相似文献
    引证文献
引用本文

张茂恒,王蓉,许成杰,等. 双分支原型学习与自适应特征筛选的文本行人搜索方法[J]. 科学技术与工程, , ():

复制
分享
相关视频

文章指标
  • 点击次数:
  • 下载次数:
  • HTML阅读次数:
  • 引用次数:
历史
  • 收稿日期:2026-06-04
  • 最后修改日期:2026-09-08
  • 录用日期:2026-10-26
  • 在线发布日期:
  • 出版日期:
×
关于《科学技术与工程》全新官方网站正式上线的通知公告
关于抵制伪造参考文献学术不端行为的声明
关于规范使用生成式人工智能的声明
亟待确认版面费归属稿件,敬请作者关注