Jianshu Zhang

Jianshu Zhang

Senior @ WHU 🖐️ | Incoming CS PhD @ NU 💜

🧑‍💻 About Me

Greetings!👋 I’m Jianshu Zhang (张鉴殊), a 4th year undergraduate at the Wuhan University and an incoming CS PhD at the Northwestern University, co-supervised by Professor Han Liu and Professor Manling Li.

My long-term goal is to enhance the interaction between humans and AI by focusing on three key attributes: (i) reliability, (ii) multimodality, and (iii) adaptability. Specifically, my primary focus will be on bridging the gap between a model’s linguistic capabilities and its proficiency in other domains (especially in vision/action). Feel free to connect and discuss exciting opportunities or collaborations! (wechat)

🔥 News

  • 2025.01:   1 paper (PVIT) has been accepted by ICLR 2025. ✊
  • 2024.11:   Dataset of PVIT-3M made it to the Top 3 Dataset downloads [November 2024] at HuggingFace🤗! （Post from X）
  • 2024.10:   I won the National Scholarship (Award Rate: 0.2% national-wide)!
  • 2024.09:   1 paper (Image Textualization) has been accepted by NeurIPS 2024 (D&B track).
  • 2024.09:   2 papers (MLLM-Protector, FIRST) have been accepted by EMNLP 2024 (Main).
  • 2024.03:   1 paper (CORE) has been accepted by Cogsci 2024 [Oral].
  • 2024.01:   FuzzLLM has been invited to present at one of the top hacker conventions – ShmooCon 2024!
  • 2023.12:   1 paper (FuzzLLM) has been accepted by ICASSP 2024.


📝 Publications

† : equal contribution (names listed in no particular order), * : corresponding author

VLM$^2$-Bench: A Closer Look at How Well VLMs Implicitly Link Explicit Matching Visual Cues
VLM$^2$-Bench: A Closer Look at How Well VLMs Implicitly Link Explicit Matching Visual Cues
Jianshu Zhang† , Dongyu Yao†, Renjie Pi, Paul Pu Liang, Yi R. (May) Fung*
Under Review
[arXiv] [project page]


Image Textualization
Image Textualization: An Automatic Framework for Generating Rich and Detailed Image Descriptions
Renjie Pi†, Jianshu Zhang† , Jipeng Zhang, Rui Pan, Zhekai Chen, Tong Zhang*
The Thirty-Eighth Annual Conference on Neural Information Processing Systems (NeurIPS 2024)
[arXiv] [code]
Personalized Visual Instruction Tuning
Personalized Visual Instruction Tuning
Renjie Pi†, Jianshu Zhang† , Tianyang Han, Jipeng Zhang, Rui Pan, Tong Zhang*
The Thirteenth International Conference on Learning Representations (ICLR 2025)
[arXiv] [code]



MLLM-Protector
MLLM-Protector: Ensuring MLLM's Safety without Hurting Performance
Renjie Pi†, Tianyang Han†, Jianshu Zhang† , Yueqi XIE, Rui Pan, Qing LIAN, Hanze Dong, Jipeng Zhang, Tong Zhang*
The 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP 2024)
[arXiv] [code]
FIRST
FIRST: Teach A Reliable Large Language Model Through Efficient Trustworthy Distillation
KaShun SHUM†, Minrui Xu†, Jianshu Zhang† , Zixin CHEN, Shizhe Diao, Hanze Dong, Jipeng Zhang, Muhammad Omer Raza
The 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP 2024)
[arXiv] [code]
FuzzLLM
FuzzLLM: A Novel and Universal Fuzzing Framework for Proactively Discovering Jailbreak Vulnerabilities in Large Language Models
Dongyu Yao†, Jianshu Zhang† , Ian G. Harris*, Marcel Carlsson
2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP 2024)
[arXiv] [code]


CORE
CORE: Mitigating Catastrophic Forgetting in Continual Learning through Cognitive Replay
Jianshu Zhang† , Yankai Fu†, Ziheng Peng†, Dongyu Yao, Kun He*
2024 Cognitive Science Society Annual Conference (Cogsci 2024) [Oral]
[arXiv] [code]
Client As Navigator
CAN: Injecting More "Federation" into Federated Continual Learning by Positioning Clients as Navigators
Jianshu Zhang† , Xuankun Rong†, Kun He, Mang Ye*
Under Review
[arXiv] [code]
Bridge-Coder
Bridge-Coder: Unlocking LLMs' Potential to Overcome Language Gaps in Low-Resource Code
Jipeng Zhang†, Jianshu Zhang† , Yuanzhe LI†, Renjie Pi, Rui Pan, Runtao Liu, Zheng Ziqiang, Tong Zhang*
Under Review
[arXiv] [code]




🏆 Awards

  • National Scholarship
  • Merit Student
  • First Class Scholarship

📖 Educations

💻 Internships