Wenqi Pei 裴文祺

I am an MPhil (master by research) student at The University of Hong Kong, supervised by Prof. Hongyang Du at NICE Lab. Prior to this, I earned my Bachelor's degree at the National University of Singapore, where I completed my capstone under the supervision of Prof. Bingsheng He. My research interests include Data Agents, LLM Evaluation, Multimodal Models, and Efficient AI. I am open to collaborating with people who share similar research interests.

I am currently seeking PhD opportunities for Fall 2027. Please feel free to reach out if you see a potential research fit.


Education and Awards
  • The University of Hong Kong
    The University of Hong Kong
    Sep. 2025 - Present
    Electrical and Computer Engineering
    Master of Philosophy
  • National University of Singapore
    National University of Singapore
    Aug. 2021 - Jul. 2025
    Computer Science
    Bachelor of Computing (Honours), Minor: Economics
  • Huazhong University of Science and Technology
    Huazhong University of Science and Technology
    Sep. 2020 - Jul. 2021
    Artificial Intelligence (dropped out)
  • Dean's List
    NUS SEM2 24-25
  • Top Student for CS Research Methodology
    NUS SEM1 24-25
Intern Experience
News
2026
Our survey on efficient tool-using LLM agents accepted to EACL Findings 🎉
Nov 13
TempCloze accepted to EMNLP Findings 🎉
Aug 23
Released our survey on efficient tool-using LLM agents 📚
Aug 06
ROSE accepted to ACL Main 🎉
Apr 04
2025
NL2SQLBench accepted to VLDB 🎉
Dec 16
Feather-SQL accepted to IJCNLP-AACL Findings 🎉
Oct 25
Started graduate study at HKU 🎓
Sep 01
InterFeedback accepted to EMNLP Findings 🎉
Aug 20
Self-Monitoring Large Language Models accepted to ACM TOIS 🎉
Aug 17
Graduated from NUS with BComp (Honours) 🎓
Jul 15
Selected Publications (view all )
MEM-SQL: Memory-Empowered Text-to-SQL
MEM-SQL: Memory-Empowered Text-to-SQL
Data Agents Efficient AI

Xiaoqi Wang*, Wenqi Pei*‡, Shizheng Hou*, Yilai Liu, Han Chen, Boyan Li, Yuyu Luo, Hongyang Du (* equal contribution, ‡ project lead)

Preprint 2026

We introduce MEM-SQL, which reuses verified query templates, SQL components, and schema mappings from query history and consults the database schema when memory provides insufficient evidence. On HARE, our benchmark for historical SQL reuse, MEM-SQL raised mean execution accuracy from 37.1% to 50.0% across six language models while reducing input tokens by 76.6% and latency by 68.8% on average relative to direct generation without history.

MEM-SQL: Memory-Empowered Text-to-SQL
Data Agents Efficient AI

Xiaoqi Wang*, Wenqi Pei*‡, Shizheng Hou*, Yilai Liu, Han Chen, Boyan Li, Yuyu Luo, Hongyang Du (* equal contribution, ‡ project lead)

Preprint 2026

We introduce MEM-SQL, which reuses verified query templates, SQL components, and schema mappings from query history and consults the database schema when memory provides insufficient evidence. On HARE, our benchmark for historical SQL reuse, MEM-SQL raised mean execution accuracy from 37.1% to 50.0% across six language models while reducing input tokens by 76.6% and latency by 68.8% on average relative to direct generation without history.

HALO: Human-Grounded Granular Evaluation of Proactive Agents
HALO: Human-Grounded Granular Evaluation of Proactive Agents
LLM Evaluation

In collaboration with 2077AI

Wenqi Pei, Xinyu Wang, Jinrui Liu, Cynthia Zhuge, Jie Sheng, Zihan Wang, Tenghao Huang, Muhao Chen

Preprint 2026

We introduce HALO, a human-grounded framework for evaluating whether, when, and how far proactive agents should assist. HALO-Arena spans four proactive need types across ten everyday domains, while HALO-8K contains 8,000 responses from ten models annotated by twenty experts. Granular human judgments of timing, intensity, and acceptability reveal over-proactivity as the dominant observed failure mode.

HALO: Human-Grounded Granular Evaluation of Proactive Agents
LLM Evaluation

In collaboration with 2077AI

Wenqi Pei, Xinyu Wang, Jinrui Liu, Cynthia Zhuge, Jie Sheng, Zihan Wang, Tenghao Huang, Muhao Chen

Preprint 2026

We introduce HALO, a human-grounded framework for evaluating whether, when, and how far proactive agents should assist. HALO-Arena spans four proactive need types across ten everyday domains, while HALO-8K contains 8,000 responses from ten models annotated by twenty experts. Granular human judgments of timing, intensity, and acceptability reveal over-proactivity as the dominant observed failure mode.

PreCoG: Preview-Edit for Efficient Video Correction in Generation-Time
PreCoG: Preview-Edit for Efficient Video Correction in Generation-Time
Multimodal Models Efficient AI

Yilai Liu, Wenqi Pei, Zhifeng Lu, Shiyuan Zhang, Zongxi Liu, Hongyang Du

Preprint 2026

We introduce PreCoG, a framework that diagnoses semantic failures in low-cost video previews and corrects cached generation trajectories before final rendering. By combining PreQA assessment with PreEdit correction, it reduces redundant computation while preserving already correct content, achieving strong video quality and alignment at lower inference cost.

PreCoG: Preview-Edit for Efficient Video Correction in Generation-Time
Multimodal Models Efficient AI

Yilai Liu, Wenqi Pei, Zhifeng Lu, Shiyuan Zhang, Zongxi Liu, Hongyang Du

Preprint 2026

We introduce PreCoG, a framework that diagnoses semantic failures in low-cost video previews and corrects cached generation trajectories before final rendering. By combining PreQA assessment with PreEdit correction, it reduces redundant computation while preserving already correct content, achieving strong video quality and alignment at lower inference cost.

TempCloze: Can Video-LLMs Identify the Missing Middle?
TempCloze: Can Video-LLMs Identify the Missing Middle?
Multimodal Models LLM Evaluation

Wenqi Pei*, Hengyuan Zhao*, Yilai Liu*, Jiahao Meng, Han Chen, Ziyu Wang, Hongyang Du (* equal contribution)

EMNLP Findings 2026

We introduce TempCloze, a video cloze benchmark for evaluating visual temporal reasoning in Video-LLMs by asking them to identify the true missing middle between beginning and ending clips. Across 1,521 videos and 31 Video-LLMs, TempCloze reveals temporal alignment as the primary bottleneck.

TempCloze: Can Video-LLMs Identify the Missing Middle?
Multimodal Models LLM Evaluation

Wenqi Pei*, Hengyuan Zhao*, Yilai Liu*, Jiahao Meng, Han Chen, Ziyu Wang, Hongyang Du (* equal contribution)

EMNLP Findings 2026

We introduce TempCloze, a video cloze benchmark for evaluating visual temporal reasoning in Video-LLMs by asking them to identify the true missing middle between beginning and ending clips. Across 1,521 videos and 31 Video-LLMs, TempCloze reveals temporal alignment as the primary bottleneck.

How to Make Tool-Using LLM Agents Efficient? A Survey
How to Make Tool-Using LLM Agents Efficient? A Survey
Efficient AI

Yunuo Hu*, Ziyang Cheng*, Wenqi Pei*‡, Miaomiao Li, Boyan Li, Yizhang Zhu, Yupeng Xie, Han Chen, Shizheng Hou, Yuyu Luo, Hongyang Du (* equal contribution, ‡ project lead)

EACL Findings 2027

We present a systematic survey and unified taxonomy of efficient tool-using LLM agents, organizing techniques across tool context, calling, reasoning, and execution. We synthesize shared mechanisms, limitations, open challenges, and future directions for reducing resource use while preserving task performance.

How to Make Tool-Using LLM Agents Efficient? A Survey
Efficient AI

Yunuo Hu*, Ziyang Cheng*, Wenqi Pei*‡, Miaomiao Li, Boyan Li, Yizhang Zhu, Yupeng Xie, Han Chen, Shizheng Hou, Yuyu Luo, Hongyang Du (* equal contribution, ‡ project lead)

EACL Findings 2027

We present a systematic survey and unified taxonomy of efficient tool-using LLM agents, organizing techniques across tool context, calling, reasoning, and execution. We synthesize shared mechanisms, limitations, open challenges, and future directions for reducing resource use while preserving task performance.

ROSE: An Intent-Centered Evaluation Metric for NL2SQL
ROSE: An Intent-Centered Evaluation Metric for NL2SQL
Data Agents LLM Evaluation

Wenqi Pei*, Shizheng Hou*, Boyan Li*, Han Chen, Zhichao Shi, Yuyu Luo (* equal contribution)

ACL Main 2026

We introduce ROSE, an intent-centered evaluation metric for NL2SQL that uses an adversarial Prover-Refuter cascade to judge whether predicted SQL captures a user's intent. ROSE aligns more closely with expert assessments and supports a large-scale re-evaluation of 19 NL2SQL methods.

ROSE: An Intent-Centered Evaluation Metric for NL2SQL
Data Agents LLM Evaluation

Wenqi Pei*, Shizheng Hou*, Boyan Li*, Han Chen, Zhichao Shi, Yuyu Luo (* equal contribution)

ACL Main 2026

We introduce ROSE, an intent-centered evaluation metric for NL2SQL that uses an adversarial Prover-Refuter cascade to judge whether predicted SQL captures a user's intent. ROSE aligns more closely with expert assessments and supports a large-scale re-evaluation of 19 NL2SQL methods.

NL2SQLBench: A Modular Benchmarking Framework for LLM-Enabled NL2SQL Solutions
NL2SQLBench: A Modular Benchmarking Framework for LLM-Enabled NL2SQL Solutions
Data Agents LLM Evaluation

Shizheng Hou, Wenqi Pei, Nuo Chen, Quang-Trung Ta, Peng Lu, Beng Chin Ooi

VLDB 2026

We present NL2SQLBench, the first modular benchmarking framework for LLM-enabled NL2SQL approaches. We dissect NL2SQL systems into three modules: Schema Selection, Candidate Generation, and Query Revision. We review existing strategies and propose fine-grained metrics that systematically quantify module-level effectiveness and efficiency.

NL2SQLBench: A Modular Benchmarking Framework for LLM-Enabled NL2SQL Solutions
Data Agents LLM Evaluation

Shizheng Hou, Wenqi Pei, Nuo Chen, Quang-Trung Ta, Peng Lu, Beng Chin Ooi

VLDB 2026

We present NL2SQLBench, the first modular benchmarking framework for LLM-enabled NL2SQL approaches. We dissect NL2SQL systems into three modules: Schema Selection, Candidate Generation, and Query Revision. We review existing strategies and propose fine-grained metrics that systematically quantify module-level effectiveness and efficiency.

InterFeedback: Unveiling Interactive Intelligence of Large Multimodal Models with Human Feedback
InterFeedback: Unveiling Interactive Intelligence of Large Multimodal Models with Human Feedback
Multimodal Models LLM Evaluation

Hengyuan Zhao*, Wenqi Pei*, Yifei Tao*, Haiyang Mei, Mike Zheng Shou (* equal contribution)

EMNLP Findings and Bi-Align @ ICLR (Oral) 2025

This work introduces InterFeedback, a novel approach to understanding and enhancing the interactive intelligence of large multimodal models through human feedback mechanisms. Our framework provides new insights into model behavior and interaction patterns.

InterFeedback: Unveiling Interactive Intelligence of Large Multimodal Models with Human Feedback
Multimodal Models LLM Evaluation

Hengyuan Zhao*, Wenqi Pei*, Yifei Tao*, Haiyang Mei, Mike Zheng Shou (* equal contribution)

EMNLP Findings and Bi-Align @ ICLR (Oral) 2025

This work introduces InterFeedback, a novel approach to understanding and enhancing the interactive intelligence of large multimodal models through human feedback mechanisms. Our framework provides new insights into model behavior and interaction patterns.

Feather-SQL: A Lightweight NL2SQL Framework with Dual-Model Collaboration Paradigm for Small Language Models
Feather-SQL: A Lightweight NL2SQL Framework with Dual-Model Collaboration Paradigm for Small Language Models
Data Agents Efficient AI

In collaboration with 4Paradigm

Wenqi Pei, Hailing Xu, Hengyuan Zhao, Shizheng Hou, Han Chen, Zining Zhang, Pingyi Luo, Bingsheng He

IJCNLP-AACL Findings and DL4C @ ICLR 2025

We present Feather-SQL, a novel framework for Natural Language to Structured Query Language (NL2SQL) that leverages dual-model collaboration paradigm specifically designed for small language models. Our approach achieves state-of-the-art (SOTA) results on the BIRD benchmark.

Feather-SQL: A Lightweight NL2SQL Framework with Dual-Model Collaboration Paradigm for Small Language Models
Data Agents Efficient AI

In collaboration with 4Paradigm

Wenqi Pei, Hailing Xu, Hengyuan Zhao, Shizheng Hou, Han Chen, Zining Zhang, Pingyi Luo, Bingsheng He

IJCNLP-AACL Findings and DL4C @ ICLR 2025

We present Feather-SQL, a novel framework for Natural Language to Structured Query Language (NL2SQL) that leverages dual-model collaboration paradigm specifically designed for small language models. Our approach achieves state-of-the-art (SOTA) results on the BIRD benchmark.

All publications