Huanzhi (Hans) Mao 冒峘志

Ph.D. student, Sky Computing Lab, UC Berkeley
Advised by Joseph E. Gonzalez and Alexandros G. Dimakis

Portrait of Huanzhi (Hans) Mao

I work on LLM agents, with a focus on tool use: how to measure what agents can actually do, and what it takes to make them reliable enough to hand real tools.

I lead the Berkeley Function Calling Leaderboard (BFCL), the de facto benchmark for evaluating function calling in language models. It began as an AST-based evaluation of single-turn calls and now spans multi-turn stateful execution and agentic settings — web search, memory management, and format sensitivity. BFCL is developed in the open as part of the Gorilla project (13k+ stars) and ships as the bfcl-eval package.

I spent the summer of 2025 at Salesforce AI Research, where I built BFCL Audio, an evaluation for audio-native function calling. Before the Ph.D. I studied Computer Science (Honors) and Data Science at Berkeley, and served seven semesters as Head TA for CS 61A, Structure and Interpretation of Computer Programs.

Publications Google Scholar →

* Equal contribution unless otherwise noted.

Selected Publications

  1. EasyPPO: Stabilizing the Critic Is Key

    Xuanyi Zhou*, Qiuyang Mang*, Huanzhi Mao*, Dacheng Li, Wenhao Chai, Mayank Mishra, Yichuan Wang, Karthik Narasimhan, Alvin Cheung, Joseph E. Gonzalez

    arXiv preprint

    Stabilizes PPO for LLM reinforcement learning through actor-only overlong filtering, noise-normalized critic regression, and smaller critic mini-batches.

  2. SlackBench: Benchmarking Agents on Collaborative Projects Grounded in Real Code Repositories

    Mingxuan Li*, Huanzhi Mao*, Ruitao Zheng, Jarod Stanbury James, Yunze Li, Natanya Anderson, Joseph E. Gonzalez, Alexandros G. Dimakis

    NeurIPS 2026 Evaluations & Datasets Track

  3. I-PTC: Interactive Programmatic Tool Calling for Stateful Tool-Augmented Agents

    Huanzhi Mao, Chengkun Cao, Shuo Yuan, Joseph E. Gonzalez

    NeurIPS 2026

  4. BFCL Audio: An Audio Function Calling Evaluation for Large Language Models

    Huanzhi Mao, Aditya Ghai, Imra Dawoodani, Tony Ginart, Shishir G. Patil, John Emmons, Joseph E. Gonzalez

    ICML 2026

    Extends BFCL to spoken input with 6.2K expert-verified audio tool-use tasks, covering both pipelined ASR-to-LLM and end-to-end audio-in settings under controlled acoustic perturbations.

  5. FrontierCS: Evolving Challenges for Evolving Intelligence

    Qiuyang Mang*, Wenhao Chai*, Zhifei Li*, Huanzhi Mao*, Shang Zhou*, Alexander Du*, Hanchen Li*, Shu Liu*, et al.

    ICML 2026

  6. The Berkeley Function Calling Leaderboard (BFCL): From Tool Use to Agentic Evaluation of Large Language Models

    Shishir G. Patil, Huanzhi Mao, Fanjia Yan, Charlie Cheng-Jie Ji, Vishnu Suresh, Ion Stoica, Joseph E. Gonzalez

    ICML 2025

    The benchmark I lead. Evaluates LLM function calling via scalable abstract-syntax-tree matching across single-turn, crowdsourced live, multi-turn, and stateful agentic settings.

  7. MFCL Vision: Benchmarking Tool Use in Multimodal Large Language Models for Visual Reasoning Tasks

    Huanzhi Mao, Jad Bendarkawi, Evan Maxwell Turner, Ritesh Sunil Chavan

    NeurIPS 2025 NORA Workshop, Spotlight

    A benchmark of 250 expert-verified tasks across five image domains that measures how reliably multimodal LLMs turn visual input into correct tool calls.

Other Publications (12)

  1. CodeWalk: Generating Coding Benchmarks by Walking a Problem Graph

    Jiaxin Fang, Runyuan He, Arnav Gupta, Sahil Kumawat, Huanzhi Mao, Alvin Cheung

    arXiv preprint

    Generates coding benchmarks by programmatically composing verified problems, without language-model generation or validation.

  2. FrontierSmith: Synthesizing Open-Ended Coding Problems at Scale

    Runyuan He*, Qiuyang Mang*, Shang Zhou, Kaiyuan Liu, Hanchen Li, Huanzhi Mao, Qizheng Zhang, Zerui Li, et al.

    NeurIPS 2026 Spotlight

  3. Assertion-Conditioned Compliance: A Provenance-Aware Vulnerability in Multi-Turn Tool-Calling Agents

    Daud Waqas*, Aaryamaan Golthi, Erika Hayashida, Huanzhi Mao*

    EACL 2026 Industry Track; * Project leads

    Measures how multi-turn tool-calling agents respond to misleading assertions from users and from tool outputs. I advised this work and co-led the project.

  4. Characterizing Agents in Production

    Melissa Z. Pan*, Negar Arabzadeh*, Riccardo Cogo, Yuxuan Zhu, Alexander Xiong, Lakshya A. Agrawal, Huanzhi Mao, Emma Shen, et al.

    ICML 2026 Oral

  5. AutoCode: LLMs as Problem Setters for Competitive Programming

    Shang Zhou*, Zihan Zheng*, Kaiyuan Liu*, Zeyu Shen*, Zerui Cheng*, Zexing Chen, Hansen He, Jianzhu Yao, Huanzhi Mao, Qiuyang Mang, et al.

    ICLR 2026

  6. SVG-EAR: Parameter-Free Linear Compensation for Sparse Video Generation via Error-aware Routing

    Xuanyi Zhou*, Qiuyang Mang*, Shuo Yang, Haocheng Xi, Jintao Zhang, Huanzhi Mao, Joseph E. Gonzalez, Kurt Keutzer, Ion Stoica, Alvin Cheung

    ECCV 2026

  7. DeployBench: Benchmarking LLM Agents for Research Artifact Deployment

    Yuanli Wang, Yaoyao Qian, Yue Zhang, Hanhan Zhou, Jindan Huang, Tianfu Fu, Qiuyang Mang, Huanzhi Mao, Wenhao Chai, Wendong Fan, Liqiang Jing

    EMNLP 2026 Findings

  8. Continuum: Efficient and Robust Multi-Turn LLM Agent Scheduling with KV Cache Time-to-Live

    Hanchen Li*, Runyuan He*, Qiuyang Mang*, Qizheng Zhang, Huanzhi Mao, Xiaokun Chen, Hangrui Zhou, Huanchen Zhang, Alvin Cheung, Joseph Gonzalez, Ion Stoica

    ICLR 2026 LLA Workshop

  9. CFLBench: Benchmarking Novel Control Flow Language Learning

    Aaroosh Rustagi, Jounghyuck Sohn, Thomas Peng, Mykaala Firdaus, Huanzhi Mao

    ICLR 2026 Workshop

    A benchmark of 19 small deterministic programming languages that tests whether tool-augmented language models can learn unfamiliar control flow. I advised this work.

  10. Concurrency without Model Changes: Future-based Asynchronous Function Calling for LLMs

    Guangyu Feng, Huanzhi Mao, Prabal Dutta, Joseph E. Gonzalez

    arXiv preprint

    An execution-layer framework that overlaps model decoding with tool execution by reasoning over symbolic futures, with no fine-tuning and no change to the function-calling protocol.

  11. OpenDeepThink: Parallel Reasoning via Bradley-Terry Aggregation

    Shang Zhou, Wenhao Chai, Kaiyuan Liu, Huanzhi Mao, Qiuyang Mang, Jingbo Shang

    arXiv preprint

  12. The Danger of Overthinking: Examining the Reasoning-Action Dilemma in Agentic Tasks

    Alejandro Cuadron, Dacheng Li, Wenjie Ma, Xingyao Wang, Yichuan Wang, Siyuan Zhuang, Shu Liu, Luis Gaspar Schroeder, Tian Xia, Huanzhi Mao, Nicholas Thumiger, et al.

    arXiv preprint

Projects

  • Berkeley Function Calling Leaderboard (BFCL)

    Project lead

    The de facto benchmark for evaluating function calling in language models. BFCL has grown from AST-based evaluation of single-turn calls (v1) through user-contributed live data (v2) and multi-turn stateful execution (v3) to agentic evaluation of web search, memory management, and format sensitivity (v4). I run the releases, dataset, evaluation harness, and model onboarding.

  • Gorilla

    Top contributor

    The Sky Computing Lab's open-source project for teaching and evaluating language models on API and tool calls — the original Gorilla model and APIBench, OpenFunctions, GoEx, RAFT, the API Zoo, and BFCL. 13k+ stars.

Past Students

Students I have mentored and their next steps.

Beyond Research

I sail, hike, and play more board games than I can justify. In 2017 I was the national champion in the Chinese Yachting Association Youth Sailing League.

If any of this overlaps with what you are working on, I would be glad to hear from you — email is the fastest way to reach me.