API-Bank

AI界的“工具百宝箱”测试：API-Bank是什么？

在人工智能（AI）的飞速发展时代，大型语言模型（LLMs），比如我们熟知的ChatGPT背后的技术，已经变得越来越聪明。它们能写诗、编故事、翻译语言，甚至进行复杂的编程。但这些“超级大脑”也有自己的局限性——它们主要擅长处理语言和知识，对于现实世界的“操作”和“计算”往往力有不逮。这就引出了一个关键的概念：API-Bank。

要理解API-Bank，我们得先从几个日常概念说起。

1. 什么是API？——程序的“接口”或“插座”

想象一下，你家里有各种电器：电饭煲、电视机、洗衣机。每个电器都有一个插头，而墙上有很多插座。通过把插头插入正确的插座，电器就能获得电力并开始工作。

在计算机的世界里，API (Application Programming Interface) 就像是程序之间的“插座”和“插头”。它定义了一套规则和方法，让不同的软件能够相互交流、交换数据，并请求对方完成特定任务。

例如，一个天气预报App可能通过调用某个天气数据服务商的API，来获取实时的天气信息并显示给你。App自己不需要去测量气温、风速，它只需要知道如何“插上”天气API这个插座，就能得到想要的数据。

2. 大型语言模型 (LLM) — 善于“动脑”的智能助手

现在，让我们把视线转向AI领域的核心——大型语言模型（LLM）。你可以M把LLM想象成一个学富五车、能言善辩的超级学者。它阅读了人类几乎所有的文字资料，因此对知识的理解和语言的运用达到了前所未有的高度。你可以向它提问，让它创作，甚至帮它出谋划策，它都能给出令人惊艳的回答。

然而，这位超级学者也有它的软肋。如果要求它：

  • “帮我预订今晚8点去北京的机票。”
  • “查询一下我银行账户里还剩多少钱？”
  • “帮我计算这堆复杂数据的平均值。”

这些任务超出了它纯粹的“语言和知识”范畴，而是需要“实际操作”或“精确计算”的能力。这就是LLM们需要“工具”帮助的地方。

3. LLM的“工具使用”——从“动脑”到“动手”

当我们的超级学者无法独立完成某些任务时，它就需要学会如何借助外部的“专业工具”。这些“工具”就是前文提到的各种API。

  • 预订机票？它需要调用“机票预订API”。
  • 查询银行余额？它需要调用“银行查询API”。
  • 执行复杂计算？它需要调用“计算器API”或“数据分析API”。

一个真正智能的AI，不仅仅要知识渊博，还要学会像人类一样，在需要时识别并使用合适的工具来解决问题。这种能力，在AI领域被称为**“工具使用”（Tool-Use）**。

4. API-Bank：评估LLM“工具使用”能力的“驾驶执照考试”

现在，终于轮到我们的主角出场了：API-Bank。

API-Bank并非一个实际的“银行”或“应用”，而是一个专门为评估大型语言模型（LLMs）如何使用外部工具（API）而设计的综合性测试基准。你可以把它想象成一个为智能助手准备的“驾驶执照考试”或“工具技能考核”。

想象一下，我们把这位懂得语言的超级学者带到一个拥有各种工具的巨大“车间”。这个车间里有53到73个常用API工具，比如日历API、天气API、地图API、购物API，甚至还有更复杂的数据库查询API等等。API-Bank的设计目的就是，要看看这个超级学者在面临一项任务时，能否：

  1. 理解任务： 准确判断需要解决的问题。
  2. 规划步骤： 思考解决问题需要哪些步骤。
  3. 选择工具： 从琳琅满目的工具中，挑选出最合适的一个或几个API。
  4. 正确调用： 按照API的使用说明，向API发出正确的指令，并提供正确的参数（就像把插头插进正确的插座，并按下正确的按钮）。
  5. 处理结果： 理解API返回的结果，并用它来完成任务或进行下一步的决策。

API-Bank通过模拟真实对话情境，设计了大量的测试题目，让LLM在这些场景中“实战”运用API。例如，给它一个请求：“帮我把下周二的会议日程添加到我的日历，会议主题是‘项目回顾’，地点在会议室A。”LLM就需要判断这需要“日历API”，然后提取出日期、主题、地点等信息，并用正确的格式调用API，完成添加日程的操作。

5. 为什么API-Bank如此重要？

API-Bank的出现，对于AI领域具有里程碑式的意义。

  • 推动LLM发展： 它为研究人员提供了一个标准化的“考场”，可以系统地衡量不同LLM在工具使用方面的优缺点。通过分析LLM在API-Bank上的表现，可以发现其不足之处，从而指导如何改进模型，让它们更好地学会“动手”操作。
  • 弥合真实世界与AI的差距： 仅仅能“说会道”的AI是不够的，如果AI能够自如地调用外部工具，它就能更好地与现实世界互动，完成更复杂的任务，比如智能家居控制、个人日程管理、自动化数据分析等。
  • 加速AI应用落地： 随着LLM工具使用能力的提升，未来的AI应用将更加强大和灵活。开发者可以更方便地将各种AI模型整合到一起，创造出更多创新性的产品和服务。

举个例子，微软的Azure API Management就提供了AI网关的功能，帮助企业管理和保护AI服务，让AI模型能够更安全、高效地使用和提供不同API能力。Postman等API平台也开始强调“AI-ready APIs”，确保API能够被AI Agent更好地理解和使用。

结语

API-Bank就像是AI世界里一个重要的“技能认证中心”，它考验着大语言模型不仅仅拥有智慧，更具备了将智慧付诸行动的“工具使用”能力。随着像API-Bank这样的评估基准不断完善和被广泛应用，我们的AI助手将不再只是善于言辞的学者，而会进化成能够掌控各种“工具”，真正解决实际问题的强大执行者。这将把人工智能从“动脑”时代，推向一个更加贴近我们生活的“知行合一”的新阶段。

The “Toolbox” Test in the AI World: What is API-Bank?

In the era of rapid development of Artificial Intelligence (AI), Large Language Models (LLMs), such as the technology behind the well-known ChatGPT, have become smarter and smarter. They can write poetry, tell stories, translate languages, and even perform complex programming. But these “super brains” also have their limitations—they are mainly good at processing language and knowledge, and are often incapable of “operations” and “calculations” in the real world. This leads to a key concept: API-Bank.

To understand API-Bank, we must first start with a few daily concepts.

1. What is an API? — The “Interface” or “Socket” of Programs

Imagine you have various appliances at home: rice cookers, TVs, washing machines. Each appliance has a plug, and there are many sockets on the wall. By plugging the plug into the correct socket, the appliance can get power and start working.

In the computer world, API (Application Programming Interface) is like the “socket” and “plug” between programs. It defines a set of rules and methods that allow different software to communicate with each other, exchange data, and request each other to complete specific tasks.

For example, a weather forecast App may call the API of a weather data provider to get real-time weather information and display it to you. The App itself does not need to measure temperature or wind speed; it only needs to know how to “plug into” the weather API to get the desired data.

2. Large Language Model (LLM) — The Intelligent Assistant Good at “Thinking”

Now, let’s turn our attention to the core of the AI field—Large Language Models (LLMs). You can imagine an LLM as a super scholar who is learned and eloquent. It has read almost all human written materials, so its understanding of knowledge and use of language has reached an unprecedented height. You can ask it questions, let it create, or even help it make suggestions, and it can give amazing answers.

However, this super scholar also has its weaknesses. If you ask it to:

  • “Book me a flight to Beijing at 8 pm tonight.”
  • “Check how much money is left in my bank account?”
  • “Help me calculate the average of this pile of complex data.”

These tasks are beyond its pure “language and knowledge” scope, but require “practical operation” or “precise calculation” capabilities. This is where LLMs need “tools” to help.

3. LLM’s “Tool Use” — From “Thinking” to “Doing”

When our super scholar cannot complete certain tasks independently, it needs to learn how to use external “professional tools”. These “tools” are the various APIs mentioned earlier.

  • Book a flight? It needs to call the “Flight Booking API”.
  • Check bank balance? It needs to call the “Bank Query API”.
  • Perform complex calculations? It needs to call the “Calculator API” or “Data Analysis API”.

A truly intelligent AI must not only be knowledgeable but also learn to identify and use appropriate tools to solve problems when needed, just like humans. This ability is called “Tool-Use” in the AI field.

4. API-Bank: The “Driving License Exam” for Assessing LLM’s “Tool Use” Ability

Now, it’s finally time for our protagonist: API-Bank.

API-Bank is not an actual “bank” or “application”, but a comprehensive benchmark designed specifically to assess how Large Language Models (LLMs) use external tools (APIs). You can think of it as a “driving license exam” or “tool skill assessment” prepared for intelligent assistants.

Imagine we take this super scholar who understands language to a huge “workshop” with various tools. There are 53 to 73 common API tools in this workshop, such as calendar API, weather API, map API, shopping API, and even more complex database query APIs, etc. The design purpose of API-Bank is to see if this super scholar can, when facing a task:

  1. Understand the Task: Accurately judge the problem to be solved.
  2. Plan Steps: Think about the steps needed to solve the problem.
  3. Select Tools: Pick the most suitable one or several APIs from the dazzling array of tools.
  4. Call Correctly: Follow the API instructions to issue correct commands to the API and provide correct parameters (like plugging the plug into the correct socket and pressing the correct button).
  5. Process Results: Understand the results returned by the API and use them to complete the task or make the next decision.

API-Bank designs a large number of test questions by simulating real dialogue situations, allowing LLMs to use APIs in “actual combat” in these scenarios. For example, give it a request: “Help me add next Tuesday’s meeting schedule to my calendar, the meeting theme is ‘Project Review’, and the location is in Conference Room A.” The LLM needs to judge that this requires the “Calendar API”, then extract information such as date, theme, location, etc., and call the API in the correct format to complete the operation of adding the schedule.

5. Why is API-Bank So Important?

The emergence of API-Bank has milestone significance for the AI field.

  • Promoting LLM Development: It provides a standardized “exam room” for researchers to systematically measure the strengths and weaknesses of different LLMs in tool use. By analyzing the performance of LLMs on API-Bank, deficiencies can be found, thereby guiding how to improve models so that they can better learn to “hands-on” operations.
  • Bridging the Gap Between Real World and AI: An AI that can only “talk” is not enough. If AI can freely call external tools, it can better interact with the real world and complete more complex tasks, such as smart home control, personal schedule management, automated data analysis, etc.
  • Accelerating AI Application Implementation: With the improvement of LLM tool use capabilities, future AI applications will be more powerful and flexible. Developers can more easily integrate various AI models to create more innovative products and services.

For example, Microsoft’s Azure API Management provides AI gateway functions to help enterprises manage and protect AI services, allowing AI models to use and provide different API capabilities more safely and efficiently. API platforms like Postman also emphasize “AI-ready APIs” to ensure that APIs can be better understood and used by AI Agents.

Conclusion

API-Bank is like an important “skill certification center” in the AI world. It tests that large language models not only possess wisdom but also have the “tool use” ability to put wisdom into action. As assessment benchmarks like API-Bank continue to improve and be widely used, our AI assistants will no longer be just scholars good at words, but will evolve into powerful executors capable of controlling various “tools” and truly solving practical problems. This will push artificial intelligence from the “thinking” era to a new stage of “unity of knowledge and action” that is closer to our lives.