How OpenAI’s Function Calling Breaks Programming Boundaries
OpenAI Function calling bridges the gap between deterministic and non-deterministic programming — leading to all sorts of possibilities
On June 13th, 2023, OpenAI released some exciting updates — cheaper ChatGPT/GPT-4 models, cheaper embeddings model, and a 4X increase in context length. But apart from this, there was a new innovation that might just completely revamp the way we write code to interact with machines. The way we code, is significantly more rigid than the way we write. The same content can be conveyed very differently by different authors — or even by one author, at different times. If I were to delete all the above content and write it again, it would likely be different word to word, but convey the same ideas. Whereas, writing code is much more prescriptive. So how does OpenAI’s function calling change the game? Let’s dive into it!
What Is Function Calling?
By now, many of us are used to the ChatGPT/GPT-4 APIs that respond to user prompts. In the “Function Calling” mode, rather than giving a standard response, the API gives the output after passing arguments through a custom “function”. The format of this function is where the magic lies:
function = [{
"name": "get_cyclone",
"description": "Get the cyclone impact information as tuples (location, impact) from the context",
"parameters": {
"type": "object",
"properties": {
"impacts": {
"type": "array",
"items":{"type": "string","description": "(location,impact)"},
"description": "(location, impact)",
},
},
"required": ["impacts"],
},
}]
The above function is made to extract hurricane-related information in tuples of (location, impact) from a context. You can call this function by adding the arguments functions and function_call to the standard completion API. Functions can be a list of different functions, and function_call denotes the name of the particular function you wish to call.
completion=openai.ChatCompletion.create(
model="gpt-4-0613",
messages=[{"role": "user", "content": f"{context}"}],
functions=function,
function_call={"name": "get_cyclone"},
)
print(completion)
And here is the output!
"choices": [
{
"finish_reason": "stop",
"index": 0,
"message": {
"content": null,
"function_call": {
"arguments": "{\n\"impacts\": [\"India's western state of Gujarat, Roofs were blown off houses and trees and electric poles uprooted, leaving thousands without power\", \n \"India's western state of Gujarat, At least two people died after being swept away by flood waters\", \n \"India and Pakistan, More than 180,000 people were evacuated\", \n \"Jakhau, Gujarat, It made landfall\", \n \"Gujarat's Bhavnagar district, Two men died while trying to rescue their cattle from being swept away during heavy rains and floods\", \n \"Kutch district of Gujarat, Power was disrupted at many places because of strong winds\", \n \"Pakistan, Rain reported in some parts of Karachi, high alert in place\", \n \"Coastal areas of Gujarat, Gale-force winds caused hundreds of trees to be uprooted and heavy rains led to electricity poles being damaged, causing thousands to be without power\", \n \"Gujarat and the neighbouring state of Rajasthan, Heavy to very heavy rainfall expected\", \n \"Hyderabad, Nooriabad and Thatta regions in Pakistan, Moderate to heavy rain expected\",\n \"All offshore oil installations, The government had directed all to ensure the immediate return of all staff to land\"]\n}",
"name": "get_cyclone"
},
"role": "assistant"
}
}
],
"created": 1687024741,
"id": "chatcmpl-7SUUPpeNRSe2YnujkJik0312h6WB8",
"model": "gpt-4-0613",
"object": "chat.completion",
"usage": {
"completion_tokens": 261,
"prompt_tokens": 704,
"total_tokens": 965
}
Notice in the above output, the “content” field has a null value. In the regular chat completion API usage, the output from ChatGPT API is in the content key. But here, the function returns an output, contained in the “arguments” field, which contains a child object “impacts”. Let’s look more closely at what is returned by the function.
reply_content = completion.choices[0].message
funcs = reply_content.to_dict()[‘function_call’][‘arguments’]
funcs = json.loads(funcs)
print(funcs["impacts"])
As you can see, it contains a list of objects of the form [location,impact]. In the first object — the location is “India’s western state of Gujarat” while the impact is “Roofs were blown off houses and trees, and electric poles uprooted, leaving thousands without power”.
["India's western state of Gujarat, Roofs were blown off houses and trees and electric poles uprooted, leaving thousands without power",
"India's western state of Gujarat, At least two people died after being swept away by flood waters",
'India and Pakistan, More than 180,000 people were evacuated',
'Jakhau, Gujarat, It made landfall',
"Gujarat's Bhavnagar district, Two men died while trying to rescue their cattle from being swept away during heavy rains and floods",
'Kutch district of Gujarat, Power was disrupted at many places because of strong winds',
'Pakistan, Rain reported in some parts of Karachi, high alert in place',
'Coastal areas of Gujarat, Gale-force winds caused hundreds of trees to be uprooted and heavy rains led to electricity poles being damaged, causing thousands to be without power',
'Gujarat and the neighbouring state of Rajasthan, Heavy to very heavy rainfall expected',
'Hyderabad, Nooriabad and Thatta regions in Pakistan, Moderate to heavy rain expected',
'All offshore oil installations, The government had directed all to ensure the immediate return of all staff to land']
Function Calling With Multiple Arguments
What if you want to extract multiple types of information from the same context? For this, you can call multiple functions as below.
function2 = [{
"name": "get_cyclone",
"description": "Get the cyclone impact information as tuples (location, impact) from the context",
"parameters": {
"type": "object",
"properties": {
"impacts": {
"type": "array",
"items":{"type": "string","description": "(location,impact)"},
"description": "(location, impact)",
},
},
"required": ["impacts"],
},
},
{
"name": "get_cyclone2",
"description": "Get the cyclone impact information as tuples from the context",
"parameters": {
"type": "object",
"properties": {
"impacts": {
"type": "array",
"items":{"type": "string","description": "(location,impact)"},
"description": "(location, impact)",
},
"hurricane_date": {
"type": "array",
"items":{"type": "string","description": "(location,date)"},
"description": "(location, date of landfall)",
}
},
"required": ["impacts","hurricane_date"],
},
}]
Notice how in the second function “get_cyclone2”, there are 2 required fields, [“impacts”,“hurricane_date”].
completion=openai.ChatCompletion.create(
model="gpt-4-0613",
messages=[{"role": "user", "content": f"{context}"}],
functions=function2,
function_call={"name": "get_cyclone2"},
)
reply_content = completion.choices[0].message
funcs = reply_content.to_dict()['function_call']['arguments']
funcs = json.loads(funcs)
print(funcs["hurricane_date"])
The response will contain these two fields — the below output contains a list of 2 objects of the form [location, date of landfall].
['Jakhau, Gujarat, India, late on Thursday',
'Karachi, Pakistan, early on Friday']
Integrating With SQL Databases
ChatGPT can also be used to query databases — by using functions to convert unstructured questions about the database into structured SQL prompts to then send as queries. In this example, I query my AnswerChatAI DB that I built for users to ask questions about websites/small strings of text, powered by ChatGPT. I also have an API for asking questions on longer documents.
First, I load the database table containing relevant information (question, answer, link, etc.) as a CSV and connect it to sqlite3, which comes built into Python.
df = pd.read_csv("./qa2_all_site.csv")
table_name='q_a'
conn = sqlite3.connect('mydb.sqlite')
query = f'Create table if not Exists {table_name} (id, link, question, answer, created_at, text_data, df,article_id, message)'
conn.execute(query)
df.to_sql(table_name,conn,if_exists='replace',index=False)
conn.commit()
functions = [
{
"name": "ask_database",
"description": "Use this function to answer user questions about the q_a table. Output should be a fully formed SQL query.",
"parameters": {
"type": "object",
"properties": {
"query": {
"type": "string",
"description": f"""
SQL query extracting info to answer the user's question.
SQL should be written using this database schema:
{database_schema_string}
The query should be returned in plain text, not in JSON.
""",
}
},
"required": ["query"],
},
}
]
As you can see, I describe a function as above and pass it through ChatGPT here, along with the question, “what are the 5 most common questions”.
messages = []
messages.append({"role": "system", "content": "Answer user questions by generating SQL queries against the q_a Database."})
messages.append({"role": "user", "content": "Hi, what are the 5 most common questions?"})
chat_response = chat_completion_request(messages, functions)
assistant_message = chat_response.json()["choices"][0]["message"]
messages.append(assistant_message)
if assistant_message.get("function_call"):
results = execute_function_call(assistant_message)
messages.append({"role": "function", "name": assistant_message["function_call"]["name"], "content": results})
messages[3]['content']
Based on the unstructured query sent to ChatGPT, you can see the answer it returns along with the counts. That’s pretty neat if I was a beginner in SQL or want to get creative with my queries!
"[('What is this?', 30), ('Who am I?', 15), ('what does it talk about?', 13), ('We need to make a description of this site', 13), ('Summarize', 13)]"
Takeaways
OpenAI’s function calling updates herald a new higher-level of programming abstraction, sitting on top of programming languages like Python, C++, Java, SQL, etc. I’ve shown examples where these functions blur the distinctions between unstructured and structured queries. In the information extraction examples, you saw how unstructured queries coupled with structured functions give structured outputs. You also saw how unstructured queries abstracting SQL queries can be used to extract information from databases.
This is just the beginning, however, as there are still issues, and there is a long way to go before this unstructured <-> structured paradigm becomes commonplace. For one, you know how exactly a SQL query works — there is no ambiguity. However, abstracting this through an unstructured GPT query can lead to ambiguity. For example, in the output, I obtained 2 pieces of information (most common question, count) — when I just asked for the most common question. Sure, this is useful, but it might not be robust. What if I asked this question in a slightly different manner — would I still get the same result? It turns out if I frame the question slightly differently — “what is the top question?” the output is different — it just gives the top results as ordered in the dataframe, without counts. Similarly, in the functions for extracting information, when I slightly change the prompt, the resulting output significantly changes.
What does this mean for the future? Of course, the underlying LLM and functions need to be improved. Maybe GPT-5 or future versions will return more reliable outputs. Or/and the community gets a better handle on how to frame prompts and functions. If this paradigm starts to be widely adopted, programming may become a much more creative occupation — prioritizing language-based logic and creativity and deprioritizing knowledge of specific programming languages. There might be job titles like “prompt data scientist” or “prompt data engineer” — the former focusing on writing unstructured prompts that abstract programming languages and the latter focused on making sure that the code written by the data scientist matches their expectations — avoiding hallucinations.
Whatever happens, I’m sure the future of programming will be very different when my kids graduate, and I echo Andrew Ng and Andrea Pasinetti’s piece that it is important that schools teach AI and coding skills to every child.
The code to the above tutorial is in this repository:
https://github.com/skandavivek/openai-function-calling
Here are some related articles:
LLM Evaluations:
4 Crucial Factors for Evaluating Large Language Models in Industry Applications
How Do You Evaluate Large Language Model Apps — When 99% is just not good enough?
Extractive vs Generative Q&A — Which is better for your business?
When Should You Fine-Tune LLMs?
Unleashing the Power of Generative AI For Your Customers
LLM Deployment:
Deploying Open-Source LLMs As APIs
LLM Systems Design:
Build Industry-Specific LLMs Using Retrieval Augmented Generation
How Do You Build A ChatGPT-Powered App?
Fine-Tune Transformer Models For Question Answering On Custom Data
LLM Economics: ChatGPT vs Open-Source
The Economics of Large Language Models
LLM Tooling:
Build A Custom AI Based ChatBot Using Langchain, Weviate, and Streamlit
How OpenAI’s New Function Calling Breaks Programming Boundaries