159 lines
5.0 KiB
Markdown
159 lines
5.0 KiB
Markdown
# Function Calling via Prompt Engineering — Pattern
|
|
|
|
## Problem
|
|
|
|
ChatGPT Web (and most chat2api projects) don't support native function calling / tool use. The `/backend-api/conversation` endpoint doesn't accept `tools` or `tool_choice` parameters.
|
|
|
|
## Solution
|
|
|
|
Inject tool definitions into the system prompt and parse JSON tool calls from the model's response.
|
|
|
|
## Implementation
|
|
|
|
### 1. Tool System Prompt Template
|
|
|
|
```python
|
|
TOOL_CALL_SYSTEM_PROMPT = """# Tools
|
|
|
|
You have access to the following tools:
|
|
|
|
{tools_json}
|
|
|
|
## Tool Use Protocol
|
|
|
|
When you need to use a tool, respond ONLY with a JSON object in the following format, wrapped in ```json code blocks:
|
|
|
|
```json
|
|
{{
|
|
"tool_calls": [
|
|
{{
|
|
"id": "call_<random>",
|
|
"type": "function",
|
|
"function": {{
|
|
"name": "<tool_name>",
|
|
"arguments": "<json_encoded_arguments>"
|
|
}}
|
|
}}
|
|
]
|
|
}}
|
|
```
|
|
|
|
After the tool call, the user will respond with the tool output. Then you can continue the conversation.
|
|
|
|
IMPORTANT:
|
|
- Only call tools when absolutely necessary
|
|
- Use proper JSON encoding for arguments
|
|
- If no tool is needed, respond normally without JSON
|
|
- Do NOT wrap your normal responses in JSON — only tool calls
|
|
"""
|
|
```
|
|
|
|
### 2. Inject into Messages
|
|
|
|
```python
|
|
def inject_tools_into_messages(messages: list, tools: list | None) -> list:
|
|
if not tools:
|
|
return messages
|
|
|
|
tools_json = json.dumps(tools, indent=2, ensure_ascii=False)
|
|
tool_system_msg = {
|
|
"role": "system",
|
|
"content": TOOL_CALL_SYSTEM_PROMPT.format(tools_json=tools_json)
|
|
}
|
|
|
|
# Find existing system message or prepend
|
|
for msg in messages:
|
|
if msg.get("role") == "system":
|
|
msg["content"] += "\n\n" + tool_system_msg["content"]
|
|
return messages
|
|
|
|
messages.insert(0, tool_system_msg)
|
|
return messages
|
|
```
|
|
|
|
### 3. Parse Tool Calls from Response
|
|
|
|
```python
|
|
def parse_tool_calls_from_response(content: str) -> tuple[list | None, str | None]:
|
|
"""Returns (tool_calls, text_content) or (None, content)"""
|
|
# Look for ```json ... ``` blocks
|
|
json_pattern = r'```json\s*\n(.*?)\n\s*```'
|
|
json_match = re.search(json_pattern, content, re.DOTALL)
|
|
|
|
if json_match:
|
|
try:
|
|
parsed = json.loads(json_match.group(1))
|
|
if isinstance(parsed, dict) and "tool_calls" in parsed:
|
|
# Extract text before/after JSON
|
|
text_before = content[:json_match.start()].strip()
|
|
text_after = content[json_match.end():].strip()
|
|
return parsed["tool_calls"], (text_before + " " + text_after).strip() or None
|
|
except json.JSONDecodeError:
|
|
pass
|
|
|
|
return None, content
|
|
```
|
|
|
|
### 4. Build OpenAI-Compatible Response
|
|
|
|
```python
|
|
def build_openai_tool_call_response(model, tool_calls, text_content=None, finish_reason="tool_calls"):
|
|
return {
|
|
"id": f"chatcmpl-{uuid.uuid4().hex[:29]}",
|
|
"object": "chat.completion",
|
|
"created": int(time.time()),
|
|
"model": model,
|
|
"choices": [{
|
|
"index": 0,
|
|
"message": {
|
|
"role": "assistant",
|
|
"content": text_content,
|
|
"tool_calls": [
|
|
{
|
|
"id": tc.get("id", f"call_{uuid.uuid4().hex[:24]}"),
|
|
"type": "function",
|
|
"function": {
|
|
"name": tc["function"]["name"],
|
|
"arguments": tc["function"]["arguments"]
|
|
if isinstance(tc["function"]["arguments"], str)
|
|
else json.dumps(tc["function"]["arguments"])
|
|
}
|
|
}
|
|
for tc in tool_calls
|
|
]
|
|
},
|
|
"finish_reason": finish_reason
|
|
}],
|
|
"usage": {"prompt_tokens": 0, "completion_tokens": 0, "total_tokens": 0}
|
|
}
|
|
```
|
|
|
|
### 5. Integration in ChatService
|
|
|
|
In `set_dynamic_data()`:
|
|
```python
|
|
self.tools = self.data.get("tools", None)
|
|
if self.tools:
|
|
self.api_messages = inject_tools_into_messages(list(self.api_messages), self.tools)
|
|
```
|
|
|
|
In `send_conversation()` — after getting response, before returning:
|
|
```python
|
|
if self.tools:
|
|
full_content = collect_all_content(response)
|
|
tool_calls, text = parse_tool_calls_from_response(full_content)
|
|
if tool_calls:
|
|
return build_openai_tool_call_response(self.resp_model, tool_calls, text)
|
|
```
|
|
|
|
## Limitations
|
|
|
|
- **Prompt engineering, not native**: The model may ignore tool instructions or hallucinate tool calls
|
|
- **No streaming tool calls**: Tool calls are detected only after full response
|
|
- **JSON parsing fragility**: Models may produce malformed JSON
|
|
- **Token waste**: System prompt with tools consumes tokens from the chat window
|
|
- **No parallel tool calls**: The pattern supports single tool_call per response
|
|
|
|
## Alternatives
|
|
|
|
For better tool calling support, use a project with native function calling like `xiaoY233/Chat2API` which has built-in prompt-engineering function calling with model mapping and context management. |