Function Calling via Prompt Engineering — Pattern

Problem

ChatGPT Web (and most chat2api projects) don't support native function calling / tool use. The /backend-api/conversation endpoint doesn't accept tools or tool_choice parameters.

Solution

Inject tool definitions into the system prompt and parse JSON tool calls from the model's response.

Implementation

1. Tool System Prompt Template

TOOL_CALL_SYSTEM_PROMPT = """# Tools

You have access to the following tools:

{tools_json}

## Tool Use Protocol

When you need to use a tool, respond ONLY with a JSON object in the following format, wrapped in ```json code blocks:

```json
{{
  "tool_calls": [
    {{
      "id": "call_<random>",
      "type": "function",
      "function": {{
        "name": "<tool_name>",
        "arguments": "<json_encoded_arguments>"
      }}
    }}
  ]
}}

After the tool call, the user will respond with the tool output. Then you can continue the conversation.

IMPORTANT:

Only call tools when absolutely necessary
Use proper JSON encoding for arguments
If no tool is needed, respond normally without JSON
Do NOT wrap your normal responses in JSON — only tool calls """


### 2. Inject into Messages

```python
def inject_tools_into_messages(messages: list, tools: list | None) -> list:
    if not tools:
        return messages
    
    tools_json = json.dumps(tools, indent=2, ensure_ascii=False)
    tool_system_msg = {
        "role": "system",
        "content": TOOL_CALL_SYSTEM_PROMPT.format(tools_json=tools_json)
    }
    
    # Find existing system message or prepend
    for msg in messages:
        if msg.get("role") == "system":
            msg["content"] += "\n\n" + tool_system_msg["content"]
            return messages
    
    messages.insert(0, tool_system_msg)
    return messages

3. Parse Tool Calls from Response

def parse_tool_calls_from_response(content: str) -> tuple[list | None, str | None]:
    """Returns (tool_calls, text_content) or (None, content)"""
    # Look for ```json ... ``` blocks
    json_pattern = r'```json\s*\n(.*?)\n\s*```'
    json_match = re.search(json_pattern, content, re.DOTALL)
    
    if json_match:
        try:
            parsed = json.loads(json_match.group(1))
            if isinstance(parsed, dict) and "tool_calls" in parsed:
                # Extract text before/after JSON
                text_before = content[:json_match.start()].strip()
                text_after = content[json_match.end():].strip()
                return parsed["tool_calls"], (text_before + " " + text_after).strip() or None
        except json.JSONDecodeError:
            pass
    
    return None, content

4. Build OpenAI-Compatible Response

def build_openai_tool_call_response(model, tool_calls, text_content=None, finish_reason="tool_calls"):
    return {
        "id": f"chatcmpl-{uuid.uuid4().hex[:29]}",
        "object": "chat.completion",
        "created": int(time.time()),
        "model": model,
        "choices": [{
            "index": 0,
            "message": {
                "role": "assistant",
                "content": text_content,
                "tool_calls": [
                    {
                        "id": tc.get("id", f"call_{uuid.uuid4().hex[:24]}"),
                        "type": "function",
                        "function": {
                            "name": tc["function"]["name"],
                            "arguments": tc["function"]["arguments"]
                            if isinstance(tc["function"]["arguments"], str)
                            else json.dumps(tc["function"]["arguments"])
                        }
                    }
                    for tc in tool_calls
                ]
            },
            "finish_reason": finish_reason
        }],
        "usage": {"prompt_tokens": 0, "completion_tokens": 0, "total_tokens": 0}
    }

5. Integration in ChatService

In set_dynamic_data():

self.tools = self.data.get("tools", None)
if self.tools:
    self.api_messages = inject_tools_into_messages(list(self.api_messages), self.tools)

In send_conversation() — after getting response, before returning:

if self.tools:
    full_content = collect_all_content(response)
    tool_calls, text = parse_tool_calls_from_response(full_content)
    if tool_calls:
        return build_openai_tool_call_response(self.resp_model, tool_calls, text)

Limitations

Prompt engineering, not native: The model may ignore tool instructions or hallucinate tool calls
No streaming tool calls: Tool calls are detected only after full response
JSON parsing fragility: Models may produce malformed JSON
Token waste: System prompt with tools consumes tokens from the chat window
No parallel tool calls: The pattern supports single tool_call per response

Alternatives

For better tool calling support, use a project with native function calling like xiaoY233/Chat2API which has built-in prompt-engineering function calling with model mapping and context management.

5.0 KiB Raw Blame History