Gemini 3 File Search + functionDeclarations are not handled correctly in REST API streaming/history

[mysubcult]

Hi, Remi
I am using LiveHelperChat REST API with Google Gemini 3 and need both:

• built-in file_search
• custom functionDeclarations

Request uses:

"tool_config": {
  "function_calling_config": {
    "mode": "VALIDATED"
  },
  "include_server_side_tool_invocations": true
}

Problem

When the user sends a normal message, Gemini returns text and LHC handles it correctly.

When the user asks something that triggers file_search, Gemini returns multipart content where the first part is not text:

{
  "candidates": [
    {
      "content": {
        "parts": [
          {
            "thoughtSignature": "...",
            "toolCall": {
              "toolType": "FILE_SEARCH",
              "id": "u7vsw9zl"
            }
          },
          {
            "text": ""
          }
        ],
        "role": "model"
      }
    }
  ]
}

LHC currently expects text at:

candidates:0:content:parts:0:text

But with Gemini tool combination, text/function calls/errors can appear in different parts[n].

Also, Gemini SSE can return a normal candidate event followed by an error event while HTTP status is still 200. In logs LHC appears to concatenate these events, causing invalid JSON parsing:

[VALID JSON] {"candidates":[...]}
[INVALID JSON] ... }{"error":{"code":429,"message":"Failed to embed content","status":"RESOURCE_EXHAUSTED"}}

Expected

Please add native support for Gemini 3 tool combination:

1. Parse each SSE data: event separately.
2. Extract text from any candidates[0].content.parts[n].text.
3. Extract functionCall from any parts[n].functionCall.
4. Detect streamed error objects even when HTTP status is 200.
5. Preserve/replay full Gemini content history, not only visible text.

Google documentation says tool combination requires preserving and returning all parts, including:

• toolCall
• toolResponse
• functionCall
• functionResponse
• thoughtSignature
• id

Docs:

• https://ai.google.dev/gemini-api/docs/tool-combination
• https://ai.google.dev/gemini-api/docs/file-search
• https://ai.google.dev/gemini-api/docs/function-calling

Without preserving these parts, Gemini 3 file_search + functionDeclarations cannot work reliably in multi-turn chats.

Example request shape

{
  "model": "gemini-3-flash-preview",
  "tools": [
    {
      "functionDeclarations": [
        { "name": "call_operator" },
        { "name": "activate_device" },
        { "name": "call_sales" }
      ]
    },
    {
      "file_search": {
        "file_search_store_names": [
          "fileSearchStores/..."
        ]
      }
    }
  ],
  "tool_config": {
    "function_calling_config": {
      "mode": "VALIDATED"
    },
    "include_server_side_tool_invocations": true
  }
}

[remdex]

You will need to update LHC and you will be able to use new syntax to match n element. Basically checks if any of internal elements matching structure instead of first one.

• https://doc.livehelperchat.com/docs/bot/rest-api#candidates0contentpartstext__exist__text-checks-that-array-item-has-key-text
• https://doc.livehelperchat.com/docs/bot/gemini-integration