Execution details

Execution details#

The trace and trace-verbose commands provide powerful execution tracing and debugging capabilities for Squirro Chat. These slash commands enable detailed monitoring of query execution patterns, performance metrics, and system behavior during conversations. The execution details are captured in the browser developer tools, helping developers and administrators understand and optimize AI agent interactions.

Privacy Note

Although the technical name for the execution details feature is telemetry, no data is transmitted to external systems or to Squirro servers. The execution details are temporarily stored only in your web browser memory and are accessible exclusively through your browser developer tools. That data remains local to your session and is not collected, stored, or transmitted anywhere.

Overview#

The trace feature in Squirro Chat captures detailed execution information as the system processes user queries. When enabled via the trace commands, the system records events throughout the request lifecycle, including:

  • Agent execution phases.

  • Query generation and execution patterns.

  • Search operations and retrieval pipelines.

  • Tool invocations and responses.

  • Document context assembly.

  • Performance metrics and timing data.

The trace system uses correlation IDs to track related events across the entire request chain, providing a complete picture of how the system processes each interaction.

Using trace#

The trace command enables standard trace mode for a single query. To use it, prefix your chat message with the /trace instruction.

/trace What are the key findings in our Q4 reports?

When the system processes this query, it:

  1. Enables execution details collection for this request.

  2. Removes the /trace prefix from your prompt.

  3. Processes the query normally while capturing the execution events.

  4. Includes execution details in the network response (visible in the browser developer tools).

The standard trace mode delivers key debugging details, such as query patterns, the number of results returned, and the overall execution flow. It is designed to offer useful insights without excessive complexity. You can access the output through your browser developer tools, as outlined in the Accessing Trace Output section.

Using trace-verbose#

The /trace-verbose command enables verbose trace mode, which captures more detailed information:

/trace-verbose Analyze customer sentiment trends across our support tickets

Verbose mode includes all standard execution events plus:

  • Detailed parameter values and configurations.

  • Comprehensive search metrics and scoring information.

  • Extended pipeline transformation data.

  • Granular timing breakdowns.

  • Additional context about decision-making processes.

Use verbose mode when you need an in-depth analysis of system behavior or when troubleshooting complex issues. You can access the output through your browser developer tools, as outlined in the Accessing Trace Output section.

Accessing Trace Output#

Trace execution details are included in the network response but are not displayed in the Squirro Chat interface. To view them, you must use your web browser developer tools:

  1. Open Developer Tools

    For Chrome, press F12 or Ctrl+Shift+I (Windows/Linux) / Cmd+Option+I (Mac).

  2. Navigate to the Network Tab

    Click on the Network tab in the developer tools panel.

  3. Filter by Fetch/XHR

    In the Network tab, filter the requests by selecting Fetch/XHR to show only API requests.

  4. Send Your Trace Command

    Type in the Squirro Chat interface your /trace or /trace-verbose command followed by the desired prompt and submit it.

  5. Locate the streaming_chat Request

    In the filtered network requests, find and click on the streaming_chat entry that was created when you submitted your query.

  6. View the EventStream Tab

    In the request details, navigate to the EventStream tab.

  7. Find the Telemetry Event

    Scroll through the event stream to locate events with the type telemetry. These events contain the execution details in JSON format.

Tip

Keep the developer tools open before sending your trace command to ensure you capture the network request from the beginning.

Execution Events#

When tracing is enabled, the system captures various event types throughout the request lifecycle. All execution events include:

  • Timestamp

    When the event occurred.

  • Event type

    Category of the event.

  • Message

    Human-readable description.

  • Data

    Event-specific structured information.

  • Correlation ID

    Unique identifier linking related events.

Agent Execution Events#

These events track the overall agent execution flow:

  • agent_execution_start

    The agent begins processing the user instruction.

  • agent_execution_complete

    The agent finishes processing and returns results.

Query Generation and Execution#

Events related to search query construction and execution:

  • squirro_queries_generated

    Number of Squirro queries generated from the user instruction.

  • squirro_query_execution_pattern

    Query execution patterns including result set ratios.

Search and Retrieval#

Detailed search operation events:

  • squirro_paragraph_search_results

    Search metrics including total paragraphs, unique documents, and clustering predictions.

  • squirro_document_context_assembly

    Paragraph-to-cluster transformation with consolidation metrics.

  • squirro_retrieval_pipeline_summary

    End-to-end transformation overview with per-document breakdown.

Tool and Output Structure#

Events capturing tool execution and response formatting:

  • squirro_final_output_structure

    Correlation between the tool output markdown structure and the LLM input for verification.

Understanding Trace Data Format#

The execution details captured in the telemetry event follow a JSON structure.

{
"telemetry_data": {
"correlation_id": "req-23085-597",
"total_traces": 3,
"event_types": [
"deployment_stream_start",
"agent_execution_start",
"llm_token_usage_summary"
],
"event_type_counts": {
"deployment_stream_start": 1,
"agent_execution_start": 1,
"llm_token_usage_summary": 1
},
"first_trace_timestamp": "2025-11-25T09:57:03.085507",
"last_trace_timestamp": "2025-11-25T09:57:10.598191",
"verbosity_level": "normal",
"traces": [
{
"timestamp": "2025-11-25T09:57:03.085507",
"event_type": "deployment_stream_start",
"message": "Deployment streaming_chat streaming started",
"data": {
"endpoint": "streaming_chat",
"agent_id": "WHAaZDUQR76ohbiVqUlUIw",
"runtime_config_keys": [
"llm_name",
"llm_api_key",
"llm_provider",
"llm_base_url",
"llm_temperature",
"llm_top_p",
"llm_api_version",
"disable_custom_llama_tool_calling",
"enable_pii_masking",
"pii_masking_config",
"llm_seed",
"llm_reasoning_effort",
"max_agent_iterations",
"persona",
"default_persona",
"default_assistant_lang_instruction",
"default_retrieval_lang_instruction",
"default_user_prompt_extra",
"chat_history",
"extended_chat_history",
"agent_config",
"squirro_cluster",
"squirro_refresh_token",
"squirro_project_id",
"squirro_tenant",
"squirro_client_args",
"squirro_query_context",
"item_id",
"llm_context_length",
"attachments",
"conversation_id"
],
"telemetry_requested": "on"
},
"correlation_id": "req-23085-597"
},
{
"timestamp": "2025-11-25T09:57:03.404086",
"event_type": "agent_execution_start",
"message": "Tool calling agent execution started",
"data": {
"instruction": "What are the key findings in our Q4 reports?",
"persona": "",
"language": {
"assistant": "",
"retrieval": "en"
},
"max_iterations": 10,
"return_intermediate_steps": false,
"tools_count": 0,
"using_extended_chat_history": true
},
"correlation_id": "req-23085-597"
},
{
"timestamp": "2025-11-25T09:57:10.598191",
"event_type": "llm_token_usage_summary",
"message": "LLM token usage summary with total/max usage per model, invocation counts, and usage metadata",
"data": {
"total_usage": {
"gpt-4.1-2025-04-14": {
"input_tokens": 7082,
"output_tokens": 452,
"total_tokens": 7534,
"input_token_details": {
"audio": 0,
"cache_read": 0,
"cache_creation": 0
},
"output_token_details": {
"audio": 0,
"reasoning": 0
}
}
},
"max_usage": {
"gpt-4.1-2025-04-14": {
"input_tokens": 7082,
"output_tokens": 452,
"total_tokens": 7534,
"input_token_details": {
"audio": 0,
"cache_read": 0
},
"output_token_details": {
"audio": 0,
"reasoning": 0
}
}
},
"invocation_counts": {
"gpt-4.1-2025-04-14": 1
},
"model_count": 1,
"has_usage_data": true
},
"correlation_id": "req-23085-597"
}
]
},
"endpoint": "streaming_chat"
}

The telemetry data includes:

  • Summary Information

    Total trace count, event types, and timestamps showing the execution duration (7.5 seconds in this example).

  • Correlation ID

    Unique identifier req-23085-597 linking all events in this request.

  • Event Traces

    Chronologically ordered events from deployment start through agent execution to final token usage.

  • Detailed Metrics

    LLM token usage showing 7,082 input tokens and 452 output tokens for the GPT-4.1 model.

Events are sorted chronologically by timestamp, making it easy to follow the execution flow from start to finish.

The telemetry event appears as one of multiple events in the EventStream response. Other event types in the stream include the actual chat response chunks, making it important to specifically look for the event with type telemetry to find the execution details.


Contents