# Execution Details The trace and trace-verbose commands provide powerful execution tracing and debugging capabilities for Squirro Chat. These slash commands enable detailed monitoring of query execution patterns, performance metrics, and system behavior during conversations. The execution details are captured in the browser developer tools, helping developers and administrators understand and optimize AI agent interactions. > **Privacy Note** > > Although the technical name for the execution details feature is `telemetry`, no data is transmitted to external systems or to Squirro servers. The execution details are temporarily stored only in your web browser memory and are accessible exclusively through your browser developer tools. That data remains local to your session and is not collected, stored, or transmitted anywhere. ## Overview The trace feature in Squirro Chat captures detailed execution information as the system processes user queries. When enabled via the trace commands, the system records events throughout the request lifecycle, including: - Agent execution phases. - Query generation and execution patterns. - Search operations and retrieval pipelines. - Tool invocations and responses. - Document context assembly. - Performance metrics and timing data. The trace system uses correlation IDs to track related events across the entire request chain, providing a complete picture of how the system processes each interaction. ## Using trace The trace command enables standard trace mode for a single query. To use it, prefix your chat message with the `/trace` instruction. ```text /trace What are the key findings in our Q4 reports? ``` When the system processes this query, it: 1. Enables execution details collection for this request. 2. Removes the `/trace` prefix from your prompt. 3. Processes the query normally while capturing the execution events. 4. Includes execution details in the network response (visible in the browser developer tools). The standard trace mode delivers key debugging details, such as query patterns, the number of results returned, and the overall execution flow. It is designed to offer useful insights without excessive complexity. You can access the output through your browser developer tools, as outlined in the [Accessing Trace Output](#accessing-trace-output) section. ## Using trace-verbose The `/trace-verbose` command enables verbose trace mode, which captures more detailed information: ```text /trace-verbose Analyze customer sentiment trends across our support tickets ``` Verbose mode includes all standard execution events plus: - Detailed parameter values and configurations. - Comprehensive search metrics and scoring information. - Extended pipeline transformation data. - Granular timing breakdowns. - Additional context about decision-making processes. Use verbose mode when you need an in-depth analysis of system behavior or when troubleshooting complex issues. You can access the output through your browser developer tools, as outlined in the [Accessing Trace Output](#accessing-trace-output) section. ## Accessing Trace Output Trace execution details are included in the network response but are not displayed in the Squirro Chat interface. To view them, you must use your web browser developer tools: 1. **Open Developer Tools** For Chrome, press `F12` or `Ctrl+Shift+I` (Windows/Linux) / `Cmd+Option+I` (Mac). 2. **Navigate to the Network Tab** Click on the **Network** tab in the developer tools panel. 3. **Filter by Fetch/XHR** In the Network tab, filter the requests by selecting **Fetch/XHR** to show only API requests. 4. **Send Your Trace Command** Type in the Squirro Chat interface your `/trace` or `/trace-verbose` command followed by the desired prompt and submit it. 5. **Locate the streaming_chat Request** In the filtered network requests, find and click on the `streaming_chat` entry that was created when you submitted your query. 6. **View the EventStream Tab** In the request details, navigate to the **EventStream** tab. 7. **Find the Telemetry Event** Scroll through the event stream to locate events with the type `telemetry`. These events contain the execution details in JSON format. > **Tip** > > Keep the developer tools open before sending your trace command to ensure you capture the network request from the beginning. ## Execution Events When tracing is enabled, the system captures various event types throughout the request lifecycle. All execution events include: - **Timestamp** When the event occurred. - **Event type** Category of the event. - **Message** Human-readable description. - **Data** Event-specific structured information. - **Correlation ID** Unique identifier linking related events. ### Agent Execution Events These events track the overall agent execution flow: - `agent_execution_start` The agent begins processing the user instruction. - `agent_execution_complete` The agent finishes processing and returns results. ### Query Generation and Execution Events related to search query construction and execution: - `squirro_queries_generated` Number of Squirro queries generated from the user instruction. - `squirro_query_execution_pattern` Query execution patterns including result set ratios. ### Search and Retrieval Detailed search operation events: - `squirro_paragraph_search_results` Search metrics including total paragraphs, unique documents, and clustering predictions. - `squirro_document_context_assembly` Paragraph-to-cluster transformation with consolidation metrics. - `squirro_retrieval_pipeline_summary` End-to-end transformation overview with per-document breakdown. ### Tool and Output Structure Events capturing tool execution and response formatting: - `squirro_final_output_structure` Correlation between the tool output markdown structure and the LLM input for verification. ## Understanding Trace Data Format The execution details captured in the `telemetry` event follow a JSON structure. ```json { "telemetry_data": { "correlation_id": "req-23085-597", "total_traces": 3, "event_types": [ "deployment_stream_start", "agent_execution_start", "llm_token_usage_summary" ], "event_type_counts": { "deployment_stream_start": 1, "agent_execution_start": 1, "llm_token_usage_summary": 1 }, "first_trace_timestamp": "2025-11-25T09:57:03.085507", "last_trace_timestamp": "2025-11-25T09:57:10.598191", "verbosity_level": "normal", "traces": [ { "timestamp": "2025-11-25T09:57:03.085507", "event_type": "deployment_stream_start", "message": "Deployment streaming_chat streaming started", "data": { "endpoint": "streaming_chat", "agent_id": "WHAaZDUQR76ohbiVqUlUIw", "runtime_config_keys": [ "llm_name", "llm_api_key", "llm_provider", "llm_base_url", "llm_temperature", "llm_top_p", "llm_api_version", "disable_custom_llama_tool_calling", "enable_pii_masking", "pii_masking_config", "llm_seed", "llm_reasoning_effort", "max_agent_iterations", "persona", "default_persona", "default_assistant_lang_instruction", "default_retrieval_lang_instruction", "default_user_prompt_extra", "chat_history", "extended_chat_history", "agent_config", "squirro_cluster", "squirro_refresh_token", "squirro_project_id", "squirro_tenant", "squirro_client_args", "squirro_query_context", "item_id", "llm_context_length", "attachments", "conversation_id" ], "telemetry_requested": "on" }, "correlation_id": "req-23085-597" }, { "timestamp": "2025-11-25T09:57:03.404086", "event_type": "agent_execution_start", "message": "Tool calling agent execution started", "data": { "instruction": "What are the key findings in our Q4 reports?", "persona": "", "language": { "assistant": "", "retrieval": "en" }, "max_iterations": 10, "return_intermediate_steps": false, "tools_count": 0, "using_extended_chat_history": true }, "correlation_id": "req-23085-597" }, { "timestamp": "2025-11-25T09:57:10.598191", "event_type": "llm_token_usage_summary", "message": "LLM token usage summary with total/max usage per model, invocation counts, and usage metadata", "data": { "total_usage": { "gpt-4.1-2025-04-14": { "input_tokens": 7082, "output_tokens": 452, "total_tokens": 7534, "input_token_details": { "audio": 0, "cache_read": 0, "cache_creation": 0 }, "output_token_details": { "audio": 0, "reasoning": 0 } } }, "max_usage": { "gpt-4.1-2025-04-14": { "input_tokens": 7082, "output_tokens": 452, "total_tokens": 7534, "input_token_details": { "audio": 0, "cache_read": 0 }, "output_token_details": { "audio": 0, "reasoning": 0 } } }, "invocation_counts": { "gpt-4.1-2025-04-14": 1 }, "model_count": 1, "has_usage_data": true }, "correlation_id": "req-23085-597" } ] }, "endpoint": "streaming_chat" } ``` The telemetry data includes: - **Summary Information** Total trace count, event types, and timestamps showing the execution duration (7.5 seconds in this example). - **Correlation ID** Unique identifier `req-23085-597` linking all events in this request. - **Event Traces** Chronologically ordered events from deployment start through agent execution to final token usage. - **Detailed Metrics** LLM token usage showing 7,082 input tokens and 452 output tokens for the GPT-4.1 model. Events are sorted chronologically by timestamp, making it easy to follow the execution flow from start to finish. The `telemetry` event appears as one of multiple events in the EventStream response. Other event types in the stream include the actual chat response chunks, making it important to specifically look for the event with type `telemetry` to find the execution details.