Datadog

​

Overview

• Query metrics and dashboards
• Search logs
• Access APM traces and service maps
• Retrieve monitor/alert status

​

Prerequisites

• Datadog account
• API Key and Application Key
• Read permissions for metrics, logs, and APM

​

Configuration

​

Step 1: Generate API Keys

1. Log in to Datadog
2. Go to Organization Settings > API Keys
3. Click New Key to generate an API Key
4. Go to Application Keys tab
5. Click New Key to generate an Application Key

​

Step 2: Add to IncidentFox

1. Go to Team Console > Integrations
2. Click Add Integration > Datadog
3. Enter:
   • API Key
   • Application Key
   • Site (e.g., datadoghq.com)
4. Click Test Connection
5. Save

{
  "tools": {
    "datadog": {
      "enabled": true,
      "api_key": "vault://secrets/datadog-api-key",
      "app_key": "vault://secrets/datadog-app-key",
      "site": "datadoghq.com"
    }
  }
}

​

Datadog Sites

​

Available Tools

​

query_datadog_metrics

@incidentfox query datadog metrics for avg:system.cpu.user by service in the last hour

• query - Datadog metrics query
• from_time - Start time (Unix timestamp or relative)
• to_time - End time

​

search_datadog_logs

@incidentfox search datadog logs for status:error service:payments

• query - Log search query
• indexes - Log indexes to search
• time_range - Time range

​

get_service_apm_metrics

@incidentfox get APM metrics for the checkout service including error rate and latency

• service - Service name
• env - Environment (optional)
• time_range - Time range

​

Use Cases

​

Investigating Service Latency

@incidentfox investigate high latency in payments service using Datadog

IncidentFox will:
1. Query APM latency metrics
2. Get error rates
3. Check for anomalies
4. Review traces with high duration

​

Log Analysis

@incidentfox search for exceptions in Datadog logs for checkout-api

IncidentFox will:
1. Search logs for error patterns
2. Identify common stack traces
3. Correlate with deployments

​

Next Steps