> ## Documentation Index
> Fetch the complete documentation index at: https://docs.incidentfox.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Data Sources Overview

> Connect IncidentFox to your observability stack

## Overview

IncidentFox connects to your existing observability and data platforms to investigate incidents. This section covers how to configure each data source.

## Supported Data Sources

### Observability Platforms

| Platform                                     | Status    | Capabilities                                        |
| -------------------------------------------- | --------- | --------------------------------------------------- |
| [Coralogix](/data-sources/coralogix)         | Supported | Log search, metrics, alerts, Olly integration       |
| [Datadog](/data-sources/datadog)             | Supported | Metrics, logs, APM traces                           |
| [Grafana](/data-sources/grafana)             | Supported | Prometheus queries, dashboards, alerts, annotations |
| [Prometheus](/data-sources/prometheus)       | Supported | PromQL queries, instant queries, alerts             |
| [Sentry](/data-sources/sentry)               | Supported | Error tracking, issues, project stats, releases     |
| [New Relic](/data-sources/newrelic)          | Supported | NRQL queries, APM summary                           |
| [Elasticsearch](/data-sources/elasticsearch) | Supported | Log search, aggregations                            |
| [Splunk](/data-sources/splunk)               | Supported | SPL queries, log search                             |
| Loki                                         | Supported | LogQL queries                                       |

### Cloud Providers

| Platform                     | Status    | Capabilities                                    |
| ---------------------------- | --------- | ----------------------------------------------- |
| [AWS](/data-sources/aws)     | Supported | CloudWatch, EC2, Lambda, RDS, ECS, CodePipeline |
| [Azure](/data-sources/azure) | Supported | Monitor, VMs, App Services                      |
| [GCP](/data-sources/gcp)     | Supported | Cloud Logging, Compute, Cloud Run               |

### Infrastructure

| Platform                               | Status    | Capabilities                                            |
| -------------------------------------- | --------- | ------------------------------------------------------- |
| [Kubernetes](/data-sources/kubernetes) | Supported | Pod logs, events, deployments, metrics                  |
| [Docker](/data-sources/docker)         | Supported | Container logs, exec, stats, events, inspect (15 tools) |
| Terraform                              | Supported | State inspection, planning                              |

### Databases

| Platform                               | Status    | Capabilities                       |
| -------------------------------------- | --------- | ---------------------------------- |
| [Snowflake](/data-sources/snowflake)   | Supported | SQL queries, data enrichment       |
| [PostgreSQL](/data-sources/postgresql) | Supported | Query execution, schema inspection |
| [MySQL](/data-sources/mysql)           | Supported | Query execution, schema inspection |
| [BigQuery](/data-sources/bigquery)     | Supported | SQL queries, analytics             |

### Code & CI/CD

| Platform | Status    | Capabilities                                            |
| -------- | --------- | ------------------------------------------------------- |
| GitHub   | Supported | Code search, PRs, Actions, commits, webhooks (16 tools) |
| GitLab   | Supported | Repositories, merge requests                            |
| Jenkins  | Supported | Build status, logs                                      |

### Documentation & Knowledge

| Platform    | Status    | Capabilities                |
| ----------- | --------- | --------------------------- |
| Confluence  | Supported | Wiki search, page retrieval |
| Notion      | Supported | Workspace search            |
| Google Docs | Supported | Runbook search              |

### Messaging & Streaming

| Platform        | Status    | Capabilities                   |
| --------------- | --------- | ------------------------------ |
| Kafka           | Supported | Topic inspection, consumer lag |
| Debezium        | Supported | CDC monitoring                 |
| Schema Registry | Supported | Schema management              |

## Data Source Architecture

```mermaid theme={null}
graph TD
    subgraph IncidentFox
    A[K8s Agent]
    B[AWS Agent]
    C[Metrics Agent]
    D[Coding Agent]
    E[Investigation Agent]
    end
    A --> F[Kubernetes Cluster]
    A --> G[Docker]
    B --> H[AWS CloudWatch/EC2/RDS]
    C --> I[Prometheus/Grafana/Datadog]
    C --> J[Sentry/New Relic]
    D --> K[GitHub/GitLab]
    E --> L[Elasticsearch/Splunk]
    E --> M[Snowflake/PostgreSQL/BigQuery]
```

## Credential Management

All credentials should be stored securely using vault references:

```json theme={null}
{
  "tools": {
    "coralogix": {
      "api_key": "vault://secrets/coralogix-api-key"
    }
  }
}
```

<Warning>
  Never store credentials in plain text in configuration files.
</Warning>

### Vault Reference Format

```
vault://path/to/secret
```

IncidentFox supports:

* AWS Secrets Manager
* HashiCorp Vault
* Environment variables (for development)

## Quick Setup

<Steps>
  <Step title="Choose Your Data Sources">
    Identify which platforms you want IncidentFox to access
  </Step>

  <Step title="Create API Keys">
    Generate read-only API keys for each platform
  </Step>

  <Step title="Store in Vault">
    Add credentials to your secrets manager
  </Step>

  <Step title="Configure in Web UI">
    Add data source configuration in Team Console
  </Step>

  <Step title="Test Connection">
    Verify IncidentFox can access each data source
  </Step>
</Steps>

## Required Permissions

Each data source requires specific permissions. Generally, IncidentFox needs **read-only** access for investigation.

| Data Source   | Required Permissions                                 |
| ------------- | ---------------------------------------------------- |
| Coralogix     | API key with read access                             |
| AWS           | CloudWatch read, EC2 describe, RDS read, Lambda read |
| Kubernetes    | Pod logs, events, describe resources                 |
| GitHub        | Repo read, issues read, PRs read                     |
| Snowflake     | SELECT on relevant tables                            |
| Datadog       | API key + App key with read access                   |
| Prometheus    | Query access to /api/v1/query endpoint               |
| Grafana       | Viewer role, API key with read access                |
| Sentry        | Project read, issue read                             |
| Elasticsearch | Read access to indices                               |
| PostgreSQL    | SELECT on relevant tables                            |
| Docker        | Docker socket access or API access                   |

<Tip>
  **Principle of least privilege**: Only grant permissions that are necessary for investigation. IncidentFox doesn't need write access unless you enable auto-remediation.
</Tip>

## Data Flow

When IncidentFox investigates an incident:

1. **Agent determines** which data sources are relevant
2. **Tools are invoked** to query each data source
3. **Data is retrieved** and processed locally
4. **Results are correlated** across sources
5. **Findings are reported** back to the user

Data is:

* Retrieved on-demand (not continuously polled)
* Processed in-memory (not stored long-term)
* Filtered by time range (typically last 1-24 hours)

## Next Steps

<CardGroup cols={2}>
  <Card title="Coralogix" icon="chart-line" href="/data-sources/coralogix">
    Connect your Coralogix account
  </Card>

  <Card title="Snowflake" icon="snowflake" href="/data-sources/snowflake">
    Set up data enrichment
  </Card>

  <Card title="AWS" icon="aws" href="/data-sources/aws">
    Configure AWS access
  </Card>

  <Card title="Kubernetes" icon="dharmachakra" href="/data-sources/kubernetes">
    Connect to your clusters
  </Card>
</CardGroup>