ELK Stack

Connect ELK (Elasticsearch / Logstash / Kibana) to Scoutflo to bring logs and search results

Once configured, Scoutflo can:

• Query Elasticsearch indices for relevant logs around incident timelines

• Correlate log patterns with Kubernetes events, Sentry errors, and cloud changes

• Use log excerpts as evidence in root‑cause analysis

---

✅ Pre requisites

Before setting up the ELK integration, ensure you have:

• An Elasticsearch cluster (cloud‑hosted or self‑hosted) with API access

• Administrative access to create API keys in Elasticsearch

• Access to Scoutflo’s Integrations tab in the platform UI

• For self‑hosted setups:

  • The Elasticsearch hostname / endpoint (for example, https://es.yourdomain.com:9200)

• Familiarity with your index patterns and retention settings

If you don’t have these, coordinate with your SRE / Platform team before proceeding.

---

🧩 Setup Guide

To connect ELK to Scoutflo, you’ll:

1. Create an API key in Elasticsearch

2. Configure the connection in Scoutflo

3. (Optional) Configure self‑hosted Elasticsearch

This enables Scoutflo’s AI engine to search your logs as part of incident investigations.

---

Step 1: Create an API key in Elasticsearch

1. Log in to your Elasticsearch cluster management interface.

2. Navigate to Management → Security Settings → API Keys.

3. Click Create API Key.

4. Configure the API key:

   • Name: Something descriptive, for example: Scoutflo AI SRE or Scoutflo Logs Integration

   • Role: Use a role with read access to the relevant indices. Common options:

     • logstash_reader (if using Logstash indices)

     • A custom role scoped to your log indices

     • elastic superuser (if no restrictions)

5. Click Create API key.

6. Copy the generated token immediately (it’s shown only once).

Store this token securely, for example:

• In the Scoutflo Integrations UI (recommended)

• As an environment variable: SCOUTFLO_ELK_API_TOKEN

Treat this API key like any other secret. Do not commit it to Git or share it in plain text.

---

Step 2: Configure ELK in Scoutflo

In the Scoutflo platform UI:

1. Navigate to Settings → Integrations → ELK.

2. Fill in the configuration fields:

Field	Description	Example
Elasticsearch URL	Full URL to your Elasticsearch endpoint	https://es.yourdomain.com:9200
API Key	Token from Step 1	eyJ2ZXIiOiI5LjAuMCIsImFkciI6W119...
Index pattern	Default indices to search (optional)	logs-*, app-logs-*

1. Click Test connection to validate:

   • Elasticsearch endpoint is reachable

   • API key authenticates successfully

   • Scoutflo can list indices or run a simple search

2. If the test succeeds, click Save to activate the integration.

---

Step 3: Configure self‑hosted Elasticsearch (optional)

The setup above already works for self‑hosted Elasticsearch. Just make sure:

1. The Elasticsearch URL includes:

   • Correct protocol (https:// recommended)

   • Correct hostname / port (es.yourdomain.com:9200)

2. The API key has read access to your log indices.

3. The Elasticsearch cluster is reachable from Scoutflo:

   • Check DNS resolution

   • Verify firewall rules (ports 9200 or 443)

   • Confirm TLS certificate is valid and trusted

If using a proxy or VPN, ensure Scoutflo’s outbound traffic can reach your Elasticsearch endpoint.

---

✅ Post‑setup validation

After completing the steps above, do these quick checks:

1. Test the connection in Scoutflo

   • In Integrations → ELK, run Test connection.

   • Confirm the status shows Connected.

2. Verify index access

   • From Scoutflo, list available indices or run a simple search.

   • Confirm your expected log indices are visible.

3. Check log flow in investigations

   • Trigger an incident or investigation that should generate logs.

   • Confirm that relevant log snippets appear in the Scoutflo investigation timeline.

---

⚠️ Most common failure points

• Invalid API key: Double‑check that the key is copied exactly (no extra spaces or missing characters). Regenerate in Elasticsearch if issues persist.

• Wrong or unreachable URL: Verify the Elasticsearch URL is correct and accessible from Scoutflo. Check DNS, ports (9200 or 443), and firewall rules.

• Insufficient permissions: Confirm the API key role has read access to your log indices. Test with a broader role temporarily if needed.

• No logs in investigations: Check that your index pattern matches the actual indices Scoutflo should search. Verify that logs are being generated around incident timestamps.