Ops mode

What you'll learn: Transform infrastructure operations from complex CLI commands to natural conversations. Discover how Scoutflo's Ops Mode enables you to chat with your cluster, query GitHub, investigate Sentry issues, and explore ELK logs using plain English.

What Ops Mode is:

Scoutflo's Ops Mode revolutionizes infrastructure operations by providing a conversational interface that lets you interact with your entire technology stack using natural language, eliminating the need to remember complex commands and syntax.

Chat naturally with Kubernetes clusters, databases, monitoring systems, and development tools
Query multiple systems simultaneously with intelligent agent routing
Get contextualized responses that understand your infrastructure relationships
Execute operations safely with built-in guardrails and confirmation prompts
Learn and improve through conversational feedback and usage patterns

Scoutflo Ops Mode acts as your intelligent operations assistant that speaks your language while understanding your infrastructure:

How Ops Mode Works

Scoutflo's Ops Mode operates through an intelligent conversation routing system that understands your intent, selects the appropriate systems, and translates natural language into precise technical operations:

Stage 1: Intent Recognition (2 seconds)

Natural language processing to understand operational intent
Context awareness based on previous conversations and current environment
Multi-system query detection for complex operations
Safety classification for destructive vs read-only operations

Stage 2: Agent Selection & Routing (3 seconds)

Intelligent selection of appropriate system agents based on query content
Multi-agent coordination for queries spanning multiple systems
Authentication and authorization validation for target systems
Query optimization and parameter extraction

Stage 3: Execution & Response (5-15 seconds)

Parallel execution across selected systems when applicable
Real-time streaming of results for long-running operations
Intelligent response formatting and context preservation
Follow-up question suggestions and related operation recommendations

Key Benefits & Metrics

Production Results: These metrics come from engineering teams using Scoutflo Ops Mode for daily operations.

  ┌─────────────────────┬─────────────────┬───────────────┬───────────────────┐
  │       Metric        │ CLI Operations  │ Conversational│    Improvement    │
  ├─────────────────────┼─────────────────┼───────────────┼───────────────────┤
  │ Query Time          │ 3-8 minutes     │ 15 seconds    │ 94% faster        │
  ├─────────────────────┼─────────────────┼───────────────┼───────────────────┤
  │ Context Switching   │ 12 tools/query  │ 1 interface   │ 92% reduction     │
  ├─────────────────────┼─────────────────┼───────────────┼───────────────────┤
  │ Command Accuracy    │ 78% correct     │ 96% correct   │ 23% improvement   │
  ├─────────────────────┼─────────────────┼───────────────┼───────────────────┤
  │ New Engineer Ramp   │ 4-6 weeks       │ 3-5 days      │ 85% faster        │
  ├─────────────────────┼─────────────────┼───────────────┼───────────────────┤
  │ Operations Knowledge│ Individual-based│ Team-shared   │ 78% improvement   │
  └─────────────────────┴─────────────────┴───────────────┴───────────────────┘

How it works: Instead of memorizing complex CLI commands and switching between multiple tools, you simply ask questions or state what you want to accomplish in plain English, and Ops Mode handles the translation to appropriate system commands.

Intent understanding that recognizes operational goals from conversational language
Multi-system awareness that routes queries to the right tools automatically
Context preservation that remembers your session and builds on previous interactions
Safety guardrails that prevent accidental destructive operations

Example: "Show me pods that are using too much memory in production" automatically queries Kubernetes metrics, identifies memory-intensive pods, and presents results with context about resource limits and recommendations.

Detailed Agent Capabilities

Kubernetes Agent:

Pod Operations: Status checks, log retrieval, resource utilization analysis
Deployment Management: Rollout status, replica scaling, health monitoring
Resource Investigation: CPU/memory usage, storage analysis, network diagnostics
Cluster Health: Node status, namespace resource allocation, event investigation

Sentry Agent:

Error Analysis: Exception tracking, error rate trends, user impact assessment
Performance Monitoring: Transaction analysis, slow query identification, bottleneck detection
Release Correlation: Error patterns across deployments, regression identification
User Experience: Performance impact on specific user segments or geographic regions

GitHub Agent:

Code Investigation: Recent changes analysis, blame tracking, dependency exploration
Pull Request Analysis: Change impact assessment, reviewer history, deployment correlation
Repository Intelligence: Code quality trends, contributor patterns, technical debt analysis
Deployment History: Release correlation with incidents, rollback analysis

Conversation Types & Use Cases

Operational Conversation Categories

Troubleshooting conversations that combine multiple data sources:

Common Patterns:

"Why is service X responding slowly?"
"Show me errors from the last hour related to authentication"
"What changed in the payment service since yesterday?"
"Are there any pods experiencing memory issues?"

Example Conversation:

User: "The checkout API seems slow today, what's going on?"

Ops Mode: "I'll investigate the checkout API performance across multiple systems..."

[Checking Kubernetes pod health...]
✅ All 12 checkout-api pods are running and healthy
📊 Average CPU: 45%, Memory: 67% (within normal ranges)

[Analyzing Sentry performance data...]
🚨 Response time increased 340% in the last 2 hours
📈 P95 latency: 2.8s (baseline: 0.8s)
🔍 Most affected endpoint: POST /api/checkout/process (67% of slow requests)

[Querying ELK for related errors...]
⚠️ Found 2,847 database timeout errors in last hour
🕒 Pattern shows timeouts correlating with payment gateway calls

[Checking recent GitHub changes...]
📝 Payment service v2.3.1 deployed 3 hours ago
🔍 Changes include new payment provider integration

Recommended next steps:
1. Check payment gateway API response times
2. Review database connection pool configuration
3. Consider rolling back payment service to v2.3.0

Would you like me to investigate any of these areas further?

Infrastructure operations and resource optimization:

Common Patterns:

"Scale up the user service to 10 replicas"
"Show me which services are using the most memory"
"List all pods that have been restarted recently"
"What's the current resource utilization across clusters?"

Example Conversation:

User: "We're expecting high traffic for the sale event. Can you help me prepare the infrastructure?"

Ops Mode: "I'll help you prepare for the traffic spike. Let me analyze current capacity and scaling options..."

[Current Resource Analysis...]
📊 Current production cluster utilization:
   • CPU: 67% average (45 nodes)
   • Memory: 72% average
   • Network: 23% utilization

🎯 Services likely to need scaling:
   • product-api: 8 replicas, 89% CPU usage
   • search-service: 5 replicas, 78% memory usage
   • user-service: 12 replicas, 45% CPU usage (good)

[Historical Traffic Analysis...]
📈 Last sale event (3 months ago):
   • Traffic increased 340% over 4-hour period
   • product-api scaled to 25 replicas
   • search-service scaled to 18 replicas
   • Database connections peaked at 387 (current limit: 400)

Recommended preparation:
1. Pre-scale product-api to 15 replicas
2. Pre-scale search-service to 12 replicas
3. Increase database connection pool to 600
4. Enable aggressive autoscaling (target 60% CPU)

Would you like me to execute these scaling operations now?

Performance monitoring and trend analysis:

Common Patterns:

"How has error rate changed over the past week?"
"Show me SLA compliance for critical services"
"What are the slowest API endpoints right now?"
"Compare performance between staging and production"

Example Conversation:

User: "Give me a health check of our critical services"

Ops Mode: "Running comprehensive health analysis across all critical services..."

[SLA Compliance Analysis...]
✅ Payment API: 99.97% uptime (SLA: 99.95%)
✅ Authentication: 99.99% uptime (SLA: 99.9%)
⚠️  Search Service: 99.89% uptime (SLA: 99.95%) - BELOW TARGET
❌ Notification Service: 99.23% uptime (SLA: 99.9%) - SLA BREACH

[Performance Metrics - Last 24h...]
🚀 Fastest services:
   • User API: 89ms avg response time
   • Product Catalog: 124ms avg response time

🐌 Slowest services:
   • Analytics API: 2.3s avg response time (+340% from baseline)
   • Report Generation: 8.7s avg response time (within expected range)

[Error Rate Analysis...]
📊 Error rates trending:
   • Overall: 0.34% (target <0.5%) ✅
   • Search Service: 1.2% (trending up 45% this week) ⚠️
   • Analytics API: 2.1% (database timeout errors) 🚨

Critical issues requiring attention:
1. Search Service SLA breach - investigate indexing performance
2. Analytics API database timeouts - review query optimization
3. Notification Service reliability - check message queue health

Would you like me to deep-dive into any of these issues?

Query Examples & Patterns

Effective Conversation Patterns

Kubernetes and infrastructure management conversations:

Resource Investigation:

User: "Show me pods using more than 80% memory"

Ops Mode:
🔍 Found 7 pods exceeding 80% memory usage:

📊 Critical (>95% memory):
• payment-api-7b8c9 (Node: prod-node-12): 97% (1.94GB/2GB)
• analytics-worker-3f2a1 (Node: prod-node-08): 96% (7.68GB/8GB)

⚠️ High (80-95% memory):
• search-index-9d4e2: 89% (3.56GB/4GB)
• user-session-cache-1a7b: 87% (6.96GB/8GB)
• notification-queue-8c3f: 84% (1.68GB/2GB)

🔧 Recommended actions:
1. payment-api-7b8c9: Potential memory leak - check recent deployments
2. analytics-worker-3f2a1: Expected high usage for batch processing
3. search-index-9d4e2: Consider increasing memory limits

Would you like me to investigate the memory leak in payment-api?

Scaling Operations:

User: "Scale the product service to handle 3x current traffic"

Ops Mode:
📊 Current product service configuration:
• Current replicas: 8
• Average CPU usage: 67%
• Average memory usage: 72%
• Current RPS: ~2,400

📈 For 3x traffic (7,200 RPS):
• Recommended replicas: 24 (3x scaling)
• Expected resource usage: ~75% CPU, ~80% memory
• Cluster capacity check: ✅ Sufficient nodes available

🔧 Scaling plan:
1. Gradually scale to 16 replicas (monitor for 5 minutes)
2. Scale to 24 replicas if performance is stable
3. Enable horizontal pod autoscaling (target: 70% CPU)

Execute scaling plan now? (y/n)

Debugging and error analysis conversations:

Multi-System Error Investigation:

User: "Investigate the spike in 500 errors for the API gateway"

Ops Mode:
🚨 API Gateway Error Analysis:

[Error Pattern Analysis - Last 2 hours]
📈 500 Error spike detected:
• Normal rate: 12 errors/hour
• Current rate: 247 errors/hour (20x increase)
• Peak time: 14:23 UTC (418 errors in 10 minutes)

[Service Health Check]
🔍 API Gateway pods: 12/12 healthy, no restarts
🔍 Load balancer: Normal traffic distribution
🔍 Network: No connectivity issues detected

[Downstream Service Analysis]
❌ User Service: Error rate 15.3% (normal: 0.2%)
✅ Product Service: Healthy (0.1% errors)
✅ Payment Service: Healthy (0.3% errors)

[Recent Changes Investigation]
📝 GitHub analysis:
• User Service v2.4.1 deployed at 14:18 UTC
• API Gateway configuration updated at 14:15 UTC

🔍 Root cause correlation: 94% confidence
• User Service deployment introduced database connection leak
• Connection pool exhaustion causing cascading failures

Recommended immediate action:
1. Rollback User Service to v2.4.0
2. Restart user service pods to clear connection leaks
3. Monitor error rates for 10 minutes

Should I execute the rollback?

Sentry Error Deep-Dive:

User: "Why are users reporting login issues?"

Ops Mode:
🔍 Login Issue Investigation:

[Sentry Error Analysis]
🚨 Authentication errors increased 450% in last hour:
• Total errors: 1,247 (affecting 892 unique users)
• Primary error: "TokenValidationError" (78% of failures)
• Secondary: "DatabaseConnectionTimeout" (22% of failures)

[User Impact Analysis]
📊 Affected user patterns:
• Geographic: 67% from EU region, 23% from US East
• Platform: 89% mobile app, 11% web interface
• Account type: 78% premium users, 22% free users

[System Correlation]
🔍 ELK log analysis shows:
• Redis cluster connectivity issues in EU region
• Session tokens failing validation due to clock skew
• Database connection pool saturated in auth service

[Timeline Reconstruction]
🕒 Issue progression:
• 13:45 UTC: Redis EU cluster maintenance began
• 13:47 UTC: Auth service failover to US Redis cluster
• 13:52 UTC: Clock synchronization issues detected
• 13:55 UTC: Token validation failures cascade

💡 Solution identified:
1. Fix NTP configuration on auth service pods
2. Flush invalid tokens from Redis cache
3. Restart auth service to clear connection pool

Estimated fix time: 5 minutes. Proceed?

Performance monitoring and optimization conversations:

Application Performance Investigation:

User: "Our checkout process feels slow, what's the bottleneck?"

Ops Mode:
🐌 Checkout Performance Analysis:

[Current Performance Metrics]
📊 Checkout API endpoint timings:
• P50: 890ms (baseline: 340ms) - 162% slower
• P95: 3.2s (baseline: 1.1s) - 191% slower
• P99: 7.8s (baseline: 2.3s) - 239% slower

[Bottleneck Identification]
🔍 Performance breakdown:
• Database queries: 2,100ms avg (78% of total time)
• Payment gateway: 280ms avg (10% of total time)
• Internal logic: 95ms avg (3% of total time)
• External APIs: 245ms avg (9% of total time)

[Database Performance Deep-Dive]
🔍 Slow query analysis:
• Query: SELECT * FROM orders WHERE... (1,890ms avg)
• Execution plan: Full table scan on 2.4M records
• Missing index on user_id + created_at columns
• Connection pool: 89/100 connections active

[Historical Comparison]
📈 Performance degradation timeline:
• 3 days ago: Normal performance
• 2 days ago: 15% slower (order volume increased 40%)
• Yesterday: 45% slower (peak traffic sustained)
• Today: 162% slower (database reaching limits)

🔧 Optimization recommendations:
1. Add composite index: user_id + created_at (immediate impact)
2. Scale database read replicas from 2 to 4 (medium term)
3. Implement query result caching (long term)
4. Optimize checkout query to limit returned fields

Immediate fix estimated impact: 70% performance improvement
Should I generate the database migration script?

Resource Utilization Analysis:

User: "Show me which services are consuming the most resources"

Ops Mode:
📊 Resource Consumption Analysis:

[Top Resource Consumers - Last 24h]
🥇 Highest CPU Usage:
1. analytics-processor: 89% avg (24 cores allocated)
2. ml-training-job: 76% avg (16 cores allocated)
3. image-processing: 71% avg (8 cores allocated)

🥇 Highest Memory Usage:
1. search-index: 15.2GB avg (16GB allocated - 95%)
2. analytics-processor: 12.8GB avg (16GB allocated - 80%)
3. user-cache: 7.9GB avg (8GB allocated - 98%)

[Resource Efficiency Analysis]
✅ Efficient services (good resource utilization):
• payment-api: 67% CPU, 72% memory (optimal range)
• notification-service: 45% CPU, 58% memory (good)

⚠️ Over-provisioned services (wasted resources):
• admin-portal: 12% CPU, 23% memory (consider downsizing)
• backup-service: 8% CPU, 15% memory (oversized)

🚨 Under-provisioned services (at risk):
• search-index: 95% memory utilization (needs more memory)
• user-cache: 98% memory utilization (critical - scale up)

[Cost Optimization Opportunities]
💰 Potential monthly savings: $4,200
• Downsize over-provisioned services: $1,800/month
• Right-size memory allocations: $1,500/month
• Optimize instance types: $900/month

Would you like me to generate resource optimization recommendations?

Best Practices for Ops Conversations

Effective Query Construction

How to ask clear, actionable questions:

Good Query Patterns:

✅ Specific and actionable:
"Show me pods with high memory usage in production"
"Scale the user service to 15 replicas"
"Find errors in payment service from the last hour"

✅ Context-aware:
"Why is checkout slow compared to yesterday?"
"What changed in authentication since the incident?"
"Compare performance between staging and production"

✅ Investigation-focused:
"Investigate the spike in database errors"
"Analyze the impact of yesterday's deployment"
"Find the root cause of login failures"

Query Patterns to Avoid:

❌ Too vague:
"Something is wrong with the system"
"Check if everything is okay"
"Why is it slow?"

❌ Missing context:
"Scale the service" (which service?)
"Show me errors" (from where? when?)
"Fix the problem" (what problem?)

❌ Overly complex:
"Scale user service to 10 replicas and check memory usage while monitoring error rates and comparing to baseline from last week unless staging shows issues"

Building effective conversation context:

Context Preservation Example:

User: "Show me pods with high CPU usage"

Ops Mode: "Found 5 pods with >80% CPU usage:
• payment-api-7b8c9: 89% CPU
• search-service-3f2a: 87% CPU
• analytics-worker-1d4e: 92% CPU
..."

User: "Check the memory usage for these pods"
# Ops Mode automatically knows "these pods" refers to the 5 high-CPU pods

Ops Mode: "Memory usage for the 5 high-CPU pods:
• payment-api-7b8c9: 67% memory (normal)
• search-service-3f2a: 94% memory (high)
• analytics-worker-1d4e: 78% memory (normal)
..."

User: "Scale up the one with high memory"
# Context preserved: refers to search-service-3f2a

Ops Mode: "Scaling search-service-3f2a from 5 to 8 replicas to address high memory usage..."

Multi-Turn Investigation Pattern:

User: "Investigate checkout performance issues"

Ops Mode: [Analyzes checkout API performance, finds slow database queries]

User: "Show me those slow queries"
# Context: knows we're investigating checkout performance
# Context: knows "those queries" refers to the slow database queries found

User: "Who committed changes to that code recently?"
# Context: checkout performance + slow queries + specific code area

User: "Can we rollback that change?"
# Context: full investigation chain preserved

Context-Aware Suggestions:

# Automatic context suggestion
context_suggestions = {
    'after_error_investigation': [
        "Would you like me to check recent deployments?",
        "Should I analyze similar errors from the past?",
        "Want me to check the health of related services?"
    ],

    'after_performance_analysis': [
        "Would you like me to compare with historical baselines?",
        "Should I check resource utilization trends?",
        "Want me to analyze user impact?"
    ],

    'after_scaling_operation': [
        "Should I monitor the scaling progress?",
        "Would you like me to set up alerts for this service?",
        "Want me to check if other services need similar scaling?"
    ]
}

Safe operations with appropriate confirmations:

Safety Classification System:

# Operation safety levels
operation_safety:
  read_only:
    confirmation_required: false
    examples:
      - "show pods"
      - "get logs"
      - "describe deployment"
      - "analyze errors"

  low_risk:
    confirmation_required: false
    examples:
      - "scale up replicas"
      - "restart healthy pods"
      - "enable autoscaling"

  medium_risk:
    confirmation_required: true
    confirmation_details: "basic"
    examples:
      - "scale down replicas"
      - "restart failing pods"
      - "apply configuration changes"

  high_risk:
    confirmation_required: true
    confirmation_details: "detailed"
    examples:
      - "delete resources"
      - "rollback deployments"
      - "modify production data"

  forbidden:
    allow: false
    examples:
      - "delete namespace kube-system"
      - "modify cluster-level resources"
      - "access sensitive secrets"

Confirmation Flow Examples:

User: "Scale down the analytics service to 2 replicas"

Ops Mode: "⚠️ Confirmation Required - Medium Risk Operation

Operation: Scale down analytics-service
Current replicas: 8
Target replicas: 2
Risk level: Medium (75% capacity reduction)

Impact assessment:
• Service capacity reduction: 75%
• Expected response time increase: ~200%
• Users potentially affected: ~500/hour during peak

This operation will reduce service capacity significantly.

Type 'yes' to confirm, 'no' to cancel, or 'details' for more information."

User: "yes"

Ops Mode: "✅ Scaling analytics-service to 2 replicas...
[Progress: ████████░░ 80%]
Successfully scaled to 2 replicas. Monitoring performance impact..."

Rollback Safety:

User: "Rollback the payment service"

Ops Mode: "🚨 High Risk Operation - Detailed Confirmation Required

Operation: Rollback payment-service deployment
Current version: v2.4.1 (deployed 2 hours ago)
Target version: v2.4.0 (last stable)

⚠️ Impact Analysis:
• Service: Critical payment processing
• Active transactions: 47 in progress
• Daily transaction volume: $2.3M
• Rollback duration: ~3-5 minutes

Prerequisites check:
✅ Target version v2.4.0 is available
✅ Database schema is backward compatible
✅ No breaking configuration changes
⚠️ 47 active transactions will be interrupted

Safety measures:
• Graceful shutdown: 30 second drain period
• Health check validation before traffic restoration
• Automatic rollback if health checks fail

Please confirm by typing: 'CONFIRM ROLLBACK payment-service v2.4.0'
Or type 'cancel' to abort."

Getting Started

Prerequisites

Infrastructure Access: Kubernetes clusters, monitoring systems, databases
API Credentials: Service accounts for GitHub, Sentry, monitoring platforms
Communication Platform: Slack, Teams, or web interface access
Basic Permissions: Read access to logs, metrics, and code repositories

Quick Setup

Agent Configuration

Connect and configure system agents

Permission Setup

Configure access controls and safety settings

Conversation Training

Learn effective interaction patterns

PreviousTroubleshooting Mode NextRBAC

Last updated 6 days ago

hashtagWhat Ops Mode is:

hashtagHow Ops Mode Works

hashtagKey Benefits & Metrics

hashtagDetailed Agent Capabilities

hashtagConversation Types & Use Cases

hashtagOperational Conversation Categories

hashtagQuery Examples & Patterns

hashtagEffective Conversation Patterns

hashtagBest Practices for Ops Conversations

hashtagEffective Query Construction

hashtagGetting Started

hashtagPrerequisites

hashtagQuick Setup

hashtag

What Ops Mode is:

How Ops Mode Works

Key Benefits & Metrics

Detailed Agent Capabilities

Conversation Types & Use Cases

Operational Conversation Categories

Query Examples & Patterns

Effective Conversation Patterns

Best Practices for Ops Conversations

Effective Query Construction

Getting Started

Prerequisites

Quick Setup