knet/docs/grafana-integration.md

# Grafana Integration Guide

This guide shows how to visualize your application metrics with Grafana using **real Prometheus**.

## Why Prometheus?

We've replaced custom tracking with actual Prometheus because:
- ✅ **Industry standard** - Battle-tested metrics collection
- ✅ **Rich ecosystem** - Thousands of existing dashboards  
- ✅ **Better performance** - Optimized time-series storage
- ✅ **Less code** - No custom database or API endpoints
- ✅ **Standard metrics** - HTTP requests, response times, etc.

## Available Metrics

### Automatic Metrics (from echo-contrib/prometheus)
- `http_requests_total` - Total HTTP requests by method/status/endpoint
- `http_request_duration_seconds` - Request duration histogram
- `http_request_size_bytes` - Request size histogram  
- `http_response_size_bytes` - Response size histogram

### Custom Application Metrics
- `knet_page_views_total{route="/"}` - Page views by route
- `knet_page_views_total{route="/draw"}` - Draw page views
- `knet_box_clicks_total{position="1"}` - Box clicks by position
- `knet_websocket_connections` - Active WebSocket connections

## Authentication

The `/metrics` endpoint uses **Basic Authentication** in production:
- **Username:** `api` (or set via `API_USERNAME` environment variable)  
- **Password:** Your API token (set via `API_TOKEN` environment variable)

**Example:**
```bash
curl -u api:your-api-token http://your-server:54321/metrics
```

## Grafana Setup

### Prometheus Data Source (Recommended)

1. **Add Prometheus Data Source** in Grafana
2. **Set URL** to: `http://your-server:54321` 
3. **Set Basic Auth**: Username: `api`, Password: `your-api-token`
4. **Set Scrape Interval**: 15s (Prometheus scrapes our `/metrics` endpoint)

That's it! Grafana will automatically discover all available metrics.

### Sample Dashboard Queries

**Traffic Overview:**
```promql
# Total requests per minute
rate(http_requests_total[1m]) * 60

# Page views
knet_page_views_total

# Request duration 95th percentile  
histogram_quantile(0.95, rate(http_request_duration_seconds_bucket[5m]))

# Error rate
rate(http_requests_total{status=~"4.."}[5m]) / rate(http_requests_total[5m]) * 100
```

**Application Metrics:**
```promql
# Box interactions
rate(knet_box_clicks_total[5m])

# Active WebSocket connections
knet_websocket_connections

# Most popular pages
topk(10, rate(knet_page_views_total[1h]))
```

## Dashboard Examples

### 1. HTTP Overview Panel
- **Metric:** `rate(http_requests_total[1m]) * 60`
- **Type:** Graph  
- **Title:** "Requests per Minute"

### 2. Response Time Panel
- **Metric:** `histogram_quantile(0.95, rate(http_request_duration_seconds_bucket[5m]))`
- **Type:** Single Stat
- **Title:** "95th Percentile Response Time"

### 3. Page Views Panel  
- **Metric:** `knet_page_views_total`
- **Type:** Graph
- **Title:** "Page Views by Route"

### 4. Box Clicks Heatmap
- **Metric:** `rate(knet_box_clicks_total[5m])`
- **Type:** Graph
- **Title:** "Box Clicks per Second"

## Environment Variables

Set these for production:
```bash
export API_TOKEN="your-secure-api-token"
export API_USERNAME="api"  # Optional, defaults to "api"
export BUILD_NUMBER="prod-v1.0"  # Enables production mode
```

## Benefits of This Approach

1. **Standard Prometheus** - Works with all Grafana features
2. **Rich Metrics** - HTTP performance + custom business metrics
3. **No Database** - Prometheus handles storage and retention
4. **Better Performance** - Optimized time-series queries
5. **Alerting Ready** - Use Prometheus AlertManager
6. **Ecosystem** - Tons of existing dashboards and tools

## Persistent Storage Setup

### Option 1: Docker Compose (Recommended)

For persistent metrics storage, run a Prometheus server alongside your app:

```bash
# 1. Copy your API token to .env
cp .env.example .env
# Edit .env and set your API_TOKEN

# 2. Start the full monitoring stack
docker-compose up -d

# 3. Access services
# - Your app: http://localhost:54321
# - Prometheus: http://localhost:9090  
# - Grafana: http://localhost:3000 (admin/admin)
```

**What this gives you:**
- ✅ **Persistent metrics** - Data survives restarts
- ✅ **30-day retention** - Configurable in docker-compose.yml
- ✅ **Grafana pre-configured** - Points to Prometheus automatically
- ✅ **Production ready** - Proper volumes and networking

### Option 2: Standalone Prometheus

If you prefer to run Prometheus separately:

```bash
# 1. Update prometheus.yml with your API token
# 2. Start Prometheus
prometheus --config.file=prometheus.yml --storage.tsdb.path=./prometheus_data

# 3. Configure Grafana to point to http://localhost:9090
```

### Grafana Data Source Configuration

With persistent Prometheus:
1. **Add Prometheus Data Source** in Grafana  
2. **Set URL** to: `http://prometheus:9090` (Docker) or `http://localhost:9090` (standalone)
3. **No authentication needed** - Prometheus handles the API token
4. **Query retention**: Up to 30 days of historical data

## Quick Start

1. **Start your server:** `./tmp/main`
2. **Test metrics:** `curl -u api:your-token http://localhost:54321/metrics`
3. **Add Prometheus data source** in Grafana: `http://localhost:54321`
4. **Create dashboard** with queries like `http_requests_total`
5. **Enjoy real-time metrics!** 📊

Your analytics are now powered by industry-standard Prometheus! <20>