knet/docs/grafana-integration.md

# Grafana Integration Guide

This guide shows how to visualize your tracking data with Grafana using the new API endpoints.

## Authentication

In production, all tracking endpoints use **Basic Authentication** via Echo's built-in middleware:
- **Username:** `api` (or set via `API_USERNAME` environment variable)
- **Password:** Your API token (set via `API_TOKEN` environment variable)

**Example:**
```bash
curl -u api:your-api-token http://your-server:54321/api/metrics
```

The authentication uses Echo's standard `middleware.BasicAuth` for reliability and security.

## API Structure for Grafana

### 1. Prometheus Metrics Endpoint
**URL:** `/api/metrics?period=24h`

Returns Prometheus-style metrics that Grafana can scrape:
```
# HELP http_requests_total Total HTTP requests
# TYPE http_requests_total counter
http_requests_total 1247

# HELP http_unique_users_total Unique users  
# TYPE http_unique_users_total gauge
http_unique_users_total 234
```

### 2. Time Series Data Endpoint
**URL:** `/api/tracking/timeseries?target=hits&interval=1h&from=2025-08-13&to=2025-08-14`

Returns data in Grafana's expected format:
```json
[
  {
    "target": "hits",
    "datapoints": [
      [42, 1628856000000],
      [38, 1628859600000],
      [51, 1628863200000]
    ]
  }
]
```

### 3. Grafana Query Endpoint (SimpleJSON)
**URL:** `/api/tracking/query` (POST)

Handles Grafana's native query format for more complex dashboards.

## Grafana Setup Options

### Option 1: Prometheus Data Source (Recommended)

1. **Add Prometheus Data Source** in Grafana
2. **Set URL** to: `http://your-server:54321` (note: no `/api/metrics` suffix!)
3. **Set Basic Auth**: Username: `api`, Password: `your-api-token`
4. **Set Scrape Interval**: 30s or 1m

Grafana will automatically query `/api/v1/query` and `/api/v1/query_range` endpoints.

**Dashboard Queries:**
```promql
# Total requests
http_requests_total

# Unique users
http_unique_users_total

# Unique endpoints
http_unique_endpoints_total

# Request rate (per minute)
rate(http_requests_total[5m]) * 60
```

### Option 2: JSON API Data Source

1. **Install** the [JSON API Grafana Plugin](https://grafana.com/grafana/plugins/marcusolsson-json-datasource/)
2. **Add JSON API Data Source**
3. **Set URL** to: `http://your-server:54321/api/tracking/timeseries`
4. **Set Basic Auth**: Username: `api`, Password: `your-api-token`

**Query Examples:**
- `hits` - Total page hits over time
- `users` - Unique users over time  
- `endpoints` - Unique endpoints accessed

### Option 3: SimpleJSON Data Source (Advanced)

For complex dashboards with custom queries:

1. **Install** the [SimpleJSON Data Source](https://grafana.com/grafana/plugins/grafana-simple-json-datasource/)
2. **Set URL** to: `http://your-server:54321/api/tracking/query`
3. **Configure** custom metrics:
   - `http_requests` - Request time series
   - `unique_users` - User count time series
   - `top_endpoints` - Top endpoints table

## Sample Dashboard Panels

### 1. Traffic Overview (Single Stat)
```json
{
  "targets": [
    {
      "target": "hits",
      "refId": "A"
    }
  ],
  "title": "Total Page Views (24h)",
  "type": "singlestat"
}
```

### 2. Traffic Trends (Graph)
```json
{
  "targets": [
    {
      "target": "hits",
      "refId": "A"
    },
    {
      "target": "users", 
      "refId": "B"
    }
  ],
  "title": "Traffic Over Time",
  "type": "graph"
}
```

### 3. Top Endpoints (Table)
```json
{
  "targets": [
    {
      "target": "top_endpoints",
      "refId": "A"
    }
  ],
  "title": "Most Popular Pages",
  "type": "table"
}
```

## Environment Variables

Set these for production:
```bash
export API_TOKEN="your-secure-api-token"
export API_USERNAME="api"  # Optional, defaults to "api"
export BUILD_NUMBER="prod-v1.0"
```

## Grafana Dashboard JSON

Here's a complete dashboard configuration:

```json
{
  "dashboard": {
    "title": "Website Analytics",
    "panels": [
      {
        "title": "Page Views (24h)",
        "type": "singlestat",
        "targets": [{"target": "hits"}],
        "gridPos": {"h": 8, "w": 6, "x": 0, "y": 0}
      },
      {
        "title": "Unique Users (24h)", 
        "type": "singlestat",
        "targets": [{"target": "users"}],
        "gridPos": {"h": 8, "w": 6, "x": 6, "y": 0}
      },
      {
        "title": "Traffic Trends",
        "type": "graph",
        "targets": [
          {"target": "hits", "refId": "A"},
          {"target": "users", "refId": "B"}
        ],
        "gridPos": {"h": 8, "w": 12, "x": 0, "y": 8}
      }
    ],
    "time": {
      "from": "now-24h",
      "to": "now"
    },
    "refresh": "30s"
  }
}
```

## Benefits of This Approach

1. **Multiple Grafana Data Sources** - Choose what works best for you
2. **Real-time Updates** - Data refreshes automatically  
3. **Flexible Time Ranges** - Hour, day, week, month views
4. **Standard Formats** - Works with existing Grafana plugins
5. **Authentication** - Secured with Basic Auth in production

## Quick Start

1. Start your server: `./tmp/main`
2. Test metrics endpoint: `curl http://localhost:54321/api/metrics`
3. Add Prometheus data source in Grafana pointing to `/api/metrics`
4. Create dashboard with `http_requests_total` and `http_unique_users_total` queries
5. Set refresh interval to 30 seconds

Your analytics will now update in real-time in Grafana! 📊