knet/docs/grafana-integration.md

Grafana Integration Guide

This guide shows how to visualize your tracking data with Grafana using the new API endpoints.

Authentication

In production, all tracking endpoints use Basic Authentication via Echo's built-in middleware:

• Username: api (or set via API_USERNAME environment variable)
• Password: Your API token (set via API_TOKEN environment variable)

Example:

curl -u api:your-api-token http://your-server:54321/api/metrics

The authentication uses Echo's standard middleware.BasicAuth for reliability and security.

API Structure for Grafana

1. Prometheus Metrics Endpoint

URL: /api/metrics?period=24h

Returns Prometheus-style metrics that Grafana can scrape:

# HELP http_requests_total Total HTTP requests
# TYPE http_requests_total counter
http_requests_total 1247

# HELP http_unique_users_total Unique users  
# TYPE http_unique_users_total gauge
http_unique_users_total 234

2. Time Series Data Endpoint

URL: /api/tracking/timeseries?target=hits&interval=1h&from=2025-08-13&to=2025-08-14

Returns data in Grafana's expected format:

[
  {
    "target": "hits",
    "datapoints": [
      [42, 1628856000000],
      [38, 1628859600000],
      [51, 1628863200000]
    ]
  }
]

3. Grafana Query Endpoint (SimpleJSON)

URL: /api/tracking/query (POST)

Handles Grafana's native query format for more complex dashboards.

Grafana Setup Options

Option 1: Prometheus Data Source (Recommended)

1. Add Prometheus Data Source in Grafana
2. Set URL to: http://your-server:54321 (note: no /api/metrics suffix!)
3. Set Basic Auth: Username: api, Password: your-api-token
4. Set Scrape Interval: 30s or 1m

Grafana will automatically query /api/v1/query and /api/v1/query_range endpoints.

Dashboard Queries:

# Total requests
http_requests_total

# Unique users
http_unique_users_total

# Unique endpoints
http_unique_endpoints_total

# Request rate (per minute)
rate(http_requests_total[5m]) * 60

Option 2: JSON API Data Source

1. Install the JSON API Grafana Plugin
2. Add JSON API Data Source
3. Set URL to: http://your-server:54321/api/tracking/timeseries
4. Set Basic Auth: Username: api, Password: your-api-token

Query Examples:

• hits - Total page hits over time
• users - Unique users over time
• endpoints - Unique endpoints accessed

Option 3: SimpleJSON Data Source (Advanced)

For complex dashboards with custom queries:

1. Install the SimpleJSON Data Source
2. Set URL to: http://your-server:54321/api/tracking/query
3. Configure custom metrics:
   • http_requests - Request time series
   • unique_users - User count time series
   • top_endpoints - Top endpoints table

Sample Dashboard Panels

1. Traffic Overview (Single Stat)

{
  "targets": [
    {
      "target": "hits",
      "refId": "A"
    }
  ],
  "title": "Total Page Views (24h)",
  "type": "singlestat"
}

2. Traffic Trends (Graph)

{
  "targets": [
    {
      "target": "hits",
      "refId": "A"
    },
    {
      "target": "users", 
      "refId": "B"
    }
  ],
  "title": "Traffic Over Time",
  "type": "graph"
}

3. Top Endpoints (Table)

{
  "targets": [
    {
      "target": "top_endpoints",
      "refId": "A"
    }
  ],
  "title": "Most Popular Pages",
  "type": "table"
}

Environment Variables

Set these for production:

export API_TOKEN="your-secure-api-token"
export API_USERNAME="api"  # Optional, defaults to "api"
export BUILD_NUMBER="prod-v1.0"

Grafana Dashboard JSON

Here's a complete dashboard configuration:

{
  "dashboard": {
    "title": "Website Analytics",
    "panels": [
      {
        "title": "Page Views (24h)",
        "type": "singlestat",
        "targets": [{"target": "hits"}],
        "gridPos": {"h": 8, "w": 6, "x": 0, "y": 0}
      },
      {
        "title": "Unique Users (24h)", 
        "type": "singlestat",
        "targets": [{"target": "users"}],
        "gridPos": {"h": 8, "w": 6, "x": 6, "y": 0}
      },
      {
        "title": "Traffic Trends",
        "type": "graph",
        "targets": [
          {"target": "hits", "refId": "A"},
          {"target": "users", "refId": "B"}
        ],
        "gridPos": {"h": 8, "w": 12, "x": 0, "y": 8}
      }
    ],
    "time": {
      "from": "now-24h",
      "to": "now"
    },
    "refresh": "30s"
  }
}

Benefits of This Approach

1. Multiple Grafana Data Sources - Choose what works best for you
2. Real-time Updates - Data refreshes automatically
3. Flexible Time Ranges - Hour, day, week, month views
4. Standard Formats - Works with existing Grafana plugins
5. Authentication - Secured with Basic Auth in production

Quick Start

1. Start your server: ./tmp/main
2. Test metrics endpoint: curl http://localhost:54321/api/metrics
3. Add Prometheus data source in Grafana pointing to /api/metrics
4. Create dashboard with http_requests_total and http_unique_users_total queries
5. Set refresh interval to 30 seconds

Your analytics will now update in real-time in Grafana! 📊