prometheus

2025-08-15 14:55:59 +02:00
parent fecbe91f5d
commit 45aa362789
6 changed files with 189 additions and 866 deletions
--- a/docs/grafana-integration.md
+++ b/docs/grafana-integration.md
@@ -1,154 +1,102 @@
 # Grafana Integration Guide

-This guide shows how to visualize your tracking data with Grafana using the new API endpoints.
+This guide shows how to visualize your application metrics with Grafana using **real Prometheus**.
+
+## Why Prometheus?
+
+We've replaced custom tracking with actual Prometheus because:
+- ✅ **Industry standard** - Battle-tested metrics collection
+- ✅ **Rich ecosystem** - Thousands of existing dashboards  
+- ✅ **Better performance** - Optimized time-series storage
+- ✅ **Less code** - No custom database or API endpoints
+- ✅ **Standard metrics** - HTTP requests, response times, etc.
+
+## Available Metrics
+
+### Automatic Metrics (from echo-contrib/prometheus)
+- `http_requests_total` - Total HTTP requests by method/status/endpoint
+- `http_request_duration_seconds` - Request duration histogram
+- `http_request_size_bytes` - Request size histogram  
+- `http_response_size_bytes` - Response size histogram
+
+### Custom Application Metrics
+- `knet_page_views_total{route="/"}` - Page views by route
+- `knet_page_views_total{route="/draw"}` - Draw page views
+- `knet_box_clicks_total{position="1"}` - Box clicks by position
+- `knet_websocket_connections` - Active WebSocket connections

 ## Authentication

-In production, all tracking endpoints use **Basic Authentication** via Echo's built-in middleware:
- **Username:** `api` (or set via `API_USERNAME` environment variable)
+The `/metrics` endpoint uses **Basic Authentication** in production:
+- **Username:** `api` (or set via `API_USERNAME` environment variable)  
 - **Password:** Your API token (set via `API_TOKEN` environment variable)

 **Example:**
 ```bash
-curl -u api:your-api-token http://your-server:54321/api/metrics
+curl -u api:your-api-token http://your-server:54321/metrics
 ```

-The authentication uses Echo's standard `middleware.BasicAuth` for reliability and security.
+## Grafana Setup

-## API Structure for Grafana
-
-### 1. Prometheus Metrics Endpoint
-**URL:** `/api/metrics?period=24h`
-
-Returns Prometheus-style metrics that Grafana can scrape:
-```
-# HELP http_requests_total Total HTTP requests
-# TYPE http_requests_total counter
-http_requests_total 1247
-
-# HELP http_unique_users_total Unique users  
-# TYPE http_unique_users_total gauge
-http_unique_users_total 234
-```
-
-### 2. Time Series Data Endpoint
-**URL:** `/api/tracking/timeseries?target=hits&interval=1h&from=2025-08-13&to=2025-08-14`
-
-Returns data in Grafana's expected format:
-```json
-[
-  {
-    "target": "hits",
-    "datapoints": [
-      [42, 1628856000000],
-      [38, 1628859600000],
-      [51, 1628863200000]
-    ]
-  }
-]
-```
-
-### 3. Grafana Query Endpoint (SimpleJSON)
-**URL:** `/api/tracking/query` (POST)
-
-Handles Grafana's native query format for more complex dashboards.
-
-## Grafana Setup Options
-
-### Option 1: Prometheus Data Source (Recommended)
+### Prometheus Data Source (Recommended)

 1. **Add Prometheus Data Source** in Grafana
-2. **Set URL** to: `http://your-server:54321` (note: no `/api/metrics` suffix!)
+2. **Set URL** to: `http://your-server:54321` 
 3. **Set Basic Auth**: Username: `api`, Password: `your-api-token`
-4. **Set Scrape Interval**: 30s or 1m
+4. **Set Scrape Interval**: 15s (Prometheus scrapes our `/metrics` endpoint)

-Grafana will automatically query `/api/v1/query` and `/api/v1/query_range` endpoints.
+That's it! Grafana will automatically discover all available metrics.

-**Dashboard Queries:**
+### Sample Dashboard Queries
+
+**Traffic Overview:**
 ```promql
-# Total requests
-http_requests_total
+# Total requests per minute
+rate(http_requests_total[1m]) * 60

-# Unique users
-http_unique_users_total
+# Page views
+knet_page_views_total

-# Unique endpoints
-http_unique_endpoints_total
+# Request duration 95th percentile  
+histogram_quantile(0.95, rate(http_request_duration_seconds_bucket[5m]))

-# Request rate (per minute)
-rate(http_requests_total[5m]) * 60
+# Error rate
+rate(http_requests_total{status=~"4.."}[5m]) / rate(http_requests_total[5m]) * 100
 ```

-### Option 2: JSON API Data Source
+**Application Metrics:**
+```promql
+# Box interactions
+rate(knet_box_clicks_total[5m])

-1. **Install** the [JSON API Grafana Plugin](https://grafana.com/grafana/plugins/marcusolsson-json-datasource/)
-2. **Add JSON API Data Source**
-3. **Set URL** to: `http://your-server:54321/api/tracking/timeseries`
-4. **Set Basic Auth**: Username: `api`, Password: `your-api-token`
+# Active WebSocket connections
+knet_websocket_connections

-**Query Examples:**
- `hits` - Total page hits over time
- `users` - Unique users over time  
- `endpoints` - Unique endpoints accessed
-
-### Option 3: SimpleJSON Data Source (Advanced)
-
-For complex dashboards with custom queries:
-
-1. **Install** the [SimpleJSON Data Source](https://grafana.com/grafana/plugins/grafana-simple-json-datasource/)
-2. **Set URL** to: `http://your-server:54321/api/tracking/query`
-3. **Configure** custom metrics:
-   - `http_requests` - Request time series
-   - `unique_users` - User count time series
-   - `top_endpoints` - Top endpoints table
-
-## Sample Dashboard Panels
-
-### 1. Traffic Overview (Single Stat)
-```json
-{
-  "targets": [
-    {
-      "target": "hits",
-      "refId": "A"
-    }
-  ],
-  "title": "Total Page Views (24h)",
-  "type": "singlestat"
-}
+# Most popular pages
+topk(10, rate(knet_page_views_total[1h]))
 ```

-### 2. Traffic Trends (Graph)
-```json
-{
-  "targets": [
-    {
-      "target": "hits",
-      "refId": "A"
-    },
-    {
-      "target": "users", 
-      "refId": "B"
-    }
-  ],
-  "title": "Traffic Over Time",
-  "type": "graph"
-}
-```
+## Dashboard Examples

-### 3. Top Endpoints (Table)
-```json
-{
-  "targets": [
-    {
-      "target": "top_endpoints",
-      "refId": "A"
-    }
-  ],
-  "title": "Most Popular Pages",
-  "type": "table"
-}
-```
+### 1. HTTP Overview Panel
+- **Metric:** `rate(http_requests_total[1m]) * 60`
+- **Type:** Graph  
+- **Title:** "Requests per Minute"
+
+### 2. Response Time Panel
+- **Metric:** `histogram_quantile(0.95, rate(http_request_duration_seconds_bucket[5m]))`
+- **Type:** Single Stat
+- **Title:** "95th Percentile Response Time"
+
+### 3. Page Views Panel  
+- **Metric:** `knet_page_views_total`
+- **Type:** Graph
+- **Title:** "Page Views by Route"
+
+### 4. Box Clicks Heatmap
+- **Metric:** `rate(knet_box_clicks_total[5m])`
+- **Type:** Graph
+- **Title:** "Box Clicks per Second"

 ## Environment Variables

@@ -156,63 +104,24 @@ Set these for production:
 ```bash
 export API_TOKEN="your-secure-api-token"
 export API_USERNAME="api"  # Optional, defaults to "api"
-export BUILD_NUMBER="prod-v1.0"
-```
-
-## Grafana Dashboard JSON
-
-Here's a complete dashboard configuration:
-
-```json
-{
-  "dashboard": {
-    "title": "Website Analytics",
-    "panels": [
-      {
-        "title": "Page Views (24h)",
-        "type": "singlestat",
-        "targets": [{"target": "hits"}],
-        "gridPos": {"h": 8, "w": 6, "x": 0, "y": 0}
-      },
-      {
-        "title": "Unique Users (24h)", 
-        "type": "singlestat",
-        "targets": [{"target": "users"}],
-        "gridPos": {"h": 8, "w": 6, "x": 6, "y": 0}
-      },
-      {
-        "title": "Traffic Trends",
-        "type": "graph",
-        "targets": [
-          {"target": "hits", "refId": "A"},
-          {"target": "users", "refId": "B"}
-        ],
-        "gridPos": {"h": 8, "w": 12, "x": 0, "y": 8}
-      }
-    ],
-    "time": {
-      "from": "now-24h",
-      "to": "now"
-    },
-    "refresh": "30s"
-  }
-}
+export BUILD_NUMBER="prod-v1.0"  # Enables production mode
 ```

 ## Benefits of This Approach

-1. **Multiple Grafana Data Sources** - Choose what works best for you
-2. **Real-time Updates** - Data refreshes automatically  
-3. **Flexible Time Ranges** - Hour, day, week, month views
-4. **Standard Formats** - Works with existing Grafana plugins
-5. **Authentication** - Secured with Basic Auth in production
+1. **Standard Prometheus** - Works with all Grafana features
+2. **Rich Metrics** - HTTP performance + custom business metrics
+3. **No Database** - Prometheus handles storage and retention
+4. **Better Performance** - Optimized time-series queries
+5. **Alerting Ready** - Use Prometheus AlertManager
+6. **Ecosystem** - Tons of existing dashboards and tools

 ## Quick Start

-1. Start your server: `./tmp/main`
-2. Test metrics endpoint: `curl http://localhost:54321/api/metrics`
-3. Add Prometheus data source in Grafana pointing to `/api/metrics`
-4. Create dashboard with `http_requests_total` and `http_unique_users_total` queries
-5. Set refresh interval to 30 seconds
+1. **Start your server:** `./tmp/main`
+2. **Test metrics:** `curl -u api:your-token http://localhost:54321/metrics`
+3. **Add Prometheus data source** in Grafana: `http://localhost:54321`
+4. **Create dashboard** with queries like `http_requests_total`
+5. **Enjoy real-time metrics!** 📊

-Your analytics will now update in real-time in Grafana! 📊
+Your analytics are now powered by industry-standard Prometheus! <EFBFBD>