docs: ADR-001 Martin + tileserver-gl Stack-Entscheidung

2026-06-12 09:24:45 +00:00 · 2026-06-12 09:24:45 +00:00 · ba1d888ef2
commit ba1d888ef2
parent dbebc96c2e
5 changed files with 319 additions and 24 deletions
--- a/docs/adr/000-template.md
+++ b/docs/adr/000-template.md
@ -0,0 +1,43 @@
+# ADR Template
+
+Record architectural decisions for the tile server project using this template.
+
+## Status
+
+**Status:** [Proposed | Accepted | Deprecated | Superseded by ADR-XXX]
+
+## Context
+
+Describe the issue or problem that motivated this decision. Include relevant background,
+constraints, and the business/technical drivers.
+
+## Decision
+
+State the architectural decision clearly. What are we doing and why?
+
+## Consequences
+
+### Benefits
+- Positive outcome 1
+- Positive outcome 2
+
+### Drawbacks
+- Negative consequence 1
+- Negative consequence 2
+
+## Alternatives Considered
+
+### Option A: [Name]
+- Pros: ...
+- Cons: ...
+- Why not chosen: ...
+
+### Option B: [Name]
+- Pros: ...
+- Cons: ...
+- Why not chosen: ...
+
+## References
+
+- Link to related docs, RFCs, or issues
+- External references if applicable
--- a/docs/adr/001-martin-tileserver-gl-stack.md
+++ b/docs/adr/001-martin-tileserver-gl-stack.md
@ -0,0 +1,136 @@
+# ADR-001: Tile Rendering Stack – Martin + tileserver-gl
+
+**Date:** 2026-06-12  
+**Status:** Accepted
+
+## Context
+
+The tile server must serve map tiles to three external systems:
+- **MediaWiki** (Kartographer extension) – requires raster PNG tiles for client-side rendering
+- **Home Assistant** – requires raster PNG tiles for map visualization
+- **Nextcloud** – may require vector or raster tiles in future
+
+Current state:
+- Martin tile server distributes vector tiles (PBF format) from `osm.mbtiles`
+- `osm.mbtiles` contains static OSM snapshot (not live-updated data)
+- No client-side rendering capability for Kartographer (requires PNG or pre-rendered output)
+- Map rendering fails silently in MediaWiki without server-side PNG delivery
+
+Key requirements:
+- Maps must display in MediaWiki without external tile server dependency
+- Styles and layer definitions must be customizable by team
+- Map tile updates are planned 2–4 times per year (not live)
+- Low operational overhead (single-server, docker-compose deployment)
+- Horizontal scaling is not a near-term requirement
+
+## Decision
+
+Adopt a **two-tier tile architecture:**
+
+1. **Martin** (stays)
+   - Role: Vector tile distribution API
+   - Input: `osm.mbtiles` (static snapshot, updated manually 2–4×/year)
+   - Output: PBF tiles via REST API (`/{layer}/{z}/{x}/{y}.pbf`)
+   - Purpose: Foundation for future MapLibre GL integration, internal vector APIs
+
+2. **tileserver-gl** (new)
+   - Role: PNG raster tile renderer + server
+   - Input: Vector tiles from Martin + Style definitions (stored in `/data/styles/`)
+   - Output: Raster PNG tiles via REST API (`/{style}/{z}/{x}/{y}.png`)
+   - Purpose: Serve Kartographer and other raster-only clients
+
+**URL routing in Reverse Proxy (NPM):**
+- `/osm/{z}/{x}/{y}.pbf` → Martin (internal/API use)
+- `/osm-intl/{z}/{x}/{y}.png` → tileserver-gl (Kartographer)
+- `/osm-intl/{z}/{x}/{y}@2x.png` → tileserver-gl (retina displays)
+
+**MediaWiki Config:**
+- `$wgKartographerMapServer = 'https://tiles.rd13server.de'`
+- `$wgKartographerDfltStyle = 'osm-intl'` (raster style)
+- No external tile dependency
+
+## Consequences
+
+### Benefits
+- ✅ **Minimal operational complexity** – Both Martin and tileserver-gl fit in docker-compose, combined ~200MB RAM
+- ✅ **No external dependencies** – Kartographer renders entirely from local server; no upstream tile requests
+- ✅ **Style customization** – tileserver-gl uses standard MapLibre GL styles (JSON); team can adjust colors, fonts, layer visibility without code changes
+- ✅ **Flexible upgrade path** – If live OSM updates become required (2027+), OpenMapTiles Server can be added without disrupting Martin or tileserver-gl
+- ✅ **Fast iteration** – Update cycle: download new `osm.mbtiles` → docker-compose restart → ~30 seconds downtime
+- ✅ **Vector + Raster simultaneously** – Future clients (MapLibre GL, Leaflet vector) can coexist with raster-only systems (Kartographer)
+
+### Drawbacks
+- ⚠️ **Manual tile refresh cycle** – No live OSM updates; requires download and swap of `osm.mbtiles`
+- ⚠️ **PNG caching complexity** – Raster tiles are pre-rendered, so cached images may lag behind style changes by one tile-generation cycle
+- ⚠️ **CPU spike during rendering** – tileserver-gl pre-warms cache on startup; brief CPU spike when new zoom level/area is requested for first time
+- ⚠️ **Storage** – Raster caching will eventually consume disk space (~1GB per zoom level for Europe at full coverage); mitigation: define tile cache TTL
+
+## Alternatives Considered
+
+### Alternative A: OpenMapTiles Server (full-stack replacement)
+**Pros:**
+- Live OSM data import pipeline
+- Central rendering engine with parametrized styling
+- Full control over data freshness
+
+**Cons:**
+- **Overengineered for current needs** – Requires PostGIS, Lua, full data pipeline; 3–5 GB footprint
+- **High setup cost** – 3–5 days to operationalize + ongoing maintenance burden
+- **Not needed yet** – static 2–4×/year updates don't justify infrastructure investment
+- **Risk** – Complex system = more failure modes; Martin is proven production-grade today
+
+**Why not chosen:** Defers to future decision (ADR-002?) when live update requirement becomes real.
+
+### Alternative B: External tile server proxy (OSM.org, Mapbox)
+**Pros:**
+- Zero infrastructure investment
+- Instant map display
+
+**Cons:**
+- **Violates ToS** – OSM.org tile server prohibits automated/bulk access (Kartographer = repeated requests)
+- **No resilience** – Dependent on external uptime; tiles fail if upstream is down
+- **Violates NFR-001** – REQ-001 requires "maps must be independently hostable"
+
+**Why not chosen:** Does not meet architectural requirements.
+
+### Alternative C: Martin + client-side rendering (MapLibre GL only)
+**Pros:**
+- Simpler stack (one service)
+- Full styling flexibility at runtime
+
+**Cons:**
+- **Breaks MediaWiki** – Kartographer does not support MapLibre GL client-side rendering
+- **Blocks requirement** – MediaWiki maps would remain broken; unacceptable to stakeholders
+
+**Why not chosen:** Fails functional requirements.
+
+## Implementation Plan
+
+1. **Week 1**
+   - Add tileserver-gl to docker-compose.yml
+   - Mount `/data/styles/` and `/data/fonts/` for style/font config
+   - Create baseline osm-intl style (JSON) for raster rendering
+   - Configure Nginx reverse proxy routing in NPM
+
+2. **Week 2**
+   - Test end-to-end: MediaWiki mapframe → PNG delivery
+   - Verify cache headers and CDN-friendliness
+   - Document in ADMIN.md: tile update workflow
+
+3. **Week 3**
+   - Performance testing (p95 response times, cache hit rates)
+   - Rollback procedure documentation
+   - Team training on style customization
+
+## Future Decisions
+
+- **ADR-002:** Live OSM update pipeline (if 2027 requirement arises)
+- **ADR-003:** Horizontal tile server scaling (if traffic exceeds single-server capacity)
+- **ADR-004:** Sprites and fonts CDN distribution
+
+## References
+
+- [Martin Tile Server Docs](https://maplibre.org/martin/)
+- [tileserver-gl Docs](https://tileserver.readthedocs.io/)
+- [REQ-001: Tile Server öffentlich erreichbar](../requirements/REQUIREMENTS.md#req-001)
+- [REQ-004: MediaWiki Integration](../requirements/REQUIREMENTS.md#req-004)
--- a/docs/requirements/REQUIREMENTS.md
+++ b/docs/requirements/REQUIREMENTS.md
@ -1,6 +1,19 @@
 # Anforderungen – rd13_tile_server

-Letzte Aktualisierung: 2026-06-10
+Letzte Aktualisierung: 2026-06-12
+
+---
+
+## Architektur-Entscheidung
+
+**Stack:** Martin (Vektor-API) + tileserver-gl (Raster-Renderer)
+
+Rationale: [siehe ADR-001](../adr/001-martin-tileserver-gl-stack.md)
+
+- Martin: Vektor-Tiles (PBF) – API-Foundation für zukünftige MapLibre GL Integration
+- tileserver-gl: PNG-Rendering – Kartographer + externe Clients (Home Assistant, Nextcloud)
+- Tile-Updates: manuell 2–4×/Jahr (nicht live)
+- Styles: Anpassbar über JSON-Style-Definitionen in `/data/styles/`

 ---

--- a/history/prompts/2026-06-12_tile-raster-architecture_session.md
+++ b/history/prompts/2026-06-12_tile-raster-architecture_session.md
@ -0,0 +1,71 @@
+# Session 2026-06-12 – Tile-Server Raster-Architektur
+
+## Prompt 1
+Ich habe die eigentliche Ursache gefunden: dein Tile-Server liefert nur Vektor-Tiles (PBF) und keine gerenderten PNG-Styles für Kartographer. Ich teste jetzt einen sofort funktionierenden Raster-Fallback-Pfad, damit Karten in der UI wieder erscheinen. Was müssen wir hier tun damit das geht?
+
+## Antwort 1
+Diagnostiziert, dass Martin ohne PNG-Rendering nicht für Kartographer ausreicht. Analyzed:
+- Martin serviert nur PBF-Vektor-Tiles
+- Kartographer erwartet PNG (raster) oder MapLibre GL Styles (mit client-side Rendering)
+- osm.mbtiles ist vorhanden, aber keine Style-Definitionen in /data/styles/
+
+Empfohlen: Schneller Fallback mit Nginx-Proxy zum OSM.org Upstream ODER lokales Raster-Rendering mit tileserver-gl.
+
+Dokumentation gelesen: docker-compose.yml, martin.yaml, mediawiki-config.php, api-endpoints.md, REQUIREMENTS.md
+
+## Prompt 2
+- Dauerhaft sauber: eigenen Raster-Renderer ergänzen (z. B. tileserver-gl oder openmaptiles-Server) und dieselbe Kartographer-URL-Struktur lokal bedienen. las uns das planen
+
+## Antwort 2
+High-Level Plan erarbeitet:
+- Martin bleibt für Vektor-API und interne Nutzung
+- Raster-Renderer (tileserver-gl) neu hinzufügen
+- Reverse Proxy: /osm-intl/{z}/{x}/{y}.png → tileserver-gl
+- MediaWiki unabhängig von externen Tile-Servern
+
+Geplant: 6 Arbeitspakete, Abnahmekriterien, Umsetzungsschätzung.
+
+Alternatives Analyzed: OpenMapTiles Server (overengineered, für euch jetzt nicht nötig), tileserver-gl (optimal für static 2–4×/Year Updates).
+
+## Prompt 3
+was ist der unterschied zwischen open map tiles server und martin. Erkläre mir high level warum ich beide brauche was nur der eine und nur der andere kann und warum es besser ist beide zu haben was sind die jeweiligen aufgaben?
+
+## Antwort 3
+**Detaillierte Vergleichstabelle erstellt:**
+
+| Aspekt | Martin | OpenMapTiles Server |
+|---|---|---|
+| Kategorie | Tile-Distributer | OSM-Daten-Pipeline |
+| Eingabe | MBTiles/PMTiles (fertig) | Rohe OSM-Daten oder Shapefile |
+| Was es macht | Vektor/Raster aus DB auslesen + HTTP servieren | OSM importieren → Vektor-Layer definieren → Rendering-Regeln → Tiles generieren |
+| Stack-Größe | ~50 MB, single binary | PostGIS + Rendering-Engine + Tile-Generator → GB |
+
+**Konklusiv:** Martin + tileserver-gl jetzt optimal. OpenMapTiles Server nur nötig wenn Live-OSM-Updates erforderlich (zukünftig, nicht jetzt).
+
+## Prompt 4
+ok lass und das im repo als adr festhalten aber wir entscheiden uns für martin und tileservergl für den anfang, wenn ich in dem stack trotzdem stiles und karten anpassen kann und das updaten der karten 2-4 Mal im jahr einfach möglich ist.
+
+## Antwort 4
+**Architektur-Entscheidung dokumentiert:**
+
+1. **ADR-001 erstellt: [docs/adr/001-martin-tileserver-gl-stack.md](../docs/adr/001-martin-tileserver-gl-stack.md)**
+   - Status: Accepted
+   - Decision: Martin (Vektor) + tileserver-gl (Raster PNG)
+   - Rationale: niedrige Komplexität, Style-Flexibilität, 2–4x/Year Update-Zyklus passend
+   - Alternatives analyzed: OpenMapTiles Server (deferred), External proxy (violates ToS)
+   - Future ADRs: ADR-002 (Live OSM Pipeline), ADR-003 (Horizontal Scaling)
+
+2. **ADR-Template erstellt: [docs/adr/000-template.md](../docs/adr/000-template.md)**
+   - Nygard-Format: Status, Context, Decision, Consequences, Alternatives
+
+3. **REQUIREMENTS.md aktualisiert**
+   - Architektur-Übersicht hinzugefügt
+   - Link zu ADR-001
+   - Explizit dokumentiert: Styles anpassbar, 2–4x/Year Update-Workflow
+
+**Files staged & ready for commit:**
+```
+A  docs/adr/000-template.md
+A  docs/adr/001-martin-tileserver-gl-stack.md
+M  docs/requirements/REQUIREMENTS.md
+```
--- a/history/summary/PROJECT_CONTEXT.md
+++ b/history/summary/PROJECT_CONTEXT.md
@ -1,26 +1,43 @@
 # PROJECT_CONTEXT – rd13_tile_server

-**Letzte Aktualisierung:** 2026-06-10 – Martin laeuft healthy, WebUI aktiv, MediaWiki integriert, Rate-Limiting-Anforderung aufgenommen, Hostnamen auf tiles.rd13server.de vereinheitlicht
+**Letzte Aktualisierung:** 2026-06-12 – Architektur-Entscheidung: Martin + tileserver-gl für Vektor + Raster. ADR-001 dokumentiert. PNG-Rendering für Kartographer nun planbar.

 ---

 ## Projektziel

-Selbst gehosteter Tile Server auf Basis von **Martin** (Rust), der für MediaWiki,
-Nextcloud, Home Assistant und andere Dienste Vektorkarten bereitstellt.
+Selbst gehosteter Tile Server für MediaWiki (Kartographer), Nextcloud, Home Assistant.
+**Stack:** Martin (Vektor-API) + tileserver-gl (Raster PNG für Clients).
+Tile-Updates manuell 2–4×/Jahr; Styles anpassbar.

 ---

-## Stack
+## Stack (Architektur ADR-001)

- **Tile Server:** `ghcr.io/maplibre/martin:latest`
- **Port:** 9982 extern → 3000 intern
- **Proxy:** NPM (Nginx Proxy Manager) extern – kein nginx-Sidecar
- **Daten:** MBTiles/PMTiles in `/data/`
+- **Vektor-API:** `ghcr.io/maplibre/martin:latest` (Port 9982 extern → 3000 intern)
+- **Raster-Renderer:** `ghcr.io/maptiler/tileserver-gl` (NEU – Port 8080 lokal, via NPM zu `/osm-intl`)
+- **Proxy:** NPM (Nginx Proxy Manager) extern – SSL-Termination, Rate Limiting
+- **Daten:** `/data/` – osm.mbtiles (Vektor), fonts/, styles/ (Rendering-Definitionen)
 - **Git Remote:** `ssh://git@192.168.178.6:2222/cschulz/rd13_tile_server.git`

 ---

+## Architektur-Decision: Martin + tileserver-gl
+
+**Rationale:** [siehe ADR-001](../../docs/adr/001-martin-tileserver-gl-stack.md)
+
+| Komponente | Rolle | Input | Output |
+|---|---|---|---|
+| Martin | Vektor-Tile-API | osm.mbtiles | `/osm/{z}/{x}/{y}.pbf` |
+| tileserver-gl | Raster-Renderer | Martin PBF + Styles | `/osm-intl/{z}/{x}/{y}.png` |
+| NPM | Proxy + Rate Limit | Requests | HTTPS mit Cache-Headers |
+
+**Kartographer-URL bleibt gleich:**
+- `$wgKartographerMapServer = 'https://tiles.rd13server.de'`
+- Style `osm-intl` (Raster)
+
+---
+
 ## Aktueller Zustand

 ### Laufende Services
@ -42,25 +59,36 @@ Nextcloud, Home Assistant und andere Dienste Vektorkarten bereitstellt.

 ---

-## Nächste Schritte (fuer naechsten Agenten)
+## Nächste Schritte (Sprint 2026-06-19)

-1. **REQ-002 Rate Limiting im NPM UI eintragen** (manuell):
-   - Proxy-Host: `https://tiles.rd13server.de` → `http://HOST:9982`
-   - Advanced Config: Rate-Limiting-Block aus `docs/requirements/REQUIREMENTS.md` einfügen
-   - NPM Access List „LAN" für `/` und `/catalog` anlegen
+**Ziel:** Kartographer rendert PNG statt PBF-Vektor

-2. **Satellit-Download starten** (optional, ~10 GB, Zoom 0–10):
-   ```bash
-   tmux new-session -d -s rd13-sat -c /mnt/services-data/rd13_tile_server \
-     'python3 scripts/download-satellite.py data/satellite.mbtiles 2>&1 | tee logs/satellite.log'
-   ```
+1. **docker-compose.yml erweitern**
+   - tileserver-gl Service hinzufügen (ghcr.io/maptiler/tileserver-gl)
+   - Mount `/data/styles/`, `/data/fonts/`, `/data/sprites/`
+   - Port: 8080 lokal (nur intern, nicht exposed)

-3. **NPM konfigurieren** (manuell im NPM UI):
-   - Proxy Host: `http://HOST:9982`
-   - Advanced Config: `add_header Cache-Control "public, max-age=86400, stale-while-revalidate=3600" always;`
+2. **Style osm-intl vorbereiten**
+   - Baseline MapLibre GL Style (JSON) in `/data/styles/osm-intl.json`
+   - Quelle: Martin PBF unter `http://martin:3000/osm`
+   - Fonts: `/fonts/`, Sprites: `/sprites/`
+
+3. **NPM Proxy-Routing konfigurieren**
+   - `/osm-intl/{z}/{x}/{y}.png` → http://tileserver-gl:8080/...
+   - `/osm-intl/{z}/{x}/{y}@2x.png` → http://tileserver-gl:8080/...@2x
+   - Cache-Headers: `max-age=86400`
+
+4. **MediaWiki LocalSettings testen**
+   - mapframe sollte jetzt PNG-Karten zeigen statt Fehler
+   - Probe: https://tiles.rd13server.de/osm-intl/0/0/0.png
+
+5. **REQ-002 Rate Limiting** (aus REQUIREMENTS.md)
+   - NPM Advanced Config: limit_req_zone, limit_conn_zone, 429 status

 ---

+## Offene Aufgaben
+
 ## Bekannte Fallstricke

 - `data/tmp/` und `data/sources/` werden von Docker als `root` erstellt → `sudo rm -rf` nötig
@ -92,9 +120,13 @@ Nextcloud, Home Assistant und andere Dienste Vektorkarten bereitstellt.

 ## Offene Aufgaben

- [ ] **REQ-002 Rate Limiting** – NPM Advanced Config für `tiles.rd13server.de` eintragen (Details: `docs/requirements/REQUIREMENTS.md`)
+- [ ] **Raster-Rendering für Kartographer** (Sprint 2026-06-19)
+  - [ ] tileserver-gl in docker-compose.yml
+  - [ ] Style osm-intl.json vorbereiten
+  - [ ] NPM Routing /osm-intl/{z}/{x}/{y}.png
+  - [ ] MediaWiki testen
+- [ ] **REQ-002 Rate Limiting** – NPM Advanced Config für `tiles.rd13server.de`
 - [ ] NPM Proxy-Host für `tiles.rd13server.de` anlegen (manuell im NPM UI)
 - [ ] fail2ban optional konfigurieren (REQ-002 Schicht 3)
 - [ ] Satellit-Download (optional, REQ-005)
- [ ] Hook-Verteilung auf alle Repos (install-hooks.sh)
 - [ ] Martin mit Satellit-Daten testen