View on GitHub

OdbDesign ODB++ Parser

REST API and C++ library for parsing ODB++ Design archives

OdbDesignServer — implementation plan (decisions applied)

   
Status ACTIVE — executing. All decisions recorded (incl. REST TLS cert source = Option C, 2026-09-10 follow-up). First tranche on nam/server-issues-impl: Phase 0 + M3.1.
Source server-issues.md (options + pros/cons + decisions) — this doc turns those decisions into sequenced, file-level work.
Related https-tls-options.md (REST TLS — DECIDED: Option C, 2026-09-10) · opt/phase2-grpc-optimizations.md (SI19 implementation spec) · gh-pages-coverage-integration.md
Method Every milestone = one nam/<feature> branch → PR → nam20485, merged with merge commits only (AGENTS.md, directive 2026-09-10). C++ milestones ride the multi-platform PR builds; docs/workflow milestones ride the static checks.

Decisions being implemented

SI Decision (from server-issues.md NOTE blocks) Milestones
SI1 Option 1 now — redoc.html in the swaggerui sidecar; annotation pass in the single swagger YAML. Spec-provenance question answered in §SI1-A below. M0.3, M0.4
SI2 Option B, branch set {development, release}. M0.5
SI3 Keep main; no changes. — (none)
SI4 Decided 2026-09-10: Option 1 — Traefik IngressRouteTCP termination, app plaintext in-cluster. Cert source decided: Option C (cert-manager private CA, IP SANs) — M3.1 unblocked. M3.1
SI5a Keycloak on k3s + JWT bearers (Crow middleware + gRPC interceptor); stopgap hardening first. M2.1–M2.3, M2.5
SI5b Option 1 — ownership sidecar metadata enforced at two choke points; device flow via Keycloak; sanitizeFilename fix included. M2.4
SI6 Items 1 (serialized-response cache) + 2 (ETag/Cache-Control); item 3 via SI7; item 4 opportunistic; 5/6 demand-driven. M1.3, M1.4, M1.5
SI7 Option 1 — single-flight background load + RequestLoadDesign RPC + POST /designs/<name>/load + upload auto-warm; progress deferred to SI8. M1.1, M1.2
SI8 Option 1 — WebSockets events channel with polling fallback by construction. M3.2
SI19 B+C hybrid — 2.1 + 2.2 now as standalone PRs; 1.3 (arenas) sequenced with SI6.1 under one benchmark. M0.1, M0.2, M1.4

Phase overview (dependencies, not calendar):

Phase 0 (now, all parallel, no interdeps):
  M0.1 proto arenas option      M0.2 compression level     M0.3 /redoc sidecar
  M0.4 spec annotation pass     M0.5 Pages per-branch coverage

Phase 1 (perf track, strictly ordered at the cache boundary):
  M1.1 single-flight + LRU eviction  ──►  M1.2 background load + RequestLoadDesign
  M1.3 ETag/Cache-Control (any time)      M1.4 response cache + arenas (one benchmark) ──► M1.5 watcher (optional)

Phase 2 (identity track, strictly ordered):
  M2.1 auth stopgaps ──► M2.2 Keycloak on k3s ──► M2.3 JWT middleware + interceptor
                       ──► M2.4 ownership + device flow ──► M2.5 Basic removal

Phase 3 (edge & events):
  M3.1 gRPC TLS (Option C — unblocked, first tranche)     M3.2 WS /events (after M1.2)

Phase 0 — quick wins (parallel)

M0.1 — SI19.2.1: enable protobuf arenas in all .proto files

M0.2 — SI19.2.2: compression level config

M0.3 — SI1: /redoc page in the swaggerui sidecar

M0.4 — SI1: annotation pass on swagger/odbdesign-server-0.9-swagger.yaml

SI1-A: spec provenance (curation + generation) — answer to the open question

The recorded question: can the spec be dynamically generated from source or live endpoints instead of hand-curated, and where does Kiota fit?

  1. From source code: no realistic path in Crow. Crow registers routes at runtime (route_dynamic/CROW_ROUTE); it has no annotation/reflection facility for OpenAPI, and no generator exists for it. The C++ frameworks that do generate OpenAPI from code (Drogon ≥1.9 via drogon_ctl, oat++ line) require a framework migration — not worth it for this. The proto layer is generatable (protoc already emits *_pb2.py-style descriptors) — and that’s exactly what the current spec’s schema blocks already mirror. The hand-curated YAML is the right source of truth for this codebase.
  2. From live endpoints: possible but strictly worse. Traffic-capture tools (mitmproxy2swagger, har-to-openapi) infer coarse schemas from observed JSON — no descriptions, wrong-guess types for proto3’s omitted-defaults, no enums/constraints. They would destroy the current detail level. Useful only as a verification signal, not a generator.
  3. Kiota: wrong direction for this problem. Kiota is an OpenAPI consumer — it generates API clients from a spec, not specs from code. Once the spec is where we want it (M0.4), Kiota is an excellent way to generate typed clients for the server, and worth a look then. It cannot help produce the spec.
  4. The pragmatic best-of-both: drift checking (adopted). Keep the curated YAML as source of truth, and add a CI check that the route surface can’t silently drift from it:
    • Add a build-flag-gated GET /debug/routes endpoint that dumps Crow’s registered routes (Crow exposes the router’s rules — enumerate and serialize as JSON), OR a small script parsing CROW_ROUTE/route_dynamic registrations from Controllers/*.cpp.
    • CI job diffs that route list against the spec’s paths: — new/removed/renamed endpoints fail the check.
    • Schema-level drift (message shape vs spec components) stays manual for now; revisit if it bites. This keeps the annotation richness and gets the generator’s main benefit (no silent staleness). Fold into M0.4’s CI lint job.

M0.5 — SI2: per-branch coverage on Pages ({development, release})


Phase 1 — performance track (touches the DesignCache hot path; ordered)

M1.1 — SI6.3: single-flight loads + byte-budget LRU eviction in DesignCache

M1.2 — SI7: background load + RequestLoadDesign + REST twin + upload auto-warm

M1.3 — SI6.2: ETag + Cache-Control on data endpoints

M1.4 — SI6.1 + SI19.1.3: serialized-response cache + arena allocation (one benchmark)

M1.5 — SI6.4 (optional, opportunistic): designs-dir watcher


Phase 2 — identity track (strictly ordered)

M2.1 — SI5a stopgaps (day one, regardless of Keycloak timing)

M2.2 — SI5a: Keycloak on k3s (Argo CD GitOps)

M2.3 — SI5a: JWT validation in Crow middleware + gRPC interceptor

M2.4 — SI5b: ownership store + enforcement + device flow + sanitizeFilename

M2.5 — SI5a completion: remove Basic


Phase 3 — edge & events

M3.1 — SI4: gRPC TLS via Traefik IngressRouteTCP + Option C cert chain

M3.2 — SI8: WebSocket events channel (/events) with polling fallback


Risk register

Risk Where Mitigation
DesignCache concurrency regressions M1.1–M1.4 The state machine + single-flight land first and alone (M1.1); everything after consumes it; concurrent-parse test is the acceptance bar.
Cache invalidation misses (stale data served) M1.4 Three explicit invalidation triggers + tests for each; when in doubt, invalidate.
Auth cutover breaks clients M2.3/M2.5 both transition mode; Basic removal is its own late milestone after the optimized client is on tokens.
Ownership rollout locks out users M2.4 Legacy-synthesis rule (shared=all) keeps every pre-existing design readable; matrix tests.
Keycloak becomes a single point of failure M2.2 Server caches JWKS with refresh-on-unknown-kid; token validation keeps working through short IdP outages (already-issued tokens verify locally).
Memory growth from eager loads M1.2 M1.1’s budget/eviction is a hard prerequisite; background-load semaphore caps parse storms.
WS through proxies silently broken M3.2 Proxy configs are per-deployment acceptance criteria (k3s AND compose), not afterthoughts.

Definition of done (per milestone)

  1. Branch nam/<feature> from nam20485; PR to nam20485; merge commit only (never squash/rebase).
  2. Multi-platform builds + tests green (C++ milestones); lint/static checks green (docs/workflow milestones).
  3. Acceptance criteria above demonstrated in the PR description (benchmark numbers for M1.4).
  4. Swagger spec + docs updated in the same PR when the API surface changes.
  5. k3s + compose deployment shapes both updated when deploy files change.