Pular para o conteúdo principal

gtm-fabric

Purpose and Production Relevance​

gtm-fabric is the data-plane worker and GTM runtime orchestrator. It manages Docker GTM containers, consumes Pub/Sub tagging events, forwards requests to containers, parses and enriches event data, writes ClickHouse analytics/logs, performs DNS/SSL operations, and reports runtime state back to node-api.

Production relevance: core production data-plane service.

Stack and Main Dependencies​

  • Java 17.
  • Spring Boot 3.1.
  • Spring WebFlux and Reactor.
  • Hexagonal/ports-and-adapters style.
  • Docker Java API.
  • Google Pub/Sub, Secret Manager, and some older Firestore dependency remnants.
  • ClickHouse JDBC.
  • ACME4j and BouncyCastle for Let's Encrypt certificates.
  • MaxMind GeoIP and UA parser.
  • Resilience4j for rate limiting/circuit breaker behavior.
  • Micrometer/Prometheus.

Important files and folders:

  • gtm-fabric/src/main/java/org/src/application
  • gtm-fabric/src/main/java/org/src/core
  • gtm-fabric/src/main/java/org/src/infrastructure
  • gtm-fabric/src/main/resources
  • gtm-fabric/docs
  • gtm-fabric/pom.xml
  • gtm-fabric/docker-compose.yml

Business Responsibilities​

gtm-fabric owns runtime work, not durable business truth:

  • Create, stop, delete, and health-check GTM Docker containers.
  • Assign and report container ports.
  • Serve or expose management endpoints needed by GTM settings and debug flows.
  • Consume tagging events from Pub/Sub.
  • Forward events to the correct GTM container.
  • Parse GA4, Meta, TikTok, and generic events.
  • Enrich events with GeoIP and user-agent data.
  • Batch insert analytics events and logs into ClickHouse.
  • Manage capture rules and identity endpoints required by customer sites.
  • Batch flush captured contacts and usage back to node-api.
  • Verify domains, run ACME challenges, persist certificates through node-api, and notify proxy reloads.

Stateless Boundary​

gtm-fabric is stateless for durable domain/config persistence. On startup it fetches GTM config from node-api, keeps local in-memory cache, and reports changes back.

Developer rule: if data defines product truth, it belongs in node-api/PostgreSQL, not only in fabric memory.

Key Data Flows and Service Integrations​

GTM Provisioning​

node-api calls fabric when a GTM is created or changed. Fabric creates preview/tagging containers, assigns ports, configures DNS/SSL if needed, and reports lifecycle state.

Tagging Worker​

Fabric consumes events from Pub/Sub:

  1. Receive event.
  2. Resolve GTM/container config.
  3. Apply rate limiter and circuit breaker.
  4. Forward HTTP request to the GTM container.
  5. ACK/NACK according to result.
  6. Parse/enrich asynchronously.
  7. Batch insert into ClickHouse.

Event Parser Chain​

Event parsing is a chain of responsibility:

  • GA4 parser for /g/collect with Measurement Protocol params.
  • Meta parser for pixel traffic.
  • TikTok parser where applicable.
  • Generic fallback parser.

CRM Capture​

Fabric exposes public capture/identity/config endpoints used by the custom loader and customer browser. It resolves identity, applies capture rules, and flushes contacts to node-api.

Domain Verification and Certificates​

Fabric verifies DNS, handles ACME HTTP-01, generates/stores cert material through node-api, and notifies gtm-proxy to reload route/cert cache when needed.

Environment, Deploy, and Runtime Notes​

Common commands:

cd gtm-fabric
mvn test
mvn package
docker compose up --build

Local/test files:

  • .env.example
  • docker-compose-local.yml
  • docker-compose-test.yml
  • test-reverse-proxy.sh
  • test_traffic.sh

Production notes:

  • Runs on the Hostinger VPS/host network according to current docs.
  • Talks to node-api via internal/Tailscale URL.
  • Writes directly to ClickHouse.
  • Publishes runtime state back to node-api.

Debugging Checklist​

  • If startup config is empty, check vmId, node-api internal endpoint, network, and auth/internal restrictions.
  • If GTM create fails, inspect Docker socket access, image/env config, port allocation, and node-api lifecycle report calls.
  • If events are not processed, check Pub/Sub subscription, worker concurrency, route config cache, and container health.
  • If ClickHouse data is missing, check parser selection, enrichment errors, batch size/timeout, and insert failures.
  • If contacts are missing, check identity endpoint, capture config cache, rule match, batch flush interval, and node-api batch response.
  • If certificates fail, check DNS target, ACME challenge path, file/cert material, node-api persistence, and proxy reload notification.

Tests or Validation Commands​

cd gtm-fabric
mvn test
mvn package

Manual smoke tests:

  • Start fabric and confirm it fetches GTMs for the current VM.
  • Create a GTM from node-api and confirm containers/ports are reported back.
  • Publish a test tagging event and confirm container forward plus ClickHouse insert.
  • Verify a pending domain in a safe environment.
  • Inspect container logs through the documented endpoints.

Mastery Checklist​

  • Explain why fabric is stateless for config but still writes analytics/logs.
  • Trace a Pub/Sub event through worker, rate limiter, container forward, parser, enrichment, and ClickHouse.
  • Add a parser or enrichment behavior without blocking the critical forwarding path.
  • Debug Docker container lifecycle from node-api request to fabric action to DB status update.
  • Know which failures should ACK, NACK, retry, or report lifecycle status.