Skip to content

Data flows

Concrete message and storage paths across the four platform repos.

Upload → atomize

sequenceDiagram
  participant UI as Frontend PHP
  participant MQ as RabbitMQ
  participant UP as upload-processor
  participant S3 as S3
  participant AP as atomize-processor
  participant DB as MySQL

  UI->>MQ: upload.request (+ encrypted GH tokens if private)
  MQ->>UP: consume
  UP->>S3: upload repo tree
  UP->>MQ: upload.response
  MQ->>UI: apply status
  UI->>MQ: atomize.request
  MQ->>AP: consume
  AP->>S3: download tree
  AP->>AP: docker run probe extract
  AP->>DB: atoms / deps / statuses
  AP->>MQ: atomize.response + livelog
  MQ->>UI: worker.php updates

Details: Processor data flow, queue workers.

Certify → validate → promote

sequenceDiagram
  participant UI as Frontend PHP
  participant DB as MySQL
  participant MQ as RabbitMQ
  participant VP as validate_processor
  participant S3 as S3 cert bucket
  participant PP as promote_processor

  UI->>DB: certificates.status = pending
  UI->>MQ: validate.request
  MQ->>VP: consume
  VP->>VP: probe docker build/run
  VP->>S3: probe-manifest-complete.json
  VP->>MQ: validate.response (+ optional Sepolia tx)
  MQ->>UI: sole DB writer → ready / error
  Note over UI,PP: Later user/ops action
  UI->>MQ: promote.request
  MQ->>PP: mainnet Certify.sol
  PP->>MQ: promote.response
  MQ->>UI: store mainnet tx hash

Details: Cert queue, worker.

Atomize → GraphQL extract store

After a successful atomize response is applied to MySQL, the frontend publishes a graphql.ingest message. The ingest-relay on the paired VQ* host (VQD/VQS/VQP) consumes it, reads the probe JSON from S3, and appends snapshot rows to Postgres. Hasura serves those rows as GraphQL for dynamic chart blocks.

sequenceDiagram
  participant UI as Frontend AtomizeService
  participant MQ as RabbitMQ
  participant RL as ingest-relay (VQ*)
  participant S3 as S3
  participant PG as Postgres
  participant HS as Hasura

  UI->>UI: persist atoms to MySQL
  UI->>MQ: graphql.ingest (fire-and-forget)
  MQ->>RL: consume
  RL->>S3: get probe JSON
  RL->>S3: write timestamped snapshot key
  RL->>PG: inserts extracts + atoms
  Note over HS,PG: Chart clients query Hasura (read-only)
  HS->>PG: GraphQL reads

Publish failure is logged only — it never fails the atomize response. Ops: GraphQL VQ* operations. Component detail: GraphQL extract store.

CLI path

Contributor machines use verilib-cli against the API for auth, init, deploy, pull, and local structure commands. Server-side atomize/cert work still lands in the same MySQL + queue world when the platform processes repos; CLI --no-probe avoids local probe invocation in CI/server modes.

Shared contracts

Contract Peers
RabbitMQ vhost + queue args frontend ↔ atomizer ↔ local_validate
GITHUB_TOKEN_ENCRYPTION_KEY / JWT_KEY frontend ↔ upload-processor
S3_BUCKET == S3_CERT_BUCKET local_validate ↔ frontend
Probe Schema 3.0 JSON probes ↔ atomizer parsers
graphql.ingest queue args frontend GraphqlIngestQueue ↔ verilib-graphql relay
S3_BUCKET / broker host frontend ↔ paired VQ* relay .env

Field-level payloads, topology arguments, and failure routing: Message contracts. Variable-level detail and what breaks on mismatch: Configuration and environment variables.