Stealth App — Server Communication

This is the version 4.13+ revision of the stealth app's network reference. It documents every network exchange the stealth agent makes — the C++ app plus the Go svcmonitor service — and reflects the current architecture.

What changed since the earlier revision

• Datadog / New Relic log streaming has been removed entirely. The agent no longer ships logs to http-intake.logs.datadoghq.eu (or any third party). Diagnostic logs stay on disk and are only ever uploaded — on demand — to your tenant's own storage via the origin server (see Log shipping below).
• The SEND_LOGS WebSocket command is gone. Log shipping is now driven by the SHIP_LOGS service-state command that the Go svcmonitor polls for, and uploaded via a presigned URL.
• Onboarding is performed in-process by the app against the …/api/v2/onboarding/… endpoints (no separate configure_user binary).
• Activity-event sync now treats only 2xx/400 as final; other statuses retry.

Key facts:

• The stealth app runs headless (service controller → activity tracker) and talks to several distinct backends, all rooted in a single hard-coded discovery host.
• The Go svcmonitor does make its own HTTPS calls (discovery, service-state polling, log upload, update checks). The small Go zen_cli helper does not talk to any server — it is a local child process the C++ side invokes over CLI args + stdout JSON.
• All C++ HTTP goes through a single REST client wrapper (60 s timeout). WebSockets go through a shared WebSocket client.

---

Backends / hosts

#	Host	Host source	Auth	Purpose
1	https://origin.in.we360.ai (the "origin server")	Hard-coded (compile-time constant)	public X-Api-Key (reads are otherwise unauthenticated)	Discovery (tenant/device manifests, app versions), service-state commands, service events, log-upload tickets, update download URLs
2	Tenant API gateway	Tenant manifest (apiGatewayUrl)	Authorization: Bearer <token>	Core "me" API, activity-event sync, screenshot/screen-recording upload tickets, tasks, WebSockets
3	Onboarding (same host as #2)	Tenant manifest (apiGatewayUrl)	tenant X-Api-Key (stealth key)	Stealth identity setup + login (/api/v2/onboarding/…)
4	Keycloak auth server	Tenant manifest (apiKeycloakUrl + realm)	client_id + refresh_token	OAuth token refresh
5	Data relay (same host as #2)	Derived from #2 (<apiGatewayUrl>/data_relay/v1)	Authorization: Bearer <token>	Fallback upload path for screenshots / screen recordings when direct cloud upload is blocked
6	Cloud object storage	Signed URL in API response	None (signature embedded in URL)	Direct PUT of screenshot / screen-recording / log-bundle files
7	WebSocket endpoint	Derived from #2 (https:→wss:)	?token=<access_token> query	Remote commands + live streaming

Shared on every authenticated HTTP call: User-Agent: Stealth/<ver> <os>, tz (timezone) on the me endpoints, plus any shared_headers returned in the tenant manifest. The access token, tenant id, auth-server URL/realm, device id, etc. are stored locally in the config store (zs.json for the tenant/stealth key; the per-user INI for tokens).

---

Host resolution — hard-coded vs. tenant manifest

Only one host is baked into the binary; every other host is discovered at runtime.

• https://origin.in.we360.ai is the only hard-coded host. It is the discovery client's fixed entry point and the single root of trust. The app always knows how to reach it with no prior configuration.
• Everything else is bootstrapped from the tenant manifest that the origin server returns and the app persists locally:

  • API gateway host ← apiGatewayUrl (stored under api_gateway_url, legacy fallback coreServiceUrl). Serves the core "me" API, onboarding, raw-event sync, upload-URL requests, the data-relay fallback, and the domain-blocking manifest.
  • Keycloak host + realm ← apiKeycloakUrl and apiKeycloakRealm. The token endpoint is {apiKeycloakUrl}/realms/{realm}/protocol/openid-connect/token.
  • WebSocket host is derived at call time from the API gateway host by swapping https:→wss:.
  • Data-relay base is derived from the API gateway host (…/data_relay/v1).
• Upload hosts (screenshots / screen recordings / log bundles) are neither. The PUT target is a signed URL returned in the body of the corresponding upload-ticket response, so it points at whatever cloud storage the backend chooses at runtime.

Implication: by returning a tenant manifest, the origin server dictates which API gateway and which Keycloak the client uses from that point on. Re-pointing the app to a different environment is done entirely through the manifest — except for the origin host itself, which requires a rebuild.

---

1. Origin server — discovery, service-state, telemetry

Base = https://origin.in.we360.ai. Discovery reads are unauthenticated; device lookup, service-events, service-state, log-upload and download-url calls carry the public X-Api-Key.

Verb & Endpoint	Sends	Receives
GET /discover/tenant_by_id/?id=<tenantId>	—	Tenant manifest (tenant id, api gateway url, keycloak url/realm, base domain, shared headers, SSO config)
GET /discover/tenant_by_base_domain/?baseDomain=<domain>	—	Tenant manifest
GET /discover/device_by_host_name/?tenant_id=&host_name= + X-Api-Key	tenant id, hostname	Device manifest
GET /discover/app_versions/	—	App versions (for the updater)
POST /discover/service-events/ + X-Api-Key	JSON service event (install/update/health/state lifecycle)	—
GET /discover/service-state/?tenantId=&machineId= + X-Api-Key	tenant id, machine id	Pending command for this device: STOP, UNINSTALL, UPDATE, RESTART, SWITCH_KEYCONFIG, SHIP_LOGS, RESEND_HISTORICAL_EVENTS
DELETE /discover/service-state/?tenantId=&machineId= + X-Api-Key	—	Ack — clears the command after the agent has accepted it
POST /discover/log-uploads/request + X-Api-Key	{tenant_id, machine_id, identity_id, request_id}	{upload_id, presigned_put_url, expires_at}
POST /discover/log-uploads/<upload_id>/confirm + X-Api-Key	{size_bytes}	200 (best-effort; bytes are already in storage)
GET /discover/v1/tenant/<tenantId>/download-urls/ + X-Api-Key	—	Per-tenant installer/asset URLs + latest versions + auto_update flag (cached in-process)

The svcmonitor refreshes the tenant manifest on a timer and polls service-state continuously; the C++ app fetches the tenant/device manifests during bootstrap.

---

2. Core "me" API — core API client

Base = tenant gateway. Every call: Authorization: Bearer <accessToken>, tz, User-Agent, Content-Type: application/json. A 401 triggers a Keycloak token refresh; 403 triggers re-onboarding.

Verb & Endpoint	Request body	Response
GET /api/v1/me/manifest/	—	User manifest (user/tenant config, feature flags)
GET /api/v1/me/always-on-mode-login/	—	{ stealth_config: {...} }
GET /api/v1/me/status/	—	User status (punch logs etc.)
GET /api/v1/me/authenticate/	—	200 if the access token is still valid (credential probe)
GET /api/v1/me/latest-ping/	—	Last server-seen ping for this user (health checks)
GET /api/v1/me/break_types/	—	break types list
POST /api/v1/me/v2/punch_in/	{in_time, time_zone, device_id, tracker:{coordinate, isManual, source}}	User status
POST /api/v1/me/punch_out/	punch-log tracker	—
POST /api/v1/me/start_break/	{start_time, name, break_type:{id}, time_zone}	User break
POST /api/v1/me/end_break/	—	—
PUT /api/v1/me/device/	{machine_id, machine_user, machine_host_name, machine_ip, cpu_arch, os_name, os_version, platform}	Device (server device id, stored locally)
GET /api/v1/me/domain-blocking-manifest/	—	map of domain→action (URL blocking rules)
GET /public/time/epochmillis	—	Server epoch (clock-drift check, ~every 6 h; unauthenticated; best-effort)

Task/project/time-log endpoints (/api/v1/me/tasks, /api/v1/projects/, /api/v1/task_activities/, /api/v1/me/time_logs/, …) are also served here and used by the standard app's task features.

---

3. Onboarding — stealth identity setup & login

Base = tenant gateway, prefix /api/v2/onboarding/<tenantId>/. Auth: tenant X-Api-Key (the stealth key from zs.json), not a Bearer token. Performed in-process by the app (the old external configure_user binary no longer does this).

Verb & Endpoint	Sends	Receives
GET …/stealth-configuration/	—	Onboarding policy (identity-generation rules, skip lists, force-refresh flag)
GET …/check-user-exists/?username=<email>	—	{ success: bool }
POST …/setup-stealth-user/?force_refresh=<bool>	identity create DTO (derived from machine identity)	created identity ({ email, … })
POST …/stealth-login/?username=<email>	—	{ access_token, refresh_token, access_token_expiry_epoch }
POST …/standard-login/?username=&password=	username + password (query)	{ access_token, refresh_token, access_token_expiry_epoch } (standard app)

A 403 on any of these means the user/tenant is not permitted; the app stops and stays idle rather than retrying tightly. Successful credentials are written to the local config store.

---

4. Activity & file sync — data sync service

Runs on a 30 s timer (backs off to a reduced frequency on failure). Skips when offline, sync-disabled, or the user is not identified. Base = tenant gateway, Bearer auth.

Verb & Endpoint	Sends	Receives
POST /api/v1/streams/raw_events/multiple/	{items:[ <activity event>, … ], security_events:[…]}	2xx or 400 → events marked synced (final). Other statuses → retried next cycle. 413 → next attempt is re-sent with inline screenshot bytes stripped.
POST /api/v1/screen-recording/get-upload-urls	{items:["<yyyy-MM-dd>/<file>.mp4", …]}	{data:{upload_urls:{file: signedUrl}}}
POST /api/v1/screenshots/get-upload-urls	{items:["<tenant>/<file>.webp", …]}	{data:{upload_urls:{key: signedUrl}}}
PUT <signed url> (cloud storage, off-gateway)	raw .mp4 / .webp bytes	200 → local file deleted
POST <apiGatewayUrl>/data_relay/v1/screen-recordings/<key> · /screenshots/<key> (Bearer)	file bytes (fallback only)	200 → local file deleted

Activity event payload — the core tracking data: id, identity_id, tenant_id, session_id, event_type, application_name, title, url, url_path, path, image_data (base64 screenshot, when inline), screenshot_object_key (when uploaded out-of-band), start_time_epoch, end_time_epoch, system_status, device_id, active_time, idle_time, duration, keystroke_count, mouse_clicks, log_status.

Screenshots can be sent inline (base64 in the event) or out-of-band (written to disk, uploaded via the signed-URL / data-relay path, with only an object key on the event). The data-relay path is the fallback used when a corporate firewall blocks direct uploads to cloud storage. Old local videos (>7 days) are purged each cycle.

---

5. Authentication — token manager

Verb & Endpoint	Sends	Receives
POST {auth_server}/realms/{realm}/protocol/openid-connect/token (x-www-form-urlencoded)	client_id=<tenantId>&refresh_token=<rt>&grant_type=refresh_token	{access_token, refresh_token, expires_in} → stored in config

Refresh is rate-limited (5-min guard) and runs proactively ~5 min before expiry; a heartbeat detects post-sleep clock drift. If the refresh grant fails, the stealth app falls back to a fresh stealth-login. The standard app obtains its first token via …/standard-login/ (see Onboarding).

---

6. Log shipping (on demand)

There is no continuous log streaming. Logs are written to disk locally and only leave the device when an administrator requests them (or the agent self-flags as unhealthy):

1. An admin triggers Ship logs for a device (e.g. from the investigator's user/device page). The backend records a SHIP_LOGS service-state for that tenant + machine.
2. The Go svcmonitor picks it up on its next service-state poll, zips recent local log files (newest-first, capped), and requests an upload ticket: POST /discover/log-uploads/request.
3. It PUTs the zip directly to the returned presigned URL (cloud storage; the origin never sees the bytes), then POST …/confirm.
4. Transient failures are retried on a timer; the command is acknowledged via DELETE /discover/service-state/.

The same upload is also triggered automatically if a device has been unhealthy for over 24 h. This flow is implemented on macOS and Windows.

---

7. WebSocket channel — remote commands + live stream

Transport: a shared WebSocket client with a health-check ping timer, an outbound queue drained on a timer, and auto-reconnect. URLs derive from the API base with https:→wss: and ?token=<accessToken>.

Inbound commands — remote command listener

• Connects (refreshed periodically when authenticated) to wss://<api-host>/websockets/subscribe/me_int?token=<token>.
• Receives JSON commands {eventType, receiver, …}. Handled types:

  • START_LIVESTREAM → add live-stream receiver, start streaming
  • SEND_SCREENSHOT → send a one-off screenshot (or enable continuous screenshot streaming)
  • PUNCH_OUT / FORCE_PUNCH_OUT → triggers a punch-out

  (The legacy SEND_LOGS command has been removed — log shipping is now the SHIP_LOGS service-state flow above.)

Outbound live stream — live streamer

• Per receiver, opens wss://<api-host>/websockets/publish/<identityId>?token=<token>.
• Pushes {payload: <base64 JSON>}:

  • App logs (on a timer): the current activity event plus identity_id, tenant_id.
  • Screenshots (when enabled): {image_data (base64), idle_time, duration, active_time, event_type, system_status, identity_id, tenant_id}.
• Auto-stops after an idle interval; sockets refreshed on a timer.

---

Go svcmonitor & zen_cli

• svcmonitor (the headless service) makes its own origin HTTPS calls: tenant discovery, service-state polling, service events, log uploads, and update checks (/discover/v1/tenant/<id>/download-urls/). It does not touch the tenant gateway's me API.
• zen_cli is a local-only helper invoked by the C++ side over CLI + stdout JSON (processes, terminate-process, and a checks health pass). It opens no network sockets; any data it produces is forwarded to the servers by the C++ app.

---

End-to-end flow (stealth)

1. Bootstrap: the discovery client hits origin.in.we360.ai/discover/* → tenant + device manifests (API gateway URL, Keycloak config, shared headers) saved to the config store.
2. Onboarding & auth: the app resolves its stealth identity via /api/v2/onboarding/… (X-Api-Key) and obtains a Bearer token; thereafter the token manager refreshes it against the tenant's Keycloak.
3. Identity: the core API client registers the device and pulls the user manifest, status, break types, and domain-blocking rules from the API gateway.
4. Tracking: the activity tracker collects app/window/IO events (using the local zen_cli helper for process data); the data-sync service batches them to the raw-events endpoint and uploads screenshots / screen recordings via signed URLs (data-relay fallback).
5. Realtime: the remote-command listener subscribes over WebSocket for commands; the live streamer publishes live app logs/screenshots over WebSocket on demand.
6. Operations: the Go svcmonitor polls service-state for admin commands (update, restart, ship-logs, …) and emits service events; updates are fetched per-tenant from the origin download-urls endpoint.