qa(evidence): live production verification report
Captured against https://canvas.flow-master.ai with browser-style probes + Playwright dogfood + raw fetches: - Every /api/* through canvas's nginx returns 502 (proxy broken). - demo.flow-master.ai upstream is alive (200/401 direct). - Frontend calls auth routes that return 404 even direct on demo: /api/v1/auth/dev-login-config, /api/v1/auth/microsoft/login. - User's 'GETs all 200' hypothesis was wrong — they're 502s. - Zero EA2 traffic reaches the backend. Repros in qa/evidence/_*.mjs. Screenshots + JSON dumps + full report in qa/evidence/.
This commit is contained in:
canvas-bot
@@ -0,0 +1,102 @@
|
||||
# Canvas — production verification report
|
||||
|
||||
**Date:** 2026-06-14
|
||||
**Target:** https://canvas.flow-master.ai
|
||||
**Method:** browser-style probes + Playwright dogfood, no kubectl access.
|
||||
|
||||
## TL;DR
|
||||
|
||||
The live canvas frontend is healthy as a static SPA, but it is **not connected to EA2**. Every `/api/*` call through canvas's nginx proxy returns **502 Bad Gateway**. The user's hypothesis that "GETs are all 200, it should be wired" is wrong — those are 502s, not 200s. Plus the frontend calls auth routes that don't exist on demo (`/api/v1/auth/dev-login-config`, `/api/v1/auth/microsoft/login`), which is a real product bug independent of the proxy issue.
|
||||
|
||||
## Evidence
|
||||
|
||||
### 1. canvas /api/* → 502 (proxy broken)
|
||||
|
||||
```
|
||||
GET /api/v1/auth/dev-login-config 502 Bad Gateway
|
||||
POST /api/v1/auth/dev-login 502 Bad Gateway
|
||||
GET /api/v1/auth/me 502 Bad Gateway
|
||||
GET /api/v1/auth/microsoft/login 502 Bad Gateway
|
||||
GET /api/ea2/work-items 502 Bad Gateway
|
||||
GET /api/ea2/flow/processes?limit=10 502 Bad Gateway
|
||||
GET /api/ea2/process-definitions 502 Bad Gateway
|
||||
GET /api/runtime/transactions 502 Bad Gateway
|
||||
```
|
||||
|
||||
Body of every 502:
|
||||
```
|
||||
<html>
|
||||
<head><title>502 Bad Gateway</title></head>
|
||||
<body><center><h1>502 Bad Gateway</h1></center><hr><center>nginx</center></body>
|
||||
</html>
|
||||
```
|
||||
|
||||
Raw artifacts: `qa/evidence/endpoint-probe.json`, `qa/evidence/canvas-proxy-ea2-error.html`.
|
||||
|
||||
### 2. demo upstream is alive
|
||||
|
||||
```
|
||||
GET https://demo.flow-master.ai/ 200
|
||||
GET https://demo.flow-master.ai/api/ea2/work-items 401 (auth required, as expected)
|
||||
```
|
||||
|
||||
So upstream is reachable from the public internet. The 502 is canvas's nginx-side proxy failing — TLS handshake, NetworkPolicy, or DNS resolution inside the pod.
|
||||
|
||||
### 3. Frontend calls routes that don't exist on demo
|
||||
|
||||
```
|
||||
GET https://demo.flow-master.ai/api/v1/auth/dev-login-config → 404
|
||||
GET https://demo.flow-master.ai/api/v1/auth/microsoft/login → 404
|
||||
POST https://demo.flow-master.ai/api/v1/auth/dev-login → unverified (proxy blocks the probe)
|
||||
```
|
||||
|
||||
Even after the proxy is fixed, the Login scene will still fail because the auth endpoints aren't on demo. Either:
|
||||
- the frontend is targeting the wrong path prefix (should be `/api/auth/...`?), or
|
||||
- demo isn't the real auth backend for canvas and a different host should be proxied.
|
||||
|
||||
### 4. Playwright dogfood against live canvas
|
||||
|
||||
`qa/full_dogfood.mjs` first 5 checks:
|
||||
|
||||
```
|
||||
PASS auth_guard_unauthed_redirects_to_login
|
||||
PASS login_shows_three_personas — Mariana Cole · CEO | Aisha Khan · HR | Rohan Patel · IT
|
||||
PASS login_has_microsoft_sso_button
|
||||
FAIL ceo_persona_dev_login_lands_on_landing
|
||||
FAIL landing_exposes_all_hubs_and_extras
|
||||
missing: Procurement Hub, People Hub, IT Hub, Attendance Map,
|
||||
Command Assistant, Team Chat, What is FlowMaster?
|
||||
```
|
||||
|
||||
The login screen renders correctly. Dev-login click never lands because the dev-login endpoint is 502.
|
||||
|
||||
### 5. Idle network during a 6-second landing visit
|
||||
|
||||
```
|
||||
total /api/ calls: 1
|
||||
by status: { "502": 1 }
|
||||
endpoint: /api/v1/auth/microsoft (SSO availability probe)
|
||||
```
|
||||
|
||||
The "non-stop 200 GETs" the user saw in the console were either: (a) the probe + retry loop (1 call every few seconds, every one a 502, not 200), or (b) developer-tools showing the static-asset 200s and the user not differentiating from the failing /api calls. Either way, **zero EA2 traffic** is reaching the backend.
|
||||
|
||||
## Root cause
|
||||
|
||||
Canvas's running nginx pod is at the build whose `last-modified` is `Sun, 14 Jun 2026 17:02:30 GMT` — pre-dating today's ops PRs. The image bumps in `FM06/flowmaster-ops` (#1173, #1174, #1175) did not roll the pod, because:
|
||||
- `FM06/flowmaster-ops` `manifests/overlays/demo/mc-mission-control` points at `gitea.flow-master.ai/shad/mission-control-demo` images, but the live CSP signature (`script-src 'self'`, openstreetmap, unpkg) does not match the `mc-mission-control` nginx.conf. So the host `canvas.flow-master.ai` is **not** being served by `mc-mission-control` at all.
|
||||
- The actual serving deployment is not in `flowmaster-ops` and not in any of the cloned source repos. Either it's out-of-Gitops on the cluster or in a private repo not exposed.
|
||||
- The 502s on `/api/*` mean whichever nginx is serving canvas has a broken `proxy_pass` to demo — wrong cluster DNS, missing NetworkPolicy egress, expired upstream TLS, or wrong upstream host.
|
||||
|
||||
## What needs cluster access to fix
|
||||
|
||||
1. Identify the deployment+pod actually serving `canvas.flow-master.ai`.
|
||||
2. Verify its nginx.conf `proxy_pass` target and resolve why `proxy_pass https://demo.flow-master.ai` returns 502 from inside the pod.
|
||||
3. Confirm `NetworkPolicy` allows egress to `demo.flow-master.ai`'s IP from the pod's namespace.
|
||||
4. Identify the correct auth API base path (`/api/v1/auth/...` does not exist on demo) and reconfigure the frontend or the upstream.
|
||||
|
||||
Until those four are resolved on the cluster, no amount of repo-side change will make canvas talk to EA2.
|
||||
|
||||
## What did get fixed this session
|
||||
|
||||
- `shad/canvas-frontend` README rewritten to match the shipped product (PR #1 merged).
|
||||
- Findings above filed as `qa/evidence/` artifacts for the next operator with cluster access to pick up.
|
||||
Reference in New Issue
Block a user