60 lines
2.1 KiB
Markdown
60 lines
2.1 KiB
Markdown
# canvas-llm-proxy
|
|
|
|
In-cluster sidecar for `canvas.flow-master.ai`. Brokers
|
|
`POST /internal/canvas-llm/chat` to the configured upstream LLM provider so
|
|
the in-app Command Assistant can synthesise fluent natural-language replies
|
|
when no deterministic tool matches.
|
|
|
|
## Why a Node sidecar (not a Python one)
|
|
|
|
This was a Python `httpx` proxy that hand-wrote OpenAI and Anthropic
|
|
calls. PR canvas-frontend#6 swapped it for a Node sidecar that depends on
|
|
[`@earendil-works/pi-ai`](https://www.npmjs.com/package/@earendil-works/pi-ai)
|
|
— the same unified provider client `@earendil-works/pi-coding-agent` uses.
|
|
Why:
|
|
|
|
- It is the OSS-first move the canvas user asked for ("you could use a fork
|
|
pi coding agent for that agent and then integrate it because pi coding
|
|
agent is very clean and small while being efficient and useful").
|
|
- `pi-ai` normalises streaming completions across OpenAI (completions +
|
|
responses), Anthropic, Google Gemini, Google Vertex, Amazon Bedrock,
|
|
Azure OpenAI, Mistral, GitHub Copilot. Canvas inherits all of them.
|
|
- A future move to pi-ai's tool-calling shape (so the agent can hit real
|
|
EA2 endpoints, not just our deterministic regex matcher) becomes
|
|
mechanical instead of a rewrite.
|
|
|
|
## Wire-compatible
|
|
|
|
The HTTP contract is the same: `POST /internal/canvas-llm/chat` with
|
|
`{messages, max_tokens}` returns `{content, provider}`. The frontend's
|
|
`src/lib/llmClient.ts` does not change.
|
|
|
|
## Configuration
|
|
|
|
Set one of: `ANTHROPIC_API_KEY`, `OPENAI_API_KEY`, `GEMINI_API_KEY`.
|
|
Optional: `ANTHROPIC_MODEL`, `OPENAI_MODEL`, `GEMINI_MODEL`, `PORT` (8080).
|
|
|
|
If no key is set, every request returns `503` so the frontend's
|
|
deterministic-tool fallback fires gracefully.
|
|
|
|
## Run
|
|
|
|
```bash
|
|
cd llm-proxy
|
|
npm install
|
|
ANTHROPIC_API_KEY=… node server.mjs
|
|
```
|
|
|
|
Container:
|
|
|
|
```bash
|
|
docker build -f Dockerfile.node -t canvas-llm-proxy .
|
|
docker run -e ANTHROPIC_API_KEY=… -p 8080:8080 canvas-llm-proxy
|
|
```
|
|
|
|
## Legacy
|
|
|
|
The previous Python sidecar (`main.py`, `requirements.txt`, `Dockerfile`)
|
|
is retained for one release so deployers can roll back. It will be
|
|
removed once the Node sidecar is in production.
|