This is the same click-to-stream chat as KobeLLM, but every message is routed through ts-llm-gateway, a multi-provider LLM gateway in strict TypeScript, to AWS Bedrock (with OpenAI as automatic failover). Behind the stream sit real production policies: per-key/IP rate limiting, retry with backoff, a circuit breaker that quarantines a downed provider, and an LRU response cache. It's the same class of system I built at work, a real-time streaming pipeline, re-expressed as an LLM proxy. Your browser only ever talks to my Go backend, which holds the gateway key; the counters below are the gateway's own live /stats, proxied through that backend.
A live chat routed through ts-llm-gateway → AWS Bedrock.
- Routing
- AWS Bedrock
- Failover
- OpenAI
- Policies
- rate-limit · retry · breaker · cache
- Transport
- SSE · cancellable
Disclaimer: responses are generated by a third-party foundation model via AWS Bedrock and may be inaccurate. Conversations are rate-limited, cost-capped, and not stored. Do not enter sensitive information.