kobe young · software · systems · inference
Building the behind-the-scenes systems that move data and run AI models, the parts people never see but use every day. Concretely: real-time data pipelines, RAG/LLM serving, and distributed backends on AWS and Azure. Former Navy avionics technician and residential electrician, now working my way down the stack toward systems and inference.
// self-hosting the whole stack on one box · owner of multiple cats
ts-llm-gateway
TypeScript · Vercel AI SDK · AWS Bedrock + OpenAI · Hono · Upstash RedisA production-style, multi-provider LLM gateway in strict TypeScript: one unified API plus an OpenAI-compatible surface (a drop-in baseURL swap) that routes to AWS Bedrock and OpenAI behind a single injected provider interface, wrapped in real production policies: per-key/IP rate limiting, retry with backoff and cross-provider failover, a circuit breaker that quarantines a downed provider and probes it back, per-call timeouts, an LRU response cache, and SSE streaming that aborts the upstream call when the client disconnects.
The point: it's the same class of system I built at work, a real-time streaming pipeline, re-expressed as an LLM proxy. Kinesis backpressure became rate limiting; retry/backoff became failover; quarantining a bad dependency became the circuit breaker; shard-consumer cancellation became streaming abort; composite routing keys became provider routing; SignalR fan-out became SSE.
one native endpoint plus a drop-in /v1/chat/completions
per-key & per-IP token buckets, constant-time key checks
exponential backoff + jitter across providers
quarantines a downed provider, half-open probes it back
SSE tokens; client disconnect cancels the upstream call
LRU response cache; in-memory or Upstash Redis, degrades gracefully
// ~6× cache-hit speedup and cross-provider failover both verified on the live deployment
Civic Lens
Go · Python · SQLite · FastAPI · React · LLM pipelineAn audit-driven system that measures sampled political discourse across news, Reddit, and X: a polite, crash-safe Go crawler, an LLM analysis pipeline where every output carries a confidence score and a verbatim evidence span, narrative clustering, and a CI regression gate for prompt changes. Open source, deployed, and documented down to the invariants.
Go crawler → SQLite → Python analysis → FastAPI → React
LLM evals & CI gate →golden set, replay mode, prompt fingerprinting
Scoring methodology →bot detection, sentiment, favorability, propaganda
Database schema →16 tables, 21 migrations, full traceability
System invariants →the correctness contract every change must keep