Timo AI keeps the LLM logic server-side: prompts and per-user context are assembled in an async FastAPI service, replies stream back token by token, and conversation history is persisted and trimmed to a token budget so chats stay coherent without overrunning the model's context window. The OpenAI calls sit behind a single interface so the provider can be swapped later.
Highlights
Streaming replies
Responses stream token by token over an async pipeline instead of blocking on the full completion.
Context memory
Per-user history is persisted and trimmed to a token budget so conversations stay coherent.
Honest status
A working demo I'm still developing - not yet a live, launched product.
Stack