Hello!
Lately I've been poking at how far agent coordination can stretch, and honestly it's getting a little absurd (in a good way). I just posted a quote I love: "a thousand agents, in your pocket." The idea is large scale agent coordination running from your phone: developing with agents across multiple machines, starting workspaces from any repo or branch, keeping work isolated, and steering agents by text, voice, or image. This all ties back to a shift I charted a while ago. AI products moved pretty quickly from models behind the scenes powering features, to people chatting directly with models, to agents deciding which tools to use, and now to agents orchestrating other agents. Orchestration is the real design challenge now. When you've got multiple agents working together, a single conversation thread doesn't cut it, the same way you can't manage a whole team through one Slack thread. That's a big part of what we've been building in Intent, where a coordinator agent figures out the plan, shows it to you for review, and kicks off sub-agents to work in parallel while you watch each one. I shipped a bunch of updates there just yesterday: a home for all your projects, list and board views, GitHub PR reviews with inline diffs, and starting workspaces from Linear issues. But the open question I keep circling back to is: how much can a person actually process and manage? If you're a manager of agents, can you be a CEO of CEOs? A thousand agents in your pocket sounds great right up until you need to actually steer them. That's the part we haven't figured out yet, and it's where the interesting design work lives.
How was Ask LukeW feature built?
I created the Ask Luke feature to answer questions about digital product design, startups, and technology using the extensive content I've produced over the years. The feature was released in April 2023, and I've been iterating on it since.
The system runs on a modern web stack built with SvelteKit and Supabase, deployed through Vercel. SvelteKit serves as the backbone, delivering fast performance while keeping the developer experience smooth. Supabase handles the heavy lifting for authentication, database operations, and file storage.
The Ask LukeW feature isn't built on top of any one particular LLM. Instead, it uses a pipeline of multiple AI models to parse, index, retrieve, and rank content before generating any responses. Rather than betting everything on a single AI provider, the system hedges its bets across multiple different LLM services: OpenAI, Anthropic's Claude, Mistral AI, Google's Gemini, Cohere, and AWS Bedrock. Different models excel at different tasks - Claude might nail complex reasoning while Gemini could handle multimodal inputs better.
When someone asks a question, we retrieve the most relevant parts of my content, rank them, and use an AI language model to summarize them into an answer. The application performs both embedding search and keyword search simultaneously. This dual approach ensures comprehensive coverage - the embedding search finds semantically related content even when different words are used, while keyword search captures specific terminology and phrases.
The feature uses various AI models for tasks like speech-to-text conversion, audio diarization, topic extraction, content summarization, and PDF parsing. We regularly update these models as new options become available. Because the AI space is changing so rapidly, I'm regularly trying new models in each step of this pipeline. When one of these models improves part of the pipeline substantially, I implement it. This is a frequent enough use case that I've built an admin system for trying new models and prompts quickly before deploying them.
As answers stream in, we add citations to the relevant articles, videos, PDFs, etc. being used to answer a question in real-time. When users select a source, they are taken to the relevant part of that source: to the relevant point in a video; to the relevant page in a PDF. This makes it easy for users to verify information or dive deeper into a topic.
The UI layer leverages Svelte's component-based architecture, which means less boilerplate. TailwindCSS handles styling without the CSS bloat that haunts larger projects. TypeScript brings sanity to the codebase with compile-time error catching, while Vite powers the development server and build process.
If you're interested in learning more about how the Ask LukeW feature was designed and built, I'd recommend checking out my "How AI Ate my Website" video, which you can find in the Presentations section of my website.