Introduction
Have you ever noticed how agonizingly slow most AI agents are? You ask them to research a topic, write a summary, and email it to you. You sit there watching a loading spinner as the agent thinks, searches, thinks again, writes, thinks again, and finally sends the email. It feels like watching someone type with one finger.
When I began engineering FRIDAY, my local-first Linux voice assistant, I realized that the standard approach to building AI agents was fundamentally flawed. The theoretical deep dive here isn't about natural language processing; it's about execution topography.
The Sequential Bottleneck of ReAct Models
Standard agents use a paradigm called 'Reason and Act' (ReAct). The LLM enters a sequential loop: it observes the state, decides on one tool to use, waits for the result, observes the new state, and repeats. If you give it five independent tasks—like checking the weather in five different cities—it will execute them one by one. This serial execution model is a massive bottleneck. Humans don't work like this. If we have independent tasks, we delegate them simultaneously.
What if we stopped treating an AI agent as a conversationalist and started treating it as a system compiler?
Architecture
To solve this, I built the TaskGraphExecutor. Instead of letting the LLM run a sequential loop, FRIDAY uses the LLM solely as a planner. The LLM's job is to take your prompt and output a Directed Acyclic Graph (DAG) of necessary tool calls. A DAG is a mathematical concept used in computer science to represent tasks and their dependencies. For example, 'Emailing the summary' depends on 'Writing the summary', which depends on 'Searching the web'. But 'Searching the web for Topic A' and 'Searching the web for Topic B' have zero dependencies on each other.
Once the DAG is generated, FRIDAY computes the topological waves of the graph. It calculates the 'in-degree' (number of unresolved dependencies) for every node. Any node with an in-degree of 0 is fired off immediately and asynchronously in parallel. As those tools return data, the graph resolves, and the next 'wave' of dependent tools is triggered. By shifting from a sequential loop to a topological wave execution, FRIDAY can accomplish complex, multi-step CLI and browser tasks in a fraction of the time.
What I prioritized
The theoretical takeaways from treating AI as a Graph Compiler:
- Directed Acyclic Graphs. Mapping LLM intents into mathematical nodes and edges to establish strict execution dependencies.
- Topological Wave Execution. Dynamically computing in-degrees to fire all independent tools simultaneously.
- Separation of Concerns. Decoupling the 'Planning' (LLM inference) from the 'Execution' (Python async loops).
- Microsecond Latency. Eliminating the need for the LLM to re-evaluate state after every single tool call.
