Building a Multi Agent based — “Auto Recursive” — Plan , Execute, Re-Plan Process

Rifx.Online
Programming , Autonomous Systems , Generative AI
27 Dec, 2024

This blog has below 3 sections :

The problem statement
The solution approach
Conclusions and References

Problem Statement

Plan, Execute & Re-Plan process is not new in the word of agentic solutions. We have been implementing these agent based plan-execute-re_plan process since last year.

So, when one of my colleague was discussing the challenges of implementing the same, I was intrigued a I thought it was a problem well discussed.

But when I was talking to him, I understood the nature and the complexities of the scenario of what he planned to do and I thought it will be a good opportunity to work together.

So, to layout the content, below are the scenarios of what I though and what the problem statement was.

Scenario 1 (what was my understanding) :

Agentic solution can be designed in multiple ways. Agents can be designed as supervisor<-> workers ; they can be designed as network of Agents talking to each other in a acyclic way; they can be hierarchical etc.

My understanding was that, in its simplest form, we can implement a plan-execute-re_plan process using one Agent and multiple tools. This was good enough for simple use cases. When I shared the multi-tool (single agent) used to solve this use case, we found that it served the simpler scenarios.

Scenario 2(a bit more advanced but aligned to Langchain) :

In this second scenario, I brought in another solution. I talked about Langchain’s own pre-built agent that can help implement this loop. Trust me, this got us excited for quite sometime as it was able to loop through and iterate of the plan to generate execution outcomes. For more details about this — check out @ https://github.com/langchain-ai/langchain/blob/master/cookbook/plan_and_execute_agent.ipynb

This is out of box agent that contains load_chat_planner and PlanAndExecute modules within the plan_and_execute Agent API.

This agent (provided by Langchain) was inspired by BabyAGI . The core of the solution focusses on planner and an executer.

What is missing in these 2 scenarios (Scenario 3):

When I was talking to my colleague, the genuine issue that he was trying to solve was the dynamic adaptation of the agents to plan, take action and re-plan based on previous action. The single Agent in scenario 2 was ok for certain scenarios — but when it comes to complex multi step process, it was struggling.

Secondly, in the plan-execute-replan process — we need more flexibility. We need to call external systems while planning and executing steps. This meant that we needed to design a recursive, multi agent system that can keep processing until it can effectively complete the goal.

So, without much ado, lets get started:

Solution

Let us start with the overall design of the plan-execute-replan process that we ended up designing. The overall solution is based on the multi-agent framework I am working on for a while. I did write more than 10 blogs on the multi-agent framework, you can find the links at the “conclusion” section below.

When the workflow starts, the initial supervisor agent kicks off. The supervisor agent can setup the initial configurations (like the acyclic DAG, memory, prompts etc.) so that the other independent agents can work in tandem to converge to the goal. This is depicted in step 2, 3, when the supervisor agent setup and initialization is achieved.

Next (in step 4) — the DAG goes through the first agent — if the agent requires a human intervention (like the agent requires a human to ask a question or confirm a decision) — then the process pauses for human to take that action and maintains state.

In step 6- the agent can call the associated tool to complete the agent action and based on that calls the next step of the process. The next step is the executor process that executes the command.

Next, the system checks if the task list is complete, if the whole set of task is complete then the system respons back to the user, else it calls the replanner agent in a recursive way as shown below.

To understand how the system behaves, I have another process flow that provides the use case level process flow (compared to the above architecture blueprint).

In this case — when the user asks a question — the planner agent is triggered, the planner will generate the detailed task list as shown. From that task list, the executor agent will be called recursively.

For the first task, the executor agent will call the appropriate tool to get the response. Based on response the re-planner agent will be called. The re-planner agent will decide if a replan is needed or not. It will also decide of all pending tasks are complete or not.

If complete, then it will collect the response and sent to user, else it will call the next step recursively.

Now let us look into the solution. The solution is designed on Azure Gen AI Foundation framework with same look and feel (check the framework @ https://medium.com/@nayan.j.paul/gen-ai-building-adoption-through-a-common-platform-instead-of-enabling-be-spoke-use-cases-b0cbc2e185a8)

The system asks us to provide a question based on which it will design the plan.

This is the same guardrails and state management which ensures that the system will not move forward unless the agent specific details are provided (in this case it is the question).

I start by asking the system to give me quick steps to prepare expresso coffee. The question tiggers the iterative and recursive plan-execute-replan process. The system then goes to a auto-drive mode recursively calling and executing next steps.

If we see below, it starts with the plan step to give me 3 steps to execute the process. Then it moved to execute plan to execute the first task.

Once the first task is complete, the system automatically then identifies that the plan is in progress and then calls the re-plan step. The replan step then calls the execute plan step again in loop untill we reach the end of all processing as below.

All of this is automated where at every step the system makes decision to go to next step and process.

This shows the iterative nature of this agentic solution where the processes are recursive and automated.

Conclusion

If you have any other question for me to try, do send them to me. I will share how the framework behaves in complex scenarios. I am testing this will customer use case now and I will keep you all posted.

Now, I have been working on Agents for quite a while now, below are some of the agentic use cases I implemented, I will mention these cases for your reference: