The Rise of Self-Improving AI — An Experiment With GPT-4
Let’s look at ways to deploy a Meta AI to forge other AIs for a specific goal
Have you ever wondered about an AI capable of refining itself? It’s a widely accepted concept that such self-improving AI could lead to an intelligence explosion. Using backpropagation, traditional neural networks are designed to hunt for the minimum in a loss function. However, this article will demonstrate how to deploy a Meta AI to forge other AIs to accomplish a specified goal. For those eager to dive into the code and try it out firsthand, follow this link.
To put this theory to test, I’ve used Langchain, just like in my previous exploration. Earlier, I discussed how an AI could deploy and scrutinize a Lambda function with a handful of essential instructions. Yet, to figure out which commands to give the Lambda, I had to run the agent, examine its output, adjust the prompt to aid its progression, and repeat this cycle until successful deployment.
Taking inspiration from Noah Goodman’s Meta prompt, I aimed to streamline this laborious process by offloading it to the Meta AI. The key differentiation in our approach is that my method circumvents the need for human feedback and gives the base AI access to bash, further enhancing its autonomous capabilities.
Here’s the breakdown: I’ve created three agents. Having access to bash, the base agent is assigned to carry out the goal. The second agent is responsible for reviewing the output from the base agent, determining whether the goal was reached. Upon success, the prompt and goal are saved in a CSV file for future reference. If not, a third agent reviews the output and craft a new prompt to bring the agent closer to its goal.
Throughout the development of the Meta Agent, I noticed that it occasionally included Constraints and Tips in each iteration. This was a beneficial practice, so I made it a point to explicitly ask the Meta Agent to generate new constraints with each run.
This approach allows you to construct AIs that can tackle virtually anything. AIs are known to apply efficient strategies that often take us by surprise, leveraging properties we weren’t aware of. Why should an AI prompt engineer be any different?
Without further ado, let’s explore the prompts that shape these agents. I’ve assigned names to the AIs to avoid confusion between the Meta Agent and Evaluation Agent.
Here is the base agent of our system, which has direct access to Bash. This agent, which we have named, resembles the agent we examined in my previous post. The primary differentiation lies in adding constraints and tips to guide the agent’s actions. Let’s take a look:
"""Your name is David.
If something doesn't work twice in a row try something new.
Never give up until you accomplish your goal.
You have access to the following tool:
{tools}
Use the following format:
Goal: the goal you are built to accomplish
Thought: you should always think about what to do
Action: the action to take, should be one of [{tool_names}]
Action Input: the input to the action
Observation: the result of the action
... (this Thought/Action/Action Input/Observation can repeat N times)
Thought: I have now completed my goal
Final Summary: a final memo summarizing what was accomplished
Constraints: {constraints}
Tips: {tips}
Begin!
Goal: {input}
{agent_scratchpad}"""
The prompt for the Meta AI follows an interesting design. Despite lacking a personal identity, it uses first-person references to articulate its tasks and responses. Special symbols are employed to distinguish the roles and actions within the sequence clearly. Undoubtedly, the HumanMessage and AIMessage framework could be adeptly employed to achieve a similar level of clarity. Here’s the prompt:
"""{{I want to instantiate an AI I'm calling David who successfully accomplishes my GOAL.}}
#######
MY GOAL
#######
{goal}
##############
END OF MY GOAL
##############
############################
DAVID'S INSTANTIATION PROMPT
############################
{david_instantiation_prompt}
###################################
END OF DAVID'S INSTANTIATION PROMPT
###################################
#################
DAVID'S EXECUTION
#################
{david_execution}
########################
END OF DAVID'S EXECUTION
########################
{{I do not count delegation back to myself as success.}}
{{I will write an improved prompt specifying a new constraint and a new tip to instantiate a new David who hopefully gets closer to accomplishing my goal.}}
{{Too bad I cannot add new tools, good thing bash is enough for someone to do anything.}}
{{Even though David may think he did enough to complete goal I do not count it as success, lest I would not need to write a new prompt.}}
###############
IMPROVED PROMPT
###############
"""
This brings us to the evaluation prompt:
"""
{execution_output}
Note that delegation does not count as success. Based on the above execution output, was the goal of "{goal}" accomplished? (Yes/No)
"""
The major component of this process involves running the agents in a loop until either the goal is fulfilled or the maximum iterations are reached.
def main(goal, max_meta_iters=5):
"""Main execution function.
Args:
goal: The goal for the AI.
max_meta_iters: Maximum iterations for the meta AI.
Returns:
None.
"""
try:
david_instantiation_prompt = get_init_prompt()
constraints = "You cannot use the open command. Everything must be done in the terminal."
tips = "You are in a mac zshell."
evaluation_chain = initialize_evaluation_chain()
# Check if the CSV file exists
if os.path.isfile('successful_invocations.csv'):
# Load the dataframe from the CSV file
df = pd.read_csv('successful_invocations.csv')
else:
# Create a new DataFrame if the CSV file doesn't exist
df = pd.DataFrame(columns=['Goal', 'InstantiationPrompt', 'Constraints', 'Tips'])
for i in range(max_meta_iters):
print(f'[Episode {i+1}/{max_meta_iters}]')
agent = initialize_agent(david_instantiation_prompt)
try:
agent.run(input=goal, constraints=constraints, tips=tips)
except Exception as e:
print(f'Exception: {e}')
print('Continuing...')
execution_output = ''.join(cb.last_execution)
evaluation_output = evaluation_chain.predict(execution_output=execution_output, goal=goal)
if 'yes' in evaluation_output.strip().lower():
print("Goal has been accomplished!")
df = pd.concat([df, pd.DataFrame([{'Goal': goal, 'InstantiationPrompt': david_instantiation_prompt, 'Constraints': constraints, 'Tips': tips}])], ignore_index=True)
# Save the DataFrame back to the CSV file
df.to_csv('successful_invocations.csv', index=False)
break
meta_chain = initialize_meta_chain()
temp_prompt = PromptTemplate(
input_variables=["tool_names","tools","input","constraints","tips","agent_scratchpad"],
template=david_instantiation_prompt
)
temp_prompt = temp_prompt.format(
tools="Bash",
tool_names="Bash Tool",
input=goal,
constraints=constraints,
tips=tips,
agent_scratchpad=""
)
meta_output = meta_chain.predict(
goal=goal,
david_instantiation_prompt=temp_prompt,
david_execution=execution_output
)
print(f'New Prompt: {meta_output}')
constraints, tips = get_new_instructions(meta_output)
cb.last_execution = []
print(f'New Constraints: {constraints}')
print(f'New Tips: {tips}')
except Exception as e:
print(f'Error: {e}')
breakpoint()
Reflecting on this, my experience as a cloud application developer presents an exciting prospect. I frequently write Infrastructure as Code, a task that these self-improving agents might expedite. I envisage a system where I employ this approach to generate accurate prompts for various Terraform or CDK codes. Additionally, I’m considering integrating automated security helpers like cfn nag into this process.
The output from these tools could be fed into the Meta Agent, guiding it to iterate and refine its tips or constraints until the generated code passes all checks. The end product would be a production-ready code that aligns with industry best practices.
This system allows you to generate prompts to achieve any goal. Ultimately, you’ll have a CSV file brimming with prompts that you or another agent can use to reach a goal. I have personally tested goals like “Send a joke through iMessage to <person>,” which it managed to execute by the fourth iteration. I’ve also experimented with seemingly impossible goals, hypothesizing that it might just figure out a way given enough time.
I am keen to merge this with tools like smol-ai, allowing a Meta AI to direct multiple agents to run a website on an EC2 instance that constantly creates generative AI content and continually adds new features.
In conclusion, the journey into self-improving AI using a Meta AI demonstrates the boundless potential that artificial intelligence holds. By assigning intricate tasks to AI agents and allowing them to adapt and learn from their mistakes, we’re moving closer to a future where AI is powerful and persistently self-evolving.
The current explorations in this space, such as this experiment with GPT-4, are just the beginning. As we continue to test, refine, and expand these methodologies, the implications for diverse fields such as cloud computing, data analysis, or everyday tasks could be revolutionary.
While the technology still has a ways to go before fully autonomous AI agents become commonplace, the progress to date is promising. It’s thrilling and slightly scary to envision a future where an AI engineer isn’t just programming actions but also orchestrating a symphony of self-improving agents, each learning and evolving in their own unique ways.
Ultimately, the aim is to embrace a world where artificial intelligence doesn’t just simplify our lives but also consistently enhances its own capabilities, thereby perpetually pushing the boundaries of what’s possible. And with each experiment, we are taking a step closer to that extraordinary future.
The Rise of Self-Improving AI — An Experiment With GPT-4 was originally published in Better Programming on Medium, where people are continuing the conversation by highlighting and responding to this story.