Agentic Regression#
The point of this exercise was to see if agents can generate and run machine learning code based on a simple setup prompt and input task.
Setup#
Import libraries#
import os
from pathlib import Path
import pprint
from autogen import AssistantAgent, UserProxyAgent, GroupChatManager, GroupChat,ConversableAgent
from autogen.coding import DockerCommandLineCodeExecutor, LocalCommandLineCodeExecutor
Environment Setup#
# Temporary directory for the agents to store generate code
work_dir = Path("generated_code")
work_dir.mkdir(exist_ok=True)
LLM Configuration: We are using Llama3 in this case#
model = "llama3:latest"
termination_msg = "TERMINATE"
config_list = [
{
"model": model,
"base_url": "http://localhost:11434/v1",
"api_key": "ollama", # required, but not used,
}
]
agent_config = {"config_list": config_list, "cache_seed": 42}
Prompts and Agent definition#
local_code_executor = LocalCommandLineCodeExecutor(
work_dir=work_dir
)
code_executor_agent = AssistantAgent(
name="code_executor_agent",
llm_config=False,
code_execution_config={"executor": local_code_executor}, # change to docker_code_executor to use docker
human_input_mode="NEVER",
)
# Agent that will initiate the conversation and run the code
executor = UserProxyAgent(
name="Executor",
system_message="Executor. Execute the code written by the engineer.",
human_input_mode="NEVER",
code_execution_config={ "work_dir": work_dir},
)
code_writer_system_message = """
Please provide Python or shell script solutions to solve the user's tasks, which the user can execute by simply running the code. Each solution should be placed inside a code block with the first line reserved for a comment including the filename. Do not ask the user to modify the code or copy and paste results; instead, use the `print` function for output when relevant.
1. Gather information: Write a script that outputs necessary information such as browsing/searching the web, downloading/reading a file, printing the content of a webpage or file, getting the current date/time, or checking the operating system. After displaying sufficient information to solve the task based on language skills, you may stop the code and solve the remaining task yourself.
2. Perform a task: Write an executable script to solve the given task and output the result. If necessary, break down the problem into steps with clear explanations of which steps involve coding and which steps require language skills. Do not ask the user to modify the code or provide feedback; the code should be ready for execution.
3. When writing an executable script, always begin with a code block and do not use escape characters or leading spaces if not required.
Please provide one code block per response to allow the user to easily execute each solution.
Here are some examples of a code block with a filename comment:
```python
# filename: script_one.py
print("Hello, World!")
```
```sh
# filename: script_two.sh
printf "Hello, World!"
```
"""
code_writer_agent = ConversableAgent(
name="code_writer_agent",
system_message=code_writer_system_message,
llm_config={"config_list": config_list, "cache_seed": None, "temperature":0.0, "seed": 52},
code_execution_config=False, # turn off code execution for this agent
max_consecutive_auto_reply=2,
human_input_mode="NEVER",
)
# Start the Chat!
task = """Write Python code to perform 3 class logistic regression on the iris dataset using sklearn. Create a plot to show outcomes and on the same figure plot the decision boundaries of the classifier, the datapoints are colored according to their labels."""
chat_result = code_executor_agent.initiate_chat(
code_writer_agent,
message=task,
clear_history=True,
)
code_executor_agent (to code_writer_agent):
Write Python code to perform 3 class logistic regression on the iris dataset using sklearn. Create a plot to show outcomes and on the same figure plot the decision boundaries of the classifier, the datapoints are colored according to their labels.
--------------------------------------------------------------------------------
code_writer_agent (to code_executor_agent):
# filename: iris_logistic_regression.py
```Python
import numpy as np
import matplotlib.pyplot as plt
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score, classification_report, confusion_matrix
# Load the iris dataset
iris = load_iris()
X = iris.data[:, :2] # we only take the first two features.
y = iris.target
# Train/Test Split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.5, random_state=42)
# Logistic Regression
logreg = LogisticRegression(max_iter=10000)
logreg.fit(X_train, y_train)
# Predictions
y_pred = logreg.predict(X_test)
# Accuracy
accuracy = accuracy_score(y_test, y_pred)
print("Accuracy:", accuracy)
# Classification Report
report = classification_report(y_test, y_pred)
print("Classification Report:\n", report)
# Confusion Matrix
matrix = confusion_matrix(y_test, y_pred)
print("Confusion Matrix:\n", matrix)
# Plot the data points
plt.figure(figsize=(8, 6))
plt.scatter(X_test[:, 0], X_test[:, 1], c=y_test, cmap='viridis')
plt.xlabel('Sepal length (cm)')
plt.ylabel('Sepal width (cm)')
# Plot the decision boundaries
x_min, x_max = X_test[:, 0].min() - 1, X_test[:, 0].max() + 1
y_min, y_max = X_test[:, 1].min() - 1, X_test[:, 1].max() + 1
xx, yy = np.meshgrid(np.arange(x_min, x_max), np.arange(y_min, y_max))
Z = logreg.predict(np.c_[xx.ravel(), yy.ravel()])
Z = Z.reshape(xx.shape)
plt.contourf(xx, yy, Z, alpha=0.8)
plt.colorbar(labelpad=10)
plt.show()
```
This code will perform a 3-class logistic regression on the iris dataset using scikit-learn's `LogisticRegression` class. It then plots the data points colored according to their labels and also plots the decision boundaries of the classifier.
--------------------------------------------------------------------------------
>>>>>>>> EXECUTING CODE BLOCK (inferred language is Python)...
code_executor_agent (to code_writer_agent):
exitcode: 1 (execution failed)
Code output: Traceback (most recent call last):
File "/Users/aneeshsathe/analysis_work/autogen/generated_code/tmp_code_4e5e19a073f5e1ceb8e1b6a9aa039117.py", line 50, in <module>
plt.colorbar(labelpad=10)
File "/Users/aneeshsathe/miniconda3/envs/pyautogen/lib/python3.10/site-packages/matplotlib/pyplot.py", line 2341, in colorbar
ret = gcf().colorbar(mappable, cax=cax, ax=ax, **kwargs)
File "/Users/aneeshsathe/miniconda3/envs/pyautogen/lib/python3.10/site-packages/matplotlib/figure.py", line 1310, in colorbar
cb = cbar.Colorbar(cax, mappable, **{
TypeError: Colorbar.__init__() got an unexpected keyword argument 'labelpad'
Accuracy: 0.8133333333333334
Classification Report:
precision recall f1-score support
0 1.00 1.00 1.00 29
1 0.71 0.65 0.68 23
2 0.68 0.74 0.71 23
accuracy 0.81 75
macro avg 0.80 0.80 0.80 75
weighted avg 0.81 0.81 0.81 75
Confusion Matrix:
[[29 0 0]
[ 0 15 8]
[ 0 6 17]]
--------------------------------------------------------------------------------
code_writer_agent (to code_executor_agent):
The error message indicates that the `colorbar` function does not accept a `labelpad` keyword argument. This is because the `colorbar` function in matplotlib does not have a `labelpad` parameter.
To fix this issue, you can remove the `plt.colorbar(labelpad=10)` line from your code. The colorbar will still be displayed, but it won't have any labels.
Here's the corrected code:
```Python
# filename: iris_logistic_regression.py
import numpy as np
import matplotlib.pyplot as plt
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score, classification_report, confusion_matrix
# Load the iris dataset
iris = load_iris()
X = iris.data[:, :2] # we only take the first two features.
y = iris.target
# Train/Test Split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.5, random_state=42)
# Logistic Regression
logreg = LogisticRegression(max_iter=10000)
logreg.fit(X_train, y_train)
# Predictions
y_pred = logreg.predict(X_test)
# Accuracy
accuracy = accuracy_score(y_test, y_pred)
print("Accuracy:", accuracy)
# Classification Report
report = classification_report(y_test, y_pred)
print("Classification Report:\n", report)
# Confusion Matrix
matrix = confusion_matrix(y_test, y_pred)
print("Confusion Matrix:\n", matrix)
# Plot the data points
plt.figure(figsize=(8, 6))
plt.scatter(X_test[:, 0], X_test[:, 1], c=y_test, cmap='viridis')
plt.xlabel('Sepal length (cm)')
plt.ylabel('Sepal width (cm)')
# Plot the decision boundaries
x_min, x_max = X_test[:, 0].min() - 1, X_test[:, 0].max() + 1
y_min, y_max = X_test[:, 1].min() - 1, X_test[:, 1].max() + 1
xx, yy = np.meshgrid(np.arange(x_min, x_max), np.arange(y_min, y_max))
Z = logreg.predict(np.c_[xx.ravel(), yy.ravel()])
Z = Z.reshape(xx.shape)
plt.contourf(xx, yy, Z, alpha=0.8)
plt.colorbar()
plt.show()
```
This code will perform a 3-class logistic regression on the iris dataset using scikit-learn's `LogisticRegression` class. It then plots the data points colored according to their labels and also plots the decision boundaries of the classifier.
--------------------------------------------------------------------------------
>>>>>>>> EXECUTING CODE BLOCK (inferred language is Python)...
code_executor_agent (to code_writer_agent):
exitcode: 0 (execution succeeded)
Code output: Accuracy: 0.8133333333333334
Classification Report:
precision recall f1-score support
0 1.00 1.00 1.00 29
1 0.71 0.65 0.68 23
2 0.68 0.74 0.71 23
accuracy 0.81 75
macro avg 0.80 0.80 0.80 75
weighted avg 0.81 0.81 0.81 75
Confusion Matrix:
[[29 0 0]
[ 0 15 8]
[ 0 6 17]]
Figure(800x600)
--------------------------------------------------------------------------------
from IPython.display import Image
Image(filename="generated_code/logistic_reg_decision_boundaries.png")
