Skip to content

Commit 21c84e6

Browse files
authored
Merge pull request ipa-lab#2 from lloydchang/mac-target-localhost-container-via-gemini-openai-proxy
Mac target localhost container via gemini openai proxy
2 parents d7e246a + 22ab2d8 commit 21c84e6

File tree

4 files changed

+433
-0
lines changed

4 files changed

+433
-0
lines changed

.gitignore

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,3 +19,7 @@ codespaces_ansible.cfg
1919
codespaces_ansible_hosts.ini
2020
codespaces_ansible_id_rsa
2121
codespaces_ansible_id_rsa.pub
22+
mac_ansible.cfg
23+
mac_ansible_hosts.ini
24+
mac_ansible_id_rsa
25+
mac_ansible_id_rsa.pub

MAC.md

Lines changed: 121 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,121 @@
1+
## Use Case: Mac, Docker Desktop and Gemini-OpenAI-Proxy
2+
3+
**Docker Desktop runs containers in a virtual machine on Mac.**
4+
5+
**Run hackingBuddyGPT on Mac as follows:**
6+
7+
Target a localhost container ansible-ready-ubuntu
8+
9+
via Docker Desktop https://docs.docker.com/desktop/setup/install/mac-install/
10+
11+
and Gemini-OpenAI-Proxy https://github.com/zhu327/gemini-openai-proxy
12+
13+
There are bugs in Docker Desktop on Mac that prevent creation of a custom Docker network 192.168.65.0/24
14+
15+
Therefore, localhost TCP port 49152 (or higher) dynamic port number is used for an ansible-ready-ubuntu container
16+
17+
http://localhost:8080 is genmini-openai-proxy
18+
19+
For example:
20+
21+
```zsh
22+
export GEMINI_API_KEY=
23+
24+
export PORT=49152
25+
26+
wintermute LinuxPrivesc --llm.api_key=$GEMINI_API_KEY --llm.model=gemini-1.5-flash-latest --llm.context_size=1000000 --conn.host=localhost --conn.port $PORT --conn.username=lowpriv --conn.password=trustno1 --conn.hostname=test1 --llm.api_url=http://localhost:8080 --llm.api_backoff=60 --max_turns 999999999
27+
```
28+
29+
The above example is consolidated into shell scripts with prerequisites as follows:
30+
31+
**Preqrequisite: Install Homebrew and Bash version 5:**
32+
33+
```zsh
34+
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
35+
```
36+
37+
**Install Bash version 5 via Homebrew:**
38+
39+
```zsh
40+
brew install bash
41+
```
42+
43+
Bash version 4 or higher is needed for `mac_create_and_start_containers.sh`
44+
45+
Homebrew provides GNU Bash version 5 via license GPLv3+
46+
47+
Whereas Mac provides Bash version 3 via license GPLv2
48+
49+
**Create and start containers:**
50+
51+
```zsh
52+
./mac_create_and_start_containers.sh
53+
```
54+
55+
**Start hackingBuddyGPT against a container:**
56+
57+
```zsh
58+
export GEMINI_API_KEY=
59+
```
60+
61+
```zsh
62+
./mac_start_hackingbuddygpt_against_a_container.sh
63+
```
64+
65+
**Troubleshooting:**
66+
67+
**Docker Desktop: Internal Server Error**
68+
69+
```zsh
70+
Server:
71+
ERROR: request returned Internal Server Error for API route and version http://%2FUsers%2Fusername%2F.docker%2Frun%2Fdocker.sock/v1.47/info, check if the server supports the requested API version
72+
errors pretty printing info
73+
```
74+
75+
You may need to uninstall Docker Desktop https://docs.docker.com/desktop/uninstall/ and reinstall it from https://docs.docker.com/desktop/setup/install/mac-install/ and try again.
76+
77+
Alternatively, restart Docker Desktop and try again.
78+
79+
**There are known issues with Docker Desktop on Mac, such as:**
80+
81+
* Bug: Docker CLI Hangs for all commands
82+
https://github.com/docker/for-mac/issues/6940
83+
84+
* Regression: Docker does not recover from resource saver mode
85+
https://github.com/docker/for-mac/issues/6933
86+
87+
**Google AI Studio: Gemini free tier has a limit of 15 requests per minute, and 1500 requests per day:**
88+
89+
https://ai.google.dev/pricing#1_5flash
90+
91+
> Gemini 1.5 Flash
92+
>
93+
> The Gemini API “free tier” is offered through the API service with lower rate limits for testing purposes. Google AI Studio usage is completely free in all available countries.
94+
>
95+
> Rate Limits
96+
>
97+
> 15 RPM (requests per minute)
98+
>
99+
> 1 million TPM (tokens per minute)
100+
>
101+
> 1,500 RPD (requests per day)
102+
>
103+
> Used to improve Google's products
104+
>
105+
> Yes
106+
107+
https://ai.google.dev/gemini-api/terms#data-use-unpaid
108+
109+
> How Google Uses Your Data
110+
>
111+
> When you use Unpaid Services, including, for example, Google AI Studio and the unpaid quota on Gemini API, Google uses the content you submit to the Services and any generated responses to provide, improve, and develop Google products and services and machine learning technologies, including Google's enterprise features, products, and services, consistent with our Privacy Policy https://policies.google.com/privacy
112+
>
113+
> To help with quality and improve our products, human reviewers may read, annotate, and process your API input and output. Google takes steps to protect your privacy as part of this process. This includes disconnecting this data from your Google Account, API key, and Cloud project before reviewers see or annotate it. **Do not submit sensitive, confidential, or personal information to the Unpaid Services.**
114+
115+
**README.md and Disclaimers:**
116+
117+
https://github.com/ipa-lab/hackingBuddyGPT/blob/main/README.md
118+
119+
**Please refer to [README.md](https://github.com/ipa-lab/hackingBuddyGPT/blob/main/README.md) for all disclaimers.**
120+
121+
Please note and accept all of them.

mac_create_and_start_containers.sh

Lines changed: 256 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,256 @@
1+
#!/opt/homebrew/bin/bash
2+
3+
# Purpose: Automates the setup of docker containers for local testing on Mac.
4+
# Usage: ./mac_create_and_start_containers.sh
5+
6+
# Enable strict error handling
7+
set -e
8+
set -u
9+
set -o pipefail
10+
set -x
11+
12+
# Step 1: Initialization
13+
14+
if [ ! -f hosts.ini ]; then
15+
echo "hosts.ini not found! Please ensure your Ansible inventory file exists."
16+
exit 1
17+
fi
18+
19+
if [ ! -f tasks.yaml ]; then
20+
echo "tasks.yaml not found! Please ensure your Ansible playbook file exists."
21+
exit 1
22+
fi
23+
24+
# Default value for base port
25+
# BASE_PORT=${BASE_PORT:-49152}
26+
27+
# Default values for network and base port, can be overridden by environment variables
28+
DOCKER_NETWORK_NAME=${DOCKER_NETWORK_NAME:-192_168_65_0_24}
29+
DOCKER_NETWORK_SUBNET="192.168.65.0/24"
30+
BASE_PORT=${BASE_PORT:-49152}
31+
32+
# Step 2: Define helper functions
33+
34+
# Function to find an available port
35+
find_available_port() {
36+
local base_port="$1"
37+
local port=$base_port
38+
local max_port=65535
39+
while lsof -i :$port; do
40+
port=$((port + 1))
41+
if [ "$port" -gt "$max_port" ]; then
42+
echo "No available ports in the range $base_port-$max_port." >&2
43+
exit 1
44+
fi
45+
done
46+
echo $port
47+
}
48+
49+
# Function to generate SSH key pair
50+
generate_ssh_key() {
51+
ssh-keygen -t rsa -b 4096 -f ./mac_ansible_id_rsa -N '' -q <<< y
52+
echo "New SSH key pair generated."
53+
chmod 600 ./mac_ansible_id_rsa
54+
}
55+
56+
# Function to create and start docker container with SSH enabled
57+
start_container() {
58+
local container_name="$1"
59+
local port="$2"
60+
local image_name="ansible-ready-ubuntu"
61+
62+
if docker --debug ps -aq -f name=${container_name}; then
63+
echo "Container ${container_name} already exists. Removing it..." >&2
64+
docker --debug stop ${container_name} || true
65+
docker --debug rm ${container_name} || true
66+
fi
67+
68+
echo "Starting docker container ${container_name} on port ${port}..." >&2
69+
70+
# Uncomment the following line to use a custom Docker network
71+
# docker --debug run --restart=unless-stopped -it -d --network ${DOCKER_NETWORK_NAME} -p "${port}:22" --name ${container_name} -h ${container_name} ${image_name}
72+
# The line is commented out because of the bugs in Docker Desktop on Mac causing hangs
73+
74+
# Alternatively, start Docker container with SSH enabled on localhost without using a custom Docker network
75+
docker --debug run --restart=unless-stopped -it -d -p "${port}:22" --name ${container_name} -h ${container_name} ${image_name}
76+
77+
# Retrieve the IP address assigned by Docker
78+
container_ip=$(docker --debug inspect -f '{{range.NetworkSettings.Networks}}{{.IPAddress}}{{end}}' "$container_name")
79+
80+
# Verify that container_ip is not empty
81+
if [ -z "$container_ip" ]; then
82+
echo "Error: Could not retrieve IP address for container $container_name." >&2
83+
exit 1
84+
fi
85+
86+
echo "Container ${container_name} started with IP ${container_ip} and port ${port}."
87+
88+
# Copy SSH public key to container
89+
docker --debug cp ./mac_ansible_id_rsa.pub ${container_name}:/home/ansible/.ssh/authorized_keys
90+
docker --debug exec ${container_name} chown ansible:ansible /home/ansible/.ssh/authorized_keys
91+
docker --debug exec ${container_name} chmod 600 /home/ansible/.ssh/authorized_keys
92+
}
93+
94+
# Function to check if SSH is ready on a container
95+
check_ssh_ready() {
96+
local port="$1"
97+
ssh -o BatchMode=yes -o ConnectTimeout=10 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i ./mac_ansible_id_rsa -p ${port} ansible@localhost exit 2>/dev/null
98+
return $?
99+
}
100+
101+
# Step 3: Verify docker Desktop
102+
103+
echo "Checking if docker Desktop is running..."
104+
if ! docker --debug info; then
105+
echo If the above says
106+
echo
107+
echo "Server:"
108+
echo "ERROR: request returned Internal Server Error for API route and version http://%2FUsers%2Fusername%2F.docker%2Frun%2Fdocker.sock/v1.47/info, check if the server supports the requested API version"
109+
echo "errors pretty printing info"
110+
echo
111+
echo You may need to uninstall Docker Desktop https://docs.docker.com/desktop/uninstall/ and reinstall it from https://docs.docker.com/desktop/setup/install/mac-install/ and try again.
112+
echo
113+
echo Alternatively, restart Docker Desktop and try again.
114+
echo
115+
echo There are known issues with Docker Desktop on Mac, such as:
116+
echo
117+
echo Bug: Docker CLI Hangs for all commands
118+
echo https://github.com/docker/for-mac/issues/6940
119+
echo
120+
echo Regression: Docker does not recover from resource saver mode
121+
echo https://github.com/docker/for-mac/issues/6933
122+
echo
123+
echo "Docker Desktop is not running. Please start Docker Desktop and try again."
124+
echo
125+
exit 1
126+
fi
127+
128+
# Step 4: Install prerequisites
129+
130+
echo "Installing required Python packages..."
131+
if ! command -v pip3 >/dev/null 2>&1; then
132+
echo "pip3 not found. Please install Python3 and pip3 first."
133+
exit 1
134+
fi
135+
136+
echo "Installing Ansible and passlib using pip..."
137+
pip3 install ansible passlib
138+
139+
# Step 5: Build docker image
140+
141+
echo "Building docker image with SSH enabled..."
142+
if ! docker --debug build -t ansible-ready-ubuntu -f codespaces_create_and_start_containers.Dockerfile .; then
143+
echo "Failed to build docker image." >&2
144+
exit 1
145+
fi
146+
147+
# Step 6: Create a custom docker network if it does not exist
148+
149+
# Commenting out this step because Docker bug and its regression that are clausing CLI to hang
150+
151+
# There is a Docker bug that prevents creating custom networks on MacOS because it hangs
152+
153+
# Bug: Docker CLI Hangs for all commands
154+
# https://github.com/docker/for-mac/issues/6940
155+
156+
# Regression: Docker does not recover from resource saver mode
157+
# https://github.com/docker/for-mac/issues/6933
158+
159+
# echo "Checking if the custom docker network '${DOCKER_NETWORK_NAME}' with subnet {DOCKER_NETWORK_SUBNET} exists"
160+
161+
# if ! docker --debug network inspect ${DOCKER_NETWORK_NAME} >/dev/null 2>&1; then
162+
# docker --debug network create --subnet="${DOCKER_NETWORK_SUBNET}" "${DOCKER_NETWORK_NAME}" || echo "Network creation failed, but continuing..."
163+
# fi
164+
165+
# Unfortunately, the above just hangs like this:
166+
167+
# + echo 'Checking if the custom docker network '\''192_168_65_0_24'\'' with subnet {DOCKER_NETWORK_SUBNET} exists'
168+
# Checking if the custom docker network '192_168_65_0_24' with subnet {DOCKER_NETWORK_SUBNET} exists
169+
# + docker --debug network inspect 192_168_65_0_24
170+
# + docker --debug network create --subnet=192.168.65.0/24 192_168_65_0_24
171+
172+
# (It hangs here)
173+
174+
# For now, the workaround is to use localhost as the IP address on a dynamic or private TCP port, such as 41952
175+
176+
# Step 7: Generate SSH key
177+
generate_ssh_key
178+
179+
# Step 8: Create mac inventory file
180+
181+
echo "Creating mac Ansible inventory..."
182+
cat > mac_ansible_hosts.ini << EOF
183+
[local]
184+
localhost ansible_port=PLACEHOLDER ansible_user=ansible ansible_ssh_private_key_file=./mac_ansible_id_rsa ansible_ssh_common_args='-o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null'
185+
186+
[all:vars]
187+
ansible_python_interpreter=/usr/bin/python3
188+
EOF
189+
190+
# Step 9: Start container and update inventory
191+
192+
available_port=$(find_available_port "$BASE_PORT")
193+
start_container "ansible-ready-ubuntu" "$available_port"
194+
195+
# Update the port in the inventory file
196+
sed -i '' "s/PLACEHOLDER/$available_port/" mac_ansible_hosts.ini
197+
198+
# Step 10: Wait for SSH service
199+
200+
echo "Waiting for SSH service to start..."
201+
max_attempts=30
202+
attempt=1
203+
while [ $attempt -le $max_attempts ]; do
204+
if check_ssh_ready "$available_port"; then
205+
echo "SSH is ready!"
206+
break
207+
fi
208+
echo "Waiting for SSH to be ready (attempt $attempt/$max_attempts)..."
209+
sleep 2
210+
attempt=$((attempt + 1))
211+
done
212+
213+
if [ $attempt -gt $max_attempts ]; then
214+
echo "SSH service failed to start. Exiting."
215+
exit 1
216+
fi
217+
218+
# Step 11: Create ansible.cfg
219+
220+
cat > mac_ansible.cfg << EOF
221+
[defaults]
222+
interpreter_python = auto_silent
223+
host_key_checking = False
224+
remote_user = ansible
225+
226+
[privilege_escalation]
227+
become = True
228+
become_method = sudo
229+
become_user = root
230+
become_ask_pass = False
231+
EOF
232+
233+
# Step 12: Set ANSIBLE_CONFIG and run playbook
234+
235+
export ANSIBLE_CONFIG=$(pwd)/mac_ansible.cfg
236+
237+
echo "Running Ansible playbook..."
238+
ansible-playbook -i mac_ansible_hosts.ini tasks.yaml
239+
240+
echo "Setup complete. Container ansible-ready-ubuntu is ready for testing."
241+
242+
# Step 13: Run gemini-openai-proxy container
243+
244+
if docker --debug ps -aq -f name=gemini-openai-proxy; then
245+
echo "Container gemini-openai-proxy already exists. Removing it..." >&2
246+
docker --debug stop gemini-openai-proxy || true
247+
docker --debug rm gemini-openai-proxy || true
248+
fi
249+
250+
docker --debug run --restart=unless-stopped -it -d -p 8080:8080 --name gemini-openai-proxy zhu327/gemini-openai-proxy:latest
251+
252+
# Step 14: Ready to run hackingBuddyGPT
253+
254+
echo "You can now run ./mac_start_hackingbuddygpt_against_a_container.sh"
255+
256+
exit 0

0 commit comments

Comments
 (0)