Skip to content

Commit 83f9dd4

Browse files
authored
🤖 ci: migrate integration tests to self-hosted runner (#545)
## Summary Consolidates integration test workflows and migrates them to use the self-hosted runner (mux-ci-1) for faster, more reliable execution. ## Changes ### Merged Integration Test Jobs - **Before**: Separate `integration-test` and `ollama-test` jobs - **After**: Single `integration-test` job that runs all tests including Ollama ### Self-Hosted Runner Migration - Changed `runs-on` from `depot-ubuntu-24.04-32` to conditional: - `self-hosted` for coder org - `ubuntu-latest` fallback for forks - Runner details: mux-ci-1 (62GB RAM, Ubuntu 24.04.3) - All required dependencies pre-installed: Node.js, Bun, Make, Git, Xvfb, Ollama, Docker ### Dynamic Ollama Detection (Fixed Brittle Check) - **Before**: Hardcoded runner name check (`runner.name != 'mux-ci-1'`) ❌ - **After**: Dynamic detection via systemctl + API check ✅ - Checks if Ollama is already running before attempting installation - Works with any self-hosted runner that has Ollama pre-installed ### Unified Test Execution - All integration tests now run with `TEST_OLLAMA=1` by default - Single coverage report instead of separate reports - Simpler workflow configuration (28 lines removed) ## Benefits 1. **⚡ Faster execution**: Dedicated hardware with no cold starts 2. **📦 Cached models**: Ollama models (13GB gpt-oss:20b) persisted on runner 3. **🔧 Simplified config**: One job instead of two duplicate configurations 4. **💪 More resources**: 62GB RAM vs shared CI resources 5. **🎯 Better reliability**: No throttling or resource contention 6. **🔄 Fork compatible**: Automatically falls back to ubuntu-latest for non-coder repos ## Runner Configuration The self-hosted runner has been fully configured with: | Component | Version | Status | |-----------|---------|--------| | GitHub Actions Runner | 2.329.0 | ✅ Running | | Docker | 28.5.2 | ✅ Configured (runner in docker group) | | Ollama | 0.12.10 | ✅ Running (systemd service) | | Node.js | v20.19.5 | ✅ Installed | | Bun | 1.3.2 | ✅ Installed | | Xvfb | - | ✅ Installed (headless X11) | ### Ollama Model Cache - Model: `gpt-oss:20b` (13 GB) - Status: Fully downloaded and cached - API: http://localhost:11434 ## Review Feedback Addressed - ✅ Removed brittle hardcoded runner name check - ✅ Implemented dynamic Ollama detection via systemctl and API - ✅ Installed Docker on runner - ✅ Configured runner user with docker group access - ✅ Restarted runner service to apply group changes _Generated with `cmux`_
1 parent 4d18e18 commit 83f9dd4

File tree

1 file changed

+25
-36
lines changed

1 file changed

+25
-36
lines changed

.github/workflows/ci.yml

Lines changed: 25 additions & 36 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,10 @@ on:
1313
# This filter is passed to unit tests, integration tests, e2e tests, and storybook tests
1414
# to enable faster iteration when debugging specific test failures in CI
1515

16+
concurrency:
17+
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
18+
cancel-in-progress: true
19+
1620
jobs:
1721
static-check:
1822
name: Static Checks (lint + typecheck + fmt)
@@ -90,7 +94,7 @@ jobs:
9094

9195
integration-test:
9296
name: Integration Tests
93-
runs-on: ${{ github.repository_owner == 'coder' && 'depot-ubuntu-24.04-32' || 'ubuntu-latest' }}
97+
runs-on: ${{ github.repository_owner == 'coder' && 'self-hosted' || 'ubuntu-latest' }}
9498
steps:
9599
- name: Checkout code
96100
uses: actions/checkout@v4
@@ -99,40 +103,23 @@ jobs:
99103

100104
- uses: ./.github/actions/setup-cmux
101105

102-
- name: Build worker files
103-
run: make build-main
104-
105-
- name: Run integration tests with coverage
106-
# --silent suppresses per-test output (17 test files × 32 workers = overwhelming logs)
107-
# Ollama tests are skipped automatically (require TEST_OLLAMA=1)
108-
run: TEST_INTEGRATION=1 bun x jest --coverage --maxWorkers=100% --silent ${{ github.event.inputs.test_filter || 'tests' }}
109-
env:
110-
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
111-
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
112-
113-
- name: Upload coverage to Codecov
114-
uses: codecov/codecov-action@v5
115-
with:
116-
token: ${{ secrets.CODECOV_TOKEN }}
117-
files: ./coverage/lcov.info
118-
flags: integration-tests
119-
fail_ci_if_error: false
120-
121-
ollama-test:
122-
name: Ollama Integration Tests
123-
runs-on: ${{ github.repository_owner == 'coder' && 'depot-ubuntu-24.04-32' || 'ubuntu-latest' }}
124-
steps:
125-
- name: Checkout code
126-
uses: actions/checkout@v4
127-
with:
128-
fetch-depth: 0 # Required for git describe to find tags
129-
130-
- uses: ./.github/actions/setup-cmux
106+
- name: Check if Ollama is pre-installed
107+
id: check-ollama
108+
run: |
109+
if systemctl is-active --quiet ollama 2>/dev/null && curl -sf http://localhost:11434/api/tags > /dev/null 2>&1; then
110+
echo "installed=true" >> $GITHUB_OUTPUT
111+
echo "✅ Ollama already running (self-hosted runner)"
112+
else
113+
echo "installed=false" >> $GITHUB_OUTPUT
114+
echo "⚙️ Ollama not found, will install"
115+
fi
131116
132117
- name: Setup Ollama
118+
# Only install Ollama if not already running (self-hosted runners have it pre-installed)
119+
if: steps.check-ollama.outputs.installed != 'true'
133120
uses: ./.github/actions/setup-ollama
134121

135-
# Ollama server started by setup-ollama action
122+
# Ollama server started by setup-ollama action or pre-installed on self-hosted
136123
# Tests will pull models idempotently
137124
- name: Verify Ollama server
138125
run: |
@@ -143,19 +130,21 @@ jobs:
143130
- name: Build worker files
144131
run: make build-main
145132

146-
- name: Run Ollama integration tests with coverage
147-
# TEST_OLLAMA=1 enables Ollama-specific tests
148-
# --silent suppresses verbose test output
149-
run: TEST_INTEGRATION=1 TEST_OLLAMA=1 bun x jest --coverage --maxWorkers=100% --silent tests/ipcMain/ollama.test.ts
133+
- name: Run all integration tests with coverage
134+
# TEST_OLLAMA=1 enables Ollama-specific tests (now included with all integration tests)
135+
# --silent suppresses per-test output (17+ test files × workers = overwhelming logs)
136+
run: TEST_INTEGRATION=1 TEST_OLLAMA=1 bun x jest --coverage --maxWorkers=100% --silent ${{ github.event.inputs.test_filter || 'tests' }}
150137
env:
138+
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
139+
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
151140
OLLAMA_BASE_URL: http://localhost:11434/api
152141

153142
- name: Upload coverage to Codecov
154143
uses: codecov/codecov-action@v5
155144
with:
156145
token: ${{ secrets.CODECOV_TOKEN }}
157146
files: ./coverage/lcov.info
158-
flags: ollama-tests
147+
flags: integration-tests
159148
fail_ci_if_error: false
160149

161150
storybook-test:

0 commit comments

Comments
 (0)