Skip to content

CLOUDP-295785 - simplify pipeline.sh script and use it for all images #543

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Oct 23, 2025
Merged

CLOUDP-295785 - simplify pipeline.sh script and use it for all images #543

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
a15e493
CLOUDP-295785 - simplify pipeline.sh script and use it for all images
MaciejKaras Oct 21, 2025
3781e1e
Merge branch 'master' into maciejk/ar-pipeline-sh-refac
MaciejKaras Oct 23, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
49 changes: 4 additions & 45 deletions .evergreen-functions.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -507,7 +507,7 @@ functions:
params:
working_dir: src/github.com/mongodb/mongodb-kubernetes
binary: scripts/release/publish_helm_chart.sh

build_multi_cluster_binary:
- command: subprocess.exec
params:
Expand Down Expand Up @@ -542,50 +542,9 @@ functions:
working_dir: src/github.com/mongodb/mongodb-kubernetes
binary: scripts/release/pipeline.sh
env:
IMAGE_NAME: ${IMAGE_NAME}
FLAGS: ${FLAGS}

pipeline_agent:
- command: subprocess.exec
retry_on_failure: true
type: setup
params:
shell: bash
<<: *e2e_include_expansions_in_env
working_dir: src/github.com/mongodb/mongodb-kubernetes
binary: scripts/release/pipeline_agent.sh
env:
FLAGS: ${FLAGS}

pipeline_ops_manager:
- command: subprocess.exec
retry_on_failure: true
type: setup
params:
shell: bash
<<: *e2e_include_expansions_in_env
working_dir: src/github.com/mongodb/mongodb-kubernetes
binary: scripts/release/pipeline_ops_manager.sh

pipeline_readiness_probe:
- command: subprocess.exec
retry_on_failure: true
type: setup
params:
shell: bash
<<: *e2e_include_expansions_in_env
working_dir: src/github.com/mongodb/mongodb-kubernetes
binary: scripts/release/pipeline_readiness_probe.sh

pipeline_version_upgrade_hook:
- command: subprocess.exec
retry_on_failure: true
type: setup
params:
shell: bash
<<: *e2e_include_expansions_in_env
working_dir: src/github.com/mongodb/mongodb-kubernetes
binary: scripts/release/pipeline_version_upgrade_hook.sh
IMAGE_NAME: ${image_name}
BUILD_SCENARIO_OVERRIDE: ${build_scenario}
Copy link
Contributor

@viveksinghggits viveksinghggits Oct 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

q: did we add BUILD_SCENARIO_OVERRIDE so that we can build the images for different build scenarios manually?

Copy link
Collaborator Author

@MaciejKaras MaciejKaras Oct 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no, we can still use BUILD_SCENARIO for that and pass it to evergreen cmd. The BUILD_SCENARIO_OVERRIDE is only used if we specify build_scenario in evergreen function call, so we can override BUILD_SCENARIO. Example:

  # pct only triggers this variant once a new agent image is out
  - name: release_agent
    # this enables us to run this variant either manually (patch) which pct does or during an OM bump (github_pr)
    allowed_requesters: [ "patch", "github_pr" ]
    commands:
      - func: clone
      - func: setup_building_host
      - func: quay_login
      - func: setup_docker_sbom
      - func: pipeline
        vars:
          image_name: agent
          build_scenario: release

FLAGS: ${flags}

teardown_cloud_qa_all:
- command: shell.exec
Expand Down
22 changes: 13 additions & 9 deletions .evergreen-release.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ tasks:
- func: build_multi_cluster_binary
- func: pipeline
vars:
IMAGE_NAME: meko-tests
image_name: meko-tests

- name: release_operator
tags: [ "image_release" ]
Expand All @@ -23,7 +23,7 @@ tasks:
- func: quay_login
- func: pipeline
vars:
IMAGE_NAME: operator
image_name: operator

# Releases init images to Quay
- name: release_init_appdb
Expand All @@ -35,7 +35,7 @@ tasks:
- func: quay_login
- func: pipeline
vars:
IMAGE_NAME: init-appdb
image_name: init-appdb

- name: release_init_database
tags: [ "image_release" ]
Expand All @@ -46,7 +46,7 @@ tasks:
- func: quay_login
- func: pipeline
vars:
IMAGE_NAME: init-database
image_name: init-database

- name: release_init_ops_manager
tags: [ "image_release" ]
Expand All @@ -57,7 +57,7 @@ tasks:
- func: quay_login
- func: pipeline
vars:
IMAGE_NAME: init-ops-manager
image_name: init-ops-manager

- name: release_database
tags: [ "image_release" ]
Expand All @@ -68,7 +68,7 @@ tasks:
- func: quay_login
- func: pipeline
vars:
IMAGE_NAME: database
image_name: database

- name: release_readiness_probe
tags: [ "image_release" ]
Expand All @@ -77,7 +77,9 @@ tasks:
- func: clone
- func: setup_building_host
- func: quay_login
- func: pipeline_readiness_probe
- func: pipeline
vars:
image_name: readiness-probe

- name: release_version_upgrade_hook
tags: [ "image_release" ]
Expand All @@ -86,7 +88,9 @@ tasks:
- func: clone
- func: setup_building_host
- func: quay_login
- func: pipeline_version_upgrade_hook
- func: pipeline
vars:
image_name: upgrade-hook

- name: prepare_and_upload_openshift_bundles
tags: [ "openshift_bundles" ]
Expand Down Expand Up @@ -123,7 +127,7 @@ tasks:
- func: install_macos_notarization_service
- func: python_venv
- func: release_kubectl_mongodb_plugin

- name: create_chart_release_pr
tags: [ "helm_chart_release_pr" ]
commands:
Expand Down
63 changes: 37 additions & 26 deletions .evergreen.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -285,9 +285,10 @@ tasks:
- func: setup_building_host
- func: quay_login
- func: setup_docker_sbom
- func: pipeline_agent
- func: pipeline
vars:
BUILD_SCENARIO: release
image_name: agent
build_scenario: release

- name: migrate_all_agents
# this enables us to run this variant manually to build all the agents for the new agent registry
Expand All @@ -296,10 +297,11 @@ tasks:
- func: clone
- func: setup_building_host
- func: quay_login
- func: pipeline_agent
- func: pipeline
vars:
BUILD_SCENARIO: release
FLAGS: "--all-agents -r quay.io/mongodb/mongodb-agent --skip-if-exists=false"
image_name: agent
build_scenario: release
flags: "--parallel --all-agents -r quay.io/mongodb/mongodb-agent --skip-if-exists=false"

- name: run_precommit_and_push
tags: [ "patch-run" ]
Expand Down Expand Up @@ -328,9 +330,10 @@ tasks:
- func: clone
- func: setup_building_host
- func: quay_login
- func: pipeline_agent
- func: pipeline
vars:
FLAGS: "--all-agents --skip-if-exists=false"
image_name: agent
flags: "--parallel --all-agents --skip-if-exists=false"

- name: rebuild_currently_used_agents
# this enables us to run this manually (patch) and rebuild current agent versions to verify
Expand All @@ -340,9 +343,10 @@ tasks:
- func: clone
- func: setup_building_host
- func: quay_login
- func: pipeline_agent
- func: pipeline
vars:
FLAGS: "--current-agents --skip-if-exists=false"
image_name: agent
flags: "--parallel --current-agents --skip-if-exists=false"

- name: build_test_image
commands:
Expand All @@ -352,7 +356,7 @@ tasks:
- func: build_multi_cluster_binary
- func: pipeline
vars:
IMAGE_NAME: meko-tests
image_name: meko-tests

- name: build_test_image_arm
commands:
Expand All @@ -362,8 +366,7 @@ tasks:
- func: build_multi_cluster_binary
- func: pipeline
vars:
IMAGE_NAME: meko-tests-arm64

image_name: meko-tests-arm64

- name: build_test_image_ibm
commands:
Expand All @@ -387,76 +390,81 @@ tasks:
- func: setup_building_host
- func: pipeline
vars:
IMAGE_NAME: mco-tests
image_name: mco-tests

- name: build_operator_ubi
commands:
- func: clone
- func: setup_building_host
- func: pipeline
vars:
IMAGE_NAME: operator
image_name: operator

- name: build_operator_race_ubi
commands:
- func: clone
- func: setup_building_host
- func: pipeline
vars:
IMAGE_NAME: operator-race
image_name: operator-race

- name: build_init_om_images_ubi
commands:
- func: clone
- func: setup_building_host
- func: pipeline
vars:
IMAGE_NAME: init-ops-manager
image_name: init-ops-manager

- name: build_init_appdb_images_ubi
commands:
- func: clone
- func: setup_building_host
- func: pipeline
vars:
IMAGE_NAME: init-appdb
image_name: init-appdb

- name: build_agent_images_ubi
commands:
- func: clone
- func: setup_building_host
- func: quay_login
- func: pipeline_agent
- func: pipeline
vars:
FLAGS: "--all-agents"
image_name: agent
flags: "--parallel --all-agents"

- name: build_init_database_image_ubi
commands:
- func: clone
- func: setup_building_host
- func: pipeline
vars:
IMAGE_NAME: init-database
image_name: init-database

- name: build_database_image_ubi
commands:
- func: clone
- func: setup_building_host
- func: pipeline
vars:
IMAGE_NAME: database
image_name: database

- name: build_readiness_probe_image
commands:
- func: clone
- func: setup_building_host
- func: pipeline_readiness_probe
- func: pipeline
vars:
image_name: readiness-probe

- name: build_version_upgrade_hook_image
commands:
- func: clone
- func: setup_building_host
- func: pipeline_version_upgrade_hook
- func: pipeline
vars:
image_name: upgrade-hook

- name: publish_helm_chart
commands:
Expand Down Expand Up @@ -513,7 +521,9 @@ tasks:
- func: clone
- func: switch_context
- func: setup_building_host
- func: pipeline_ops_manager
- func: pipeline
vars:
image_name: ops-manager

- name: publish_ops_manager
commands:
Expand All @@ -522,9 +532,10 @@ tasks:
- func: setup_building_host
- func: quay_login
- func: setup_docker_sbom
- func: pipeline_ops_manager
- func: pipeline
vars:
BUILD_SCENARIO: release
image_name: ops-manager
build_scenario: release

- name: prepare_and_upload_openshift_bundles_for_e2e
commands:
Expand Down
36 changes: 35 additions & 1 deletion scripts/release/pipeline.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,4 +4,38 @@ set -Eeou pipefail

source scripts/dev/set_env_context.sh

scripts/dev/run_python.sh scripts/release/pipeline.py "${IMAGE_NAME}" --build-scenario "${BUILD_SCENARIO}" --version "${OPERATOR_VERSION}" ${FLAGS:+${FLAGS}}
args=("${IMAGE_NAME}")
args+=(--build-scenario "${BUILD_SCENARIO_OVERRIDE:-${BUILD_SCENARIO}}")

case ${IMAGE_NAME} in
"agent")
IMAGE_VERSION=""
;;

"ops-manager")
IMAGE_VERSION="${OM_VERSION}"
;;

"readiness-probe")
IMAGE_VERSION="${READINESS_PROBE_VERSION}"
;;

"upgrade-hook")
IMAGE_VERSION="${VERSION_UPGRADE_HOOK_VERSION}"
;;

*)
IMAGE_VERSION="${OPERATOR_VERSION}"
;;
esac

if [[ "${IMAGE_VERSION:-}" != "" ]]; then
args+=(--version "${IMAGE_VERSION}")
fi

if [[ "${FLAGS:-}" != "" ]]; then
IFS=" " read -ra flags <<< "${FLAGS}"
args+=("${flags[@]}")
fi

scripts/dev/run_python.sh scripts/release/pipeline.py "${args[@]}"
16 changes: 0 additions & 16 deletions scripts/release/pipeline_agent.sh
View file
Open in desktop

This file was deleted.

Loading