Skip to content
This repository was archived by the owner on Feb 24, 2025. It is now read-only.

Commit 1a88f9e

Browse files
ayoyayoy
authored
Generate appcast2.xml and upload files to S3 in a CI workflow (#2184)
Task/Issue URL: https://app.asana.com/0/0/1206459737646671/f Description: This change adds new workflow called sparkle_internal.yml that should be triggered manually after smoke testing. It takes Asana release task URL and the tag to be published, and generates new appcast file and uploads missing files to S3. The workflow fetches DMG for the release from S3 (based on the assumption that Code Freeze and/or Bump Internal Release uploaded the DMG there – this was introduced in #2165). appcastManager and upload_to_s3 scripts were updated for running in CI (support non-interactive shell). Additionally, appcastManager generates a list of added/updated files. Completion is reported to the Automation subtask and also to the release task. Upon successful completion, the release DRI is assigned a task to verify that the update worked, with instructions on how to revert the appcast in case it's broken. If the workflow fails, the user is assigned a task with instructions for generating appcast and uploading to S3 locally.
1 parent 7c7738c commit 1a88f9e

22 files changed

+599
-86
lines changed

‎.github/actions/asana-add-comment/action.yml

+78
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,78 @@
1+
name: Add a Comment to Asana Task
2+
description: Adds a comment to the Asana task.
3+
inputs:
4+
access-token:
5+
description: "Asana access token"
6+
required: true
7+
type: string
8+
task-url:
9+
description: "Task URL"
10+
required: false
11+
type: string
12+
task-id:
13+
description: "Task ID"
14+
required: false
15+
type: string
16+
comment:
17+
description: "Comment to add to the Asana task"
18+
required: false
19+
type: string
20+
template-name:
21+
description: |
22+
Name of a template file (without extension) for the comment, relative to 'templates' subdirectory of the action.
23+
The file is processed by envsubst before being sent to Asana.
24+
required: false
25+
type: string
26+
runs:
27+
using: "composite"
28+
steps:
29+
- id: extract-task-id
30+
if: ${{ inputs.task-url }}
31+
uses: ./.github/actions/asana-extract-task-id
32+
with:
33+
task-url: ${{ inputs.task-url }}
34+
access-token: ${{ inputs.access-token }}
35+
36+
- id: process-template-payload
37+
if: ${{ inputs.template-name }}
38+
shell: bash
39+
env:
40+
TEMPLATE_PATH: ${{ github.action_path }}/templates/${{ inputs.template-name }}.yml
41+
run: |
42+
if [ ! -f $TEMPLATE_PATH ]; then
43+
echo "::error::Template file not found at $TEMPLATE_PATH"
44+
exit 1
45+
fi
46+
47+
# Process the template file with envsubst, turn into JSON, remove leading spaces and newlines on non-empty lines, and compact the JSON
48+
payload="$(envsubst < $TEMPLATE_PATH | yq -o=j | sed -E 's/\\n( *)([^\\n])/\2/g' | jq -c)"
49+
echo "payload-base64=$(base64 <<< $payload)" >> $GITHUB_OUTPUT
50+
51+
- id: process-comment-payload
52+
if: ${{ inputs.comment }}
53+
shell: bash
54+
env:
55+
COMMENT: ${{ inputs.comment }}
56+
WORKFLOW_URL: https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }}
57+
run: |
58+
payload="{ \"data\": { \"text\": \"${COMMENT}\n\nWorkflow URL: ${WORKFLOW_URL}\" } }"
59+
echo "payload-base64=$(base64 <<< $payload)" >> $GITHUB_OUTPUT
60+
61+
- id: add-comment
62+
shell: bash
63+
env:
64+
ASANA_ACCESS_TOKEN: ${{ inputs.access-token }}
65+
TASK_ID: ${{ inputs.task-id || steps.extract-task-id.outputs.task-id }}
66+
PAYLOAD_BASE64: ${{ steps.process-template-payload.outputs.payload-base64 || steps.process-comment-payload.outputs.payload-base64 }}
67+
run: |
68+
return_code=$(curl -fLSs "https://app.asana.com/api/1.0/tasks/${TASK_ID}/stories" \
69+
-H "Authorization: Bearer ${ASANA_ACCESS_TOKEN}" \
70+
-H 'content-type: application/json' \
71+
--write-out '%{http_code}' \
72+
--output /dev/null \
73+
-d "$(base64 -d <<< $PAYLOAD_BASE64)")
74+
75+
if [ $return_code -ne 201 ]; then
76+
echo "::error::Failed to add a comment to the Asana task"
77+
exit 1
78+
fi

‎.github/actions/asana-add-comment/templates/appcast-failed.yml

+9
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
data:
2+
html_text: |
3+
<body>
4+
<h2>[ACTION NEEDED] Publishing ${TAG} internal release to Sparkle failed</h2>
5+
<a data-asana-gid='${ASSIGNEE_ID}'/>, please proceed with generating appcast2.xml and uploading files to S3 from your local machine, <a data-asana-gid='${TASK_ID}' data-asana-dynamic='false'>according to instructions</a>.
6+
7+
8+
🔗 Workflow URL: <a href='${WORKFLOW_URL}'>${WORKFLOW_URL}</a>.
9+
</body>

‎.github/actions/asana-add-comment/templates/debug-symbols-uploaded.yml

+8
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
data:
2+
html_text: |
3+
<body>
4+
🐛 Debug symbols archive for ${TAG} build is uploaded to <code>${DSYM_S3_PATH}</code>.
5+
6+
7+
🔗 Workflow URL: <a href='${WORKFLOW_URL}'>${WORKFLOW_URL}</a>.
8+
</body>

‎.github/actions/asana-add-comment/templates/dmg-uploaded.yml

+8
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
data:
2+
html_text: |
3+
<body>
4+
📥 DMG for ${TAG} is available from <a href='${DMG_URL}'>${DMG_URL}</a>.
5+
6+
7+
🔗 Workflow URL: <a href='${WORKFLOW_URL}'>${WORKFLOW_URL}</a>.
8+
</body>

‎.github/actions/asana-add-comment/templates/internal-release-complete.yml

+8
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
data:
2+
html_text: |
3+
<body>
4+
Build ${TAG} is now available for internal testing through Sparkle and TestFlight.
5+
6+
7+
<a href='${DMG_URL}'>📥 DMG download link</a>
8+
</body>

‎.github/actions/asana-add-comment/templates/internal-release-ready-merge-failed.yml

+17
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
data:
2+
# yq -o=j | sed -E 's/\\n( *)([^\\n])/\2/g'
3+
html_text: |
4+
<body>
5+
<h2>[ACTION NEEDED] Internal release build ${TAG} ready</h2>
6+
<ul>
7+
<li>📥 DMG is available from <a href='${DMG_URL}'>${DMG_URL}</a>.</li>
8+
<li>🏷️ Repository is tagged with <code>${TAG}</code> tag.</li>
9+
<li>🚢 GitHub <a href='${RELEASE_URL}'>${TAG} pre-release</a> is created.</li>
10+
<li><b>❗️ Merging <code>${BRANCH}</code> to <code>${BASE_BRANCH}</code> failed.</b>
11+
<ul>
12+
<li><a data-asana-gid='${ASSIGNEE_ID}'/>, please proceed with manual merging <a data-asana-gid='${TASK_ID}' data-asana-dynamic='false'>according to instructions</a>.</li>
13+
</ul></li>
14+
</ul>
15+
16+
🔗 Workflow URL: <a href='${WORKFLOW_URL}'>${WORKFLOW_URL}</a>.
17+
</body>

‎.github/actions/asana-add-comment/templates/internal-release-ready-tag-failed.yml

+16
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
data:
2+
html_text: |
3+
<body>
4+
<h2>[ACTION NEEDED] Internal release build ${TAG} ready</h2>
5+
<ul>
6+
<li>📥 DMG is available from <a href='${DMG_URL}'>${DMG_URL}</a>.</li>
7+
<li><b>❗️ Tagging repository failed.</b></li>
8+
<li><b>⚠️ GitHub release creation was skipped.</b></li>
9+
<li><b>⚠️ Merging <code>${BRANCH}</code> to <code>${BASE_BRANCH}</code> was skipped.</b></li>
10+
</ul>
11+
12+
<a data-asana-gid='${ASSIGNEE_ID}'/>, please proceed with manual tagging and merging <a data-asana-gid='${TASK_ID}' data-asana-dynamic='false'>according to instructions</a>.
13+
14+
15+
🔗 Workflow URL: <a href='${WORKFLOW_URL}'>${WORKFLOW_URL}</a>.
16+
</body>

‎.github/actions/asana-add-comment/templates/internal-release-ready.yml

+14
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
data:
2+
# yq -o=j | sed -E 's/\\n( *)([^\\n])/\2/g'
3+
html_text: |
4+
<body>
5+
<h2>Internal release build ${TAG} ready ✅</h2>
6+
<ul>
7+
<li>📥 DMG is available from <a href='${DMG_URL}'>${DMG_URL}</a>.</li>
8+
<li>🏷️ Repository is tagged with <code>${TAG}</code> tag.</li>
9+
<li>🚢 GitHub <a href='${RELEASE_URL}'>${TAG} pre-release</a> is created.</li>
10+
<li>🔱 <code>${BRANCH}</code> branch has been successfully merged to <code>${BASE_BRANCH}</code>.</li>
11+
</ul>
12+
13+
🔗 Workflow URL: <a href='${WORKFLOW_URL}'>${WORKFLOW_URL}</a>.
14+
</body>

‎.github/actions/asana-add-comment/templates/validate-check-for-updates-internal.yml

+11
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
data:
2+
html_text: |
3+
<body>
4+
<h2>Build ${TAG} is available for internal testing through Sparkle 🚀</h2>
5+
<ul>
6+
<li>🌟 New appcast file has been generated and uploaded to S3, together with binary delta files.</li>
7+
<li>👀 <a data-asana-gid='${ASSIGNEE_ID}'/>, please proceed by following instructions in <a data-asana-gid='${TASK_ID}'/> which concludes the internal release process.</li>
8+
</ul>
9+
10+
🔗 Workflow URL: <a href='${WORKFLOW_URL}'>${WORKFLOW_URL}</a>.
11+
</body>

‎.github/actions/asana-create-action-item/templates/appcast-failed.yml

+37
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
data:
2+
name: Generate appcast2.xml for ${TAG} internal release and upload assets to S3
3+
assignee: "${ASSIGNEE_ID}"
4+
html_notes: |
5+
<body>
6+
Publishing internal release ${TAG} failed in CI. Please follow the steps to generate the appcast file and upload files to S3 from your local machine.
7+
8+
<ol>
9+
<li>Download <a href='${DMG_URL}'>the DMG for ${TAG} release</a>.</li>
10+
<li>Create a new file called <code>release-notes.txt</code> on your disk.
11+
<ul>
12+
<li>Add each release note as a separate line and don't add bullet points (•) – the script will add them automatically.</li>
13+
</ul></li>
14+
<li>Run <code>appcastManager</code>:
15+
<ul>
16+
<li><code>./scripts/appcast_manager/appcastManager.swift --release-to-internal-channel --dmg ~/Downloads/${DMG_NAME} --release-notes release-notes.txt</code></li>
17+
</ul></li>
18+
<li>Verify that the new build is in the appcast file with the following internal channel tag:
19+
<ul>
20+
<li><code>&lt;sparkle:channel&gt;internal-channel&lt;/sparkle:channel&gt;</code></li>
21+
</ul></li>
22+
<li>Run <code>upload_to_s3.sh</code> script:
23+
<ul>
24+
<li><code>./scripts/upload_to_s3/upload_to_s3.sh --run</code></li>
25+
</ul></li>
26+
</ol>
27+
When done, please verify that "Check for Updates" works correctly:
28+
<ol>
29+
<li>Launch a debug version of the app with an old version number.</li>
30+
<li>Identify as an internal user in the app.</li>
31+
<li>Go to Main Menu → DuckDuckGo → Check for Updates...</li>
32+
<li>Verify that you're being offered to update to ${TAG}.</li>
33+
<li>Verify that the update works.</li>
34+
</ol>
35+
36+
🔗 Workflow URL: <a href='${WORKFLOW_URL}'>${WORKFLOW_URL}</a>.
37+
</body>

‎.github/actions/asana-create-action-item/templates/validate-check-for-updates-internal.yml

+41
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
data:
2+
name: Validate that 'Check For Updates' upgrades to ${TAG} for internal users
3+
assignee: "${ASSIGNEE_ID}"
4+
html_notes: |
5+
<body>
6+
<h1>Build ${TAG} has been released internally via Sparkle 🎉</h1>
7+
Please verify that "Check for Updates" works correctly:
8+
<ol>
9+
<li>Launch a debug version of the app with an old version number.</li>
10+
<li>Identify as an internal user in the app.</li>
11+
<li>Go to Main Menu → DuckDuckGo → Check for Updates...</li>
12+
<li>Verify that you're being offered to update to ${TAG}.</li>
13+
<li>Verify that the update works.</li>
14+
</ol>
15+
<h1>🚨In case "Check for Updates" is broken</h1>
16+
You can restore previous version of the appcast2.xml:
17+
<ol>
18+
<li>Download the ${OLD_APPCAST_NAME} file attached to this task.</li>
19+
<li>Log in to AWS session:
20+
<ul>
21+
<li><code>aws --profile ddg-macos sso login</code></li>
22+
</ul></li>
23+
<li>Overwrite appcast2.xml with the old version:
24+
<ul>
25+
<li><code>aws --profile ddg-macos s3 cp ${OLD_APPCAST_NAME} s3://${RELEASE_BUCKET_NAME}/${RELEASE_BUCKET_PREFIX}/appcast2.xml --acl public-read</code></li>
26+
</ul></li>
27+
</ol>
28+
29+
<hr>
30+
<h1>Summary of automated changes</h1>
31+
<h2>Changes to appcast2.xml</h2>
32+
See the attached <em>${APPCAST_PATCH_NAME}</em> file.
33+
<h2>Release notes</h2>
34+
See the attached <em>${RELEASE_NOTES_FILE}</em> file for release notes extracted automatically from <a data-asana-gid='${RELEASE_TASK_ID}' data-asana-dynamic='false'>the release task</a> description.
35+
<h2>List of files uploaded to S3</h2>
36+
<ol>
37+
${FILES_UPLOADED}
38+
</ol>
39+
40+
🔗 Workflow URL: <a href='${WORKFLOW_URL}'>${WORKFLOW_URL}</a>.
41+
</body>

‎.github/actions/asana-log-message/action.yml

+6-42
Original file line numberDiff line numberDiff line change
@@ -47,46 +47,10 @@ runs:
4747
exit 1
4848
fi
4949
50-
- id: process-template-payload
51-
if: ${{ inputs.template-name }}
52-
shell: bash
53-
env:
54-
TEMPLATE_PATH: ${{ github.action_path }}/templates/${{ inputs.template-name }}.yml
55-
run: |
56-
if [ ! -f $TEMPLATE_PATH ]; then
57-
echo "::error::Template file not found at $TEMPLATE_PATH"
58-
exit 1
59-
fi
60-
61-
# Process the template file with envsubst, turn into JSON, remove leading spaces and newlines on non-empty lines, and compact the JSON
62-
payload="$(envsubst < $TEMPLATE_PATH | yq -o=j | sed -E 's/\\n( *)([^\\n])/\2/g' | jq -c)"
63-
echo "payload-base64=$(base64 <<< $payload)" >> $GITHUB_OUTPUT
64-
65-
- id: process-comment-payload
66-
if: ${{ inputs.comment }}
67-
shell: bash
68-
env:
69-
COMMENT: ${{ inputs.comment }}
70-
WORKFLOW_URL: https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }}
71-
run: |
72-
payload="{ \"data\": { \"text\": \"${COMMENT}\n\nWorkflow URL: ${WORKFLOW_URL}\" } }"
73-
echo "payload-base64=$(base64 <<< $payload)" >> $GITHUB_OUTPUT
74-
7550
- id: add-comment
76-
shell: bash
77-
env:
78-
ASANA_ACCESS_TOKEN: ${{ inputs.access-token }}
79-
TASK_ID: ${{ steps.get-automation-subtask.outputs.automation-task-id }}
80-
PAYLOAD_BASE64: ${{ steps.process-template-payload.outputs.payload-base64 || steps.process-comment-payload.outputs.payload-base64 }}
81-
run: |
82-
return_code=$(curl -fLSs "https://app.asana.com/api/1.0/tasks/${TASK_ID}/stories" \
83-
-H "Authorization: Bearer ${ASANA_ACCESS_TOKEN}" \
84-
-H 'content-type: application/json' \
85-
--write-out '%{http_code}' \
86-
--output /dev/null \
87-
-d "$(base64 -d <<< $PAYLOAD_BASE64)")
88-
89-
if [ $return_code -ne 201 ]; then
90-
echo "::error::Failed to add a comment to the Asana task"
91-
exit 1
92-
fi
51+
uses: ./.github/actions/asana-add-comment
52+
with:
53+
access-token: ${{ inputs.access-token }}
54+
task-id: ${{ steps.get-automation-subtask.outputs.automation-task-id }}
55+
comment: ${{ inputs.comment }}
56+
template-name: ${{ inputs.template-name }}

‎.github/actions/asana-upload/action.yml

+1-1
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@ runs:
1717
using: "composite"
1818
steps:
1919
- run: |
20-
curl -s "https://app.asana.com/api/1.0/tasks/${{ inputs.task-id }}/attachments" \
20+
curl -fLSs "https://app.asana.com/api/1.0/tasks/${{ inputs.task-id }}/attachments" \
2121
-H "Authorization: Bearer ${{ inputs.access-token }}" \
2222
--form "file=@${{ inputs.file-name }}"
2323
shell: bash

‎.github/workflows/build_appstore.yml

+1-1
Original file line numberDiff line numberDiff line change
@@ -82,7 +82,7 @@ jobs:
8282
${{ secrets.SSH_PRIVATE_KEY_FASTLANE_MATCH }}
8383
8484
- name: Check out the code
85-
uses: actions/checkout@v3
85+
uses: actions/checkout@v4
8686
with:
8787
submodules: recursive
8888
ref: ${{ inputs.branch || github.ref_name }}

‎.github/workflows/build_notarized.yml

+3-2
Original file line numberDiff line numberDiff line change
@@ -108,7 +108,7 @@ jobs:
108108
steps:
109109

110110
- name: Check out the code
111-
uses: actions/checkout@v3
111+
uses: actions/checkout@v4
112112
with:
113113
submodules: recursive
114114
ref: ${{ env.branch }}
@@ -296,7 +296,8 @@ jobs:
296296
run: |
297297
aws s3 cp \
298298
${{ github.workspace }}/${{ steps.create-dmg.outputs.dmg }} \
299-
s3://${{ env.RELEASE_BUCKET_NAME }}/${{ env.RELEASE_BUCKET_PREFIX }}/
299+
s3://${{ env.RELEASE_BUCKET_NAME }}/${{ env.RELEASE_BUCKET_PREFIX }}/ \\
300+
--acl public-read
300301
301302
- name: Report success
302303
if: ${{ env.upload-to == 's3' }}

‎.github/workflows/bump_internal_release.yml

+2-2
Original file line numberDiff line numberDiff line change
@@ -31,7 +31,7 @@ jobs:
3131
esac
3232
3333
- name: Check out the code
34-
uses: actions/checkout@v3
34+
uses: actions/checkout@v4
3535
with:
3636
submodules: recursive
3737

@@ -71,7 +71,7 @@ jobs:
7171
steps:
7272

7373
- name: Check out the code
74-
uses: actions/checkout@v3
74+
uses: actions/checkout@v4
7575
with:
7676
submodules: recursive
7777
ref: ${{ github.ref_name }}

‎.github/workflows/code_freeze.yml

+2-2
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,7 @@ jobs:
2626
fi
2727
2828
- name: Check out the code
29-
uses: actions/checkout@v3
29+
uses: actions/checkout@v4
3030
with:
3131
submodules: recursive
3232

@@ -95,7 +95,7 @@ jobs:
9595
steps:
9696

9797
- name: Check out the code
98-
uses: actions/checkout@v3
98+
uses: actions/checkout@v4
9999
with:
100100
submodules: recursive
101101
ref: ${{ needs.create_release_branch.outputs.release_branch_name }}

0 commit comments

Comments
 (0)