Update release automation to support Privacy Pro section in release notes #2710

ayoy · 2024-04-26T12:05:48Z

Task/Issue URL: https://app.asana.com/0/1177771139624306/1207157777571909/f
CC: @loremattei

Description:
This change makes it possible to display a separate section in release notes, dedicated to Privacy Pro:

appcastManager.swift was updated to take pre-formatted HTML document with release notes
extract_release_notes.sh was updated to generate HTML or Asana rich text release notes, in addition to raw output.
update_asana_for_release.sh was updated to rely on extract_release_notes.sh for generating
release notes for the Asana release task.
workflows were updated to use extract_release_notes.sh to generate HTML documents
extract_release_notes.sh is now unit-tested using Bats (Bash Automated Testing System).
PR checks workflow is updated with a job dedicated to running shell scripts unit tests.

Steps to test this PR:

See this test release task.
See the workflow run that uses the new automation.
Check out release/0.81.0 branch locally.
Update release notes in the test release task - change formatting, e.g. remove the H2 for the Privacy Pro release notes header, add more line breaks, etc.

This requires you to have an Asana access token defined in ASANA_ACCESS_TOKEN environment variable:

Run the following script locally (while on release/0.81.0):
./scripts/update_asana_for_release.sh internal 1207179392091595 1206716192269178 0.81.0
The script may take ~10s to run. Verify that it reformats Asana task so that release notes are back into proper shape (refresh Asana website if needed).

Example release notes screen, as generated by the updated appcastManager.swift:

Internal references:

Pull Request Review Checklist
Software Engineering Expectations
Technical Design Template
Pull Request Documentation

…empty or placeholder is provided

github-actions · 2024-04-26T12:07:50Z

	Warnings
⚠️	PR has more than 500 lines of code changing. Consider splitting into smaller PRs if possible.

Generated by 🚫 dangerJS against 4bea8bd

ayoy · 2024-04-26T12:50:15Z

.github/workflows/bump_internal_release.yml

-          release_notes="$(<release_notes.txt)"
-          if [[ ${#release_notes} == 0 || "$release_notes" == "<-- Add release notes here -->" ]]; then
-            echo "::error::Release notes are empty. Please add release notes to the Asana task and restart the workflow."
+            | ./scripts/extract_release_notes.sh -r > raw_release_notes.txt


For validating release notes we don't need formatting, so extract raw release notes here.

ayoy · 2024-04-26T12:50:57Z

.github/workflows/bump_internal_release.yml

-            echo "::error::Release notes are empty. Please add release notes to the Asana task and restart the workflow."
+            | ./scripts/extract_release_notes.sh -r > raw_release_notes.txt
+          raw_release_notes="$(<raw_release_notes.txt)"
+          if [[ ${#raw_release_notes} == 0 || "$raw_release_notes" == *"<-- Add release notes here -->"* ]]; then


Small improvement to validation – we're now looking for the occurrence of the placeholder text within the release notes, not for the exact match.

ayoy · 2024-04-26T12:53:34Z

.github/workflows/code_freeze.yml

@@ -172,6 +172,7 @@ jobs:
    uses: ./.github/workflows/tag_release.yml
    with:
      asana-task-url: ${{ needs.create_release_branch.outputs.asana_task_url }}
+      base-branch: ${{ github.ref_name }}


Required for testing automation (it was accidentally removed in the original automation PR) but harmless for actual release runs, because this workflow asserts earlier that it's run from main.

ayoy · 2024-04-26T12:53:44Z

.github/workflows/pr.yml

+      run: brew install bats-core
+
+    - name: Run Bats tests
+      run: bats --formatter junit scripts/tests/* > bats-tests.xml


ayoy · 2024-04-26T12:54:24Z

.github/workflows/publish_dmg_release.yml

+            | jq -r .data.notes > release_task_content.txt
+          raw_release_notes="$(./scripts/extract_release_notes.sh -r < release_task_content.txt)"
+          if [[ ${#raw_release_notes} == 0 || "$raw_release_notes" == *"<-- Add release notes here -->"* ]]; then
+            echo "::error::Release notes are empty or contain a placeholder. Please add release notes to the Asana task and restart the workflow."


Similar changes as in bump_internal_release.yml

ayoy · 2024-04-26T12:55:35Z

.github/workflows/publish_dmg_release.yml

            exit 1
          fi
-          echo "RELEASE_NOTES_FILE=release_notes.txt" >> $GITHUB_ENV
+          ./scripts/extract_release_notes.sh < release_task_content.txt > release_notes.html


extract_release_notes.sh was called above to extract raw release notes for validation, and now we need to run it again to get HTML output for the appcast.

ayoy · 2024-04-26T13:05:05Z

scripts/appcast_manager/appcastManager.swift

+    let releaseNotesPath = arguments.parameters["--release-notes"]
+    let releaseNotesHTMLPath = arguments.parameters["--release-notes-html"]
+    guard releaseNotesPath != nil || releaseNotesHTMLPath != nil else {


Require either of the plain text or html release notes. If both are provided, plain text takes precedence 🎲 but it's not expected anyway.

Sounds good to me, this approach makes sense.

ayoy · 2024-04-26T13:05:54Z

scripts/appcast_manager/appcastManager.swift

@@ -605,6 +626,27 @@ final class AppcastDownloader {

 // MARK: - Handling of Release Notes

+func handleReleaseNotesHTML(path: String, updatesDirectoryURL: URL, dmgURL: URL) {


Super simple handling: take the HTML input file and copy it to the expected destination. No processing in between.

ayoy · 2024-04-26T13:07:28Z

scripts/extract_release_notes.sh

This script has become complex, but on the positive side, it also got unit tests 💪
Try it yourself:

$ brew install bats-core $ bats scripts/tests/extract_release_notes/extract_release_notes.bats

ayoy · 2024-04-26T13:08:42Z

scripts/update_asana_for_release.sh

@@ -48,7 +48,7 @@ fetch_current_release_notes() {
 	curl -fLSs "${asana_api_url}/tasks/${release_task_id}?opt_fields=notes" \
 		-H "Authorization: Bearer ${ASANA_ACCESS_TOKEN}" \
 		| jq -r .data.notes \
-		| "${cwd}"/extract_release_notes.sh
+		| "${cwd}"/extract_release_notes.sh -a


-a stands for Asana rich text output. Then the release notes processed this way are ready to be added verbatim to the Asana release task description (so we're removing construct_release_notes function)

…utdown # By Dax the Duck (9) and others # Via GitHub (6) and others * main: (47 commits) Update BSK to 141.1.1 (#2713) macOS: VPN Metadata Improvements (#2704) duck page suggestions (#2666) macOS: Add pixels to track VPN wake and stop attempts (#2694) Trusted url indicator (#2665) remove Tab.TabContent.url (#2647) Bump version to 1.85.0 (176) Update release automation to support Privacy Pro section in release notes (#2710) Fix for VPN stop issues. (#2689) Hard-codes the VPN waitlist flags to ON (#2709) Add subscription status to the macOS metadata (#2680) Bump version to 1.85.0 (175) Call finish in the correct place (#2702) Fix layout issue on DBP (#2700) Update autoconsent to v10.6.1 (#2618) Make Clear All History shortcut available without entering Main Menu (#2682) Fix DataBrokerProtectionProcessor.swift lint Fix SwiftLint Add parameter allowed encoding to error descriptions (#2691) Remove debug flags ... # Conflicts: # DuckDuckGo.xcodeproj/project.pbxproj # DuckDuckGo.xcodeproj/project.xcworkspace/xcshareddata/swiftpm/Package.resolved

ayoy added 14 commits April 26, 2024 09:27

Update appcastManager.swift to accept release notes HTML file

ac2fb81

Update extract_release_notes.sh to generate HTML output

2c451bb

Update extract_release_notes.sh to generate Asana output

dfd534e

Make links in Asana output from extract_release_notes.sh

3bcd564

Add initial unit tests for extract_release_notes.sh

ad32d3e

Ignore Privacy Pro release notes in the HTML/Asana output if they're …

a922ef5

…empty or placeholder is provided

Put test assets inside test file

6420a98

Run BATS tests in PR checks

3d2b9b0

Don't run PR checks on push to the branch

9468ad3

Clean up extract_release_notes script

e7625ac

Actually use HTML file in appcastManager

4b3f413

Fix base branch for Tag Release in code_freeze.yml

a0b3bad

Update appcastManager

112d3a4

Fix appcastManager

4bea8bd

ayoy requested a review from samsymons April 26, 2024 12:07

ayoy assigned samsymons Apr 26, 2024

ayoy commented Apr 26, 2024

View reviewed changes

ayoy added the release branch PR label Apr 26, 2024

ayoy requested a review from diegoreymendez April 26, 2024 13:41

samsymons approved these changes Apr 26, 2024

View reviewed changes

ayoy assigned ayoy and unassigned samsymons Apr 26, 2024

diegoreymendez approved these changes Apr 26, 2024

View reviewed changes

ayoy merged commit d931d5c into release/1.85.0 Apr 26, 2024
31 checks passed

ayoy deleted the dominik/pp-release-notes branch April 26, 2024 14:27

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update release automation to support Privacy Pro section in release notes #2710

Update release automation to support Privacy Pro section in release notes #2710

ayoy commented Apr 26, 2024 •

edited

Loading

github-actions bot commented Apr 26, 2024

ayoy Apr 26, 2024

ayoy Apr 26, 2024

ayoy Apr 26, 2024

ayoy Apr 26, 2024

samsymons Apr 26, 2024

ayoy Apr 26, 2024

ayoy Apr 26, 2024

ayoy Apr 26, 2024

samsymons Apr 26, 2024

ayoy Apr 26, 2024

ayoy Apr 26, 2024

ayoy Apr 26, 2024

		@@ -605,6 +626,27 @@ final class AppcastDownloader {

		// MARK: - Handling of Release Notes

		func handleReleaseNotesHTML(path: String, updatesDirectoryURL: URL, dmgURL: URL) {

Update release automation to support Privacy Pro section in release notes #2710

Update release automation to support Privacy Pro section in release notes #2710

Conversation

ayoy commented Apr 26, 2024 • edited Loading

Internal references:

github-actions bot commented Apr 26, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ayoy commented Apr 26, 2024 •

edited

Loading