NewReleaseCycleProcess

Differences between revisions 221 and 222
Revision 221 as of 2020-11-02 17:20:21
Size: 16658
Editor: cjwatson
Comment: initialize-distroseries now copies chroots, so we don't need a manual step here
Revision 222 as of 2020-11-16 17:06:55
Size: 17149
Editor: brian-murray
Comment: Hirsute opening changes part 1
Deletions are marked like this. Additions are marked like this.
Line 42: Line 42:
 1. Notify Colin Watson to set up seed mirrors and germinate output for the new distroseries.
 1. Disable the Soyuz publisher cron jobs.
 1. Request Launchpad disable the primary Ubuntu publisher cron jobs:
  * https://wiki.canonical.com/InformationInfrastructure/OSA/LaunchpadRollout#Disable_fragile_Soyuz_cron_jobs_.2835_mins_before_rollout.29
Line 45: Line 45:
  * Copy the Release Manager and Drivers forward
Line 50: Line 51:
 1. Visit http://launchpad.net/ubuntu/<series> and click "Initialize Series" in the top right menu. Select which Debian parents you want for this series (it defaults to the existing ones), which architectures you want, ensure "Copy Source And Binaries" is selected and click the "Initialize Series" button.  1. Visit http://launchpad.net/ubuntu/<series> and click "Initialize Series" in the top right menu. Select which Debian parents you want for this series (it defaults to the existing ones), which architectures you want, ensure "Copy Source And Binaries" is selected and click the "Initialize Series" button. Wait for this to finish before running the publisher, this will take 15 to 30 minutes.
Line 53: Line 54:
 1. After the first publisher run has completed, ask #IS to run "ANALYZE sourcepackagepublishinghistory; ANALYZE binarypackagepublishinghistory;" on launchpad_prod.
 1. Re-enable the Soyuz publisher cron job for a second publisher run.
 1. After the first publisher run has completed, ask ~IS to run "ANALYZE sourcepackagepublishinghistory; ANALYZE binarypackagepublishinghistory;" on launchpad_prod.
 1. Re-enable the Soyuz publisher cron job for a second publisher run. Watch the logs to make sure this work.
Line 56: Line 57:
 1. Permanently re-enable the Soyuz publisher cron job
Line 63: Line 65:
 1. Move the bootstrap archive on snakefruit to use the new series.
 1. As lp_buildd@pepo, run `/srv/launchpad.net/production/launchpad/scripts/add-missing-builds.py -s NEW-proposed` plus `-a ARCH` for each architecture in NEW (Adam Conrad and Colin Watson have access, or ask IS).
 1. Move the bootstrap archive on snakefruit to use the new series, so the chroots can reference it.
 1. Ask the launchpad team (cjwatson) to run manage-chroot operations to copy OLDRELEASE buildd chroots forward to NEWRELEASE

 1. As lp_buildd@anonster, run `/srv/launchpad.net/production/launchpad/scripts/add-missing-builds.py -s NEW-proposed` plus `-a ARCH` for each architecture in NEW (Steve Langasek and Colin Watson have access, or ask IS).
Line 71: Line 74:
 1. Notify Colin Watson to set up `merge-o-matic` to point to the new distroseries.
 1. Update `global.conf` in `lp:~ubuntu-transition-trackers/ubuntu-transition-tracker/configs`
 1. Notify someone in the 'merges' LDAP group to set up `merge-o-matic` to point to the new distroseries. Switch to the merge user, cd /srv/patches.ubuntu.com/code/, bzr pull.
 1. Update `global.conf` and go in `lp:~ubuntu-transition-trackers/ubuntu-transition-tracker/configs`
Line 77: Line 80:
 1. Notify a Launchpad admin to set the status of the new distroseries to `DEVELOPMENT`.
 1. Inform `#ubuntu-devel` and `ubuntu-devel-announce` that the new release is now open for uploads, pointing to `merge-o-matic` output.
 1. Once the tool chain is ready:
  * Notify a Launchpad admin to set the status of the new distroseries to `DEVELOPMENT`.
  * Inform `#ubuntu-devel` and `ubuntu-devel-announce` that the new release is now open for uploads, pointing to `merge-o-matic` output.
Line 91: Line 95:
 1. Notify stgraber, vorlon, or bdmurray to follow README.series on db@limequat (the ISO tracker database).  1. Notify stgraber, vorlon, or bdmurray to follow README.series on db@limequat (the ISO tracker database) after sudo'ing to db. Then confirm that the new series has a manifest in the ISO tracker administrative interface.
Line 103: Line 107:
 1. Update `/usr/share/python-apt/templates/Ubuntu.info` in `python-apt` to include the new release name.
 1. Update `ubuntu-release-upgrader` to handle the new release e.g. https://bazaar.launchpad.net/~ubuntu-core-dev/ubuntu-release-upgrader/trunk/revision/3181 and DistUpgradeQuirks e.g. https://bazaar.launchpad.net/~ubuntu-core-dev/ubuntu-release-upgrader/trunk/revision/3232.
Line 107: Line 113:
 1. Review the people listed in '!regression-alert' ([[https://wiki.ubuntu.com/StableReleaseUpdates#Regressions|StableReleaseUpdates#Regressions]]) as needed. '!regression-alert' is handled by [[https://wiki.ubuntu.com/IRC/Bots|ubotto]] and flagged in #ubuntu-devel. Changes can be made and tested in #ubuntu-ops. The people listed should be the tech leads for foundations, desktop, server, security, and QA, along with the release manager and key community members.
Line 111: Line 116:
 1. Merge `lintian`, `vim`, `emacs-goodies-el` and `clang` if necessary and update lists of Ubuntu release names to include the new release name.  1. Merge `lintian` if necessary and update lists of Ubuntu release names to include the new release name.
 1. Merge `vim` if necessary and update lists of Ubuntu release names to include the new release name.
 1. Merge `emacs-goodies-el` if necessary and update lists of Ubuntu release names to include the new release name.
 1. Merge `clang` if necessary and update lists of Ubuntu release names to include the new release name.
Line 114: Line 122:
 1. Update `/usr/share/python-apt/templates/Ubuntu.info` in `python-apt` to include the new release name.
 1. Update `ubuntu-release-upgrader` to handle the new release e.g. https://bazaar.launchpad.net/~ubuntu-core-dev/ubuntu-release-upgrader/trunk/revision/3181 and DistUpgradeQuirks e.g. https://bazaar.launchpad.net/~ubuntu-core-dev/ubuntu-release-upgrader/trunk/revision/3232.
Line 117: Line 123:
 1. Update `syslinux-themes-ubuntu` to handle the new release.
Line 127: Line 132:
 1. Add new series to `production/current-triggers` on nusakan.  1. Add new series to `production/current-triggers` on cdimage-builder.internal.

To be carried out by: Ubuntu Release Team

Goals

  • Unblock development process for new release as quickly as possible.
  • Prepare for first milestone CD release.

Stages

Bugs related to this process are tagged with new-release-cycle-process. If you discover new glitches or things that can be automated, please file a bug with that tag.

Previous release minus 1 month

  1. Contact Soyuz development/production teams to ensure there are no known outstanding issues that will prevent on-time opening of the new distroseries.
  2. If there is a release scheduled to End of Life, start off EndOfLifeProcess for it.

  3. Remind toolchain developers to begin preparing the new toolchain.
  4. Confirm final schedule for the new release and communicate key release dates

Previous release minus 2 weeks

  1. Double-check with Soyuz development/production teams.
  2. Ask for the maillist distroseries-changes to be set up by sending email to ubuntu-platform@rt.canonical.com to file an RT ticket.

  3. Create a release schedule for the new series. If the release name is not yet known, substitute the letter twice, e.g. FF for the F-series.

Previous release plus 1 day

  1. Create new seed branches based on those for the previous release, and push them to the appropriate subdirectories of git+ssh://git.launchpad.net/~ubuntu-core-dev/ubuntu-seeds/+git. The branch-seeds script in lp:ubuntu-archive-tools can do the hard work here.

  2. Update lp:ubuntu-archive-scripts update-seeds, update-germinate etc. for the new name and pull it into snakefruit:~ubuntu-archive/bin/
  3. Request Launchpad disable the primary Ubuntu publisher cron jobs:
  4. Notify a Launchpad admin to create new distroseries with status FROZEN.

    • Copy the Release Manager and Drivers forward
  5. Create milestones in the new series:
    • ubuntu-$version-month-1 etc., set at monthly intervals after the previous release until feature freeze

    • ubuntu-$version-beta or ubuntu-$version-beta-1 etc. as required

    • ubuntu-$version at the next release date

  6. Check that the new distroseries exists with status FROZEN, and that the previous distroseries has status CURRENT.

  7. Visit http://launchpad.net/ubuntu/<series> and click "Initialize Series" in the top right menu. Select which Debian parents you want for this series (it defaults to the existing ones), which architectures you want, ensure "Copy Source And Binaries" is selected and click the "Initialize Series" button. Wait for this to finish before running the publisher, this will take 15 to 30 minutes.

  8. Re-enable the Soyuz publisher cron job for a single publisher run.
  9. Add the new series name to the official Launchpad bug tags, and add verification-needed-PREVIOUS and verification-done-PREVIOUS where PREVIOUS is the name of the previous series.

  10. After the first publisher run has completed, ask ~IS to run "ANALYZE sourcepackagepublishinghistory; ANALYZE binarypackagepublishinghistory;" on launchpad_prod.
  11. Re-enable the Soyuz publisher cron job for a second publisher run. Watch the logs to make sure this work.
  12. As ubuntu-archive@snakefruit, use compare-archives to compare dists trees under ~/mirror/ubuntu/ for previous and current distroseries and sign off on any differences; the only differences should be the distroseries name, that custom uploads (installer-*, dist-upgrader-all, and i18n) are missing from dists/DSN/main, that Release.gpg does not yet exist (this will be created when the full publisher cron job next runs), and that Contents-*.gz do not yet exist (these will be created when generate-contents next runs).

  13. Permanently re-enable the Soyuz publisher cron job
  14. Copy the pending syncs which we didn’t want to accept from previous-proposed UNAPPROVED, and then reject them
  15. Notify Iain Lane (Laney) to run through https://wiki.ubuntu.com/ProposedMigration/AutopkgtestInfrastructure#Opening_up_a_new_series

  16. Initialise the partner repository on archive.canonical.com for the new series. Either:

    • copy packages forward from partner for the previous series (use curl -s http://archive.canonical.com/dists/PREVIOUS/partner/source/Sources.xz | xzcat | grep-dctrl -sPackage,Version '' to list the contents, and copy-package --from ubuntu/partner --from-suite PREVIOUS --to-suite NEW --include-binaries --auto-approve SOURCE to copy individual source packages); or

    • tell Launchpad to create it even though it's empty (mark-suite-dirty -A ubuntu/partner -s NEW).

  17. Grant ~ubuntu-sru queue admin access to the previous distroseries (for pocket in proposed updates; do edit-acl -p ubuntu-sru -S PREVIOUS --pocket $pocket -t admin add; done) and remove ~ubuntu-release's queue admin access (for pocket in release proposed; do edit-acl -p ubuntu-release -S PREVIOUS --pocket $pocket -t admin delete; done).

  18. Check that the process of initialising the new distroseries granted queue admin access to ~ubuntu-release to the new distroseries (edit-acl -p ubuntu-release -S NEW -t admin query).

  19. Move the bootstrap archive on snakefruit to use the new series, so the chroots can reference it.
  20. Ask the launchpad team (cjwatson) to run manage-chroot operations to copy OLDRELEASE buildd chroots forward to NEWRELEASE
  21. As lp_buildd@anonster, run /srv/launchpad.net/production/launchpad/scripts/add-missing-builds.py -s NEW-proposed plus -a ARCH for each architecture in NEW (Steve Langasek and Colin Watson have access, or ask IS).

  22. Add a request to Launchpad devs and admins to open Launchpad translations for the new distroseries.

    • Also make sure this actually happens. (TODO: Work out a better process for keeping track of this in conjunction with Launchpad.)
  23. Modify various reports on snakefruit (at least britney) to point to the new distroseries.

    • britney: ~ubuntu-archive/public_html/proposed-migration: Update symlinks
    • See the bzr log in ~ubuntu-archive/ubuntu-archive-tools for the changes done for the last release and update to the current one.
    • Add the new release to ~ubuntu-archive/.madison-lite/config on snakefruit so that rmadison starts working.
  24. Notify someone in the 'merges' LDAP group to set up merge-o-matic to point to the new distroseries. Switch to the merge user, cd /srv/patches.ubuntu.com/code/, bzr pull.

  25. Update global.conf and go in lp:~ubuntu-transition-trackers/ubuntu-transition-tracker/configs

  26. Notify toolchain developers to upload new toolchain. Iterate uploads as necessary until this has successfully built on all architectures.
  27. Make sure that proposed-migration is configured properly for the new series.
    1. Check that https://people.canonical.com/~ubuntu-archive/proposed-migration/<NEWSERIES>/results.cache exists, as required by bileto. If not, this can be done by creating a new symlink to the proposed-migration’s data directory.

  28. Remove the "block-all source" hint from proposed-migration.
  29. Once the tool chain is ready:
    • Notify a Launchpad admin to set the status of the new distroseries to DEVELOPMENT.

    • Inform #ubuntu-devel and ubuntu-devel-announce that the new release is now open for uploads, pointing to merge-o-matic output.

  30. Check whether there are any uploads in the previous release's -updates pocket not in the new release, and copy them over if so.
  31. Copy everything from PREVIOUS-proposed to DEVEL-proposed, and delete the ones from PREVIOUS-proposed that aren't SRUs.
  32. Inform the SRU team that the bulk copy has been done, so that they know to sort out PREVIOUS-proposed vs. DEVEL-proposed for any new SRUs they accept.
  33. Drop the --dry-run flag from the auto-sync job in ubuntu-archive@snakefruit's crontab.

  34. Contact owners of each image with seeded snaps to have snap channels opened and closed for the new release.
  35. Create new live filesystem configurations for the new distroseries, using branch-livefses in lp:ubuntu-archive-tools. This requires someone in ~launchpad-ppa-admins/~launchpad-ppa-self-admins, ~ubuntu-cdimage and ~launchpad-livefs-builders. This currently needs to be done for OWNER where OWNER is:

    • ubuntu-cdimage
    • cloud-images
    • cloudware
    • cloud-images-release-managers
  36. Create data/RELEASE, tools/RELEASE, and tools/boot/RELEASE directories in debian-cd based on corresponding directories for the previous release. Set OFFICIAL to "Alpha" in CONF.sh for the new release.

  37. Adjust cdimage code to be aware of the new release, and configure ~cdimage/.isotracker.conf

  38. Notify stgraber, vorlon, or bdmurray to follow README.series on db@limequat (the ISO tracker database) after sudo'ing to db. Then confirm that the new series has a manifest in the ISO tracker administrative interface.
  39. Target issues from the previous release's release notes to be fixed in the new release.
  40. Notify Brian Murray or a real ~ubuntu-archive member to create an Apport retracer apt configuration (example for 17.04) for the new release and roll it out to porter-i386:/home/ubuntu-archive/config. Additionally, enable -proposed for the version of Ubuntu which was just released. It'd also be good to clean out the retracer cache (in /home/ubuntu-archive/cache-$arch/) for the previous development release as crashes should no longer be submitted to Launchpad.

  41. Check http://ddebs.ubuntu.com for the new release. (In principle that should happen by itself, but in practice it may crash/hang on archive.ubuntu.com not yet having indexes, or run into time outs on the gigantic first import after opening/copying the new release). This is running on numen.canonical.comsudo -u ddebs -i; This has a checkout of lp:ddeb-retriever and is cron driven.

  42. Notify Iain Lane or someone with access to the user update config.json to add the new release to the list in git lp:~ubuntu-desktop/+git/appstream-cloud (*make sure the json is valid*), and then as prod-ue-appstream-back@wendigo run juju set appstream-generator config="$(cat config.json)" after pulling. See 9163c1a2bc5fb3c2c2ba1c27df8f7573a54b4045.

  43. Notify William Grant to update the ftbfs on qa.ubuntuwire.com.
  44. Notify Rhonda D'Vine (Rhonda) to update packages.ubuntu.com.
  45. Notify Iain Lane (Laney) to update people.c.c/~ubuntu-archive/transitions and udd.debian.org
  46. Update lp:ubuntu-manpage-repository for the new release codename e.g. adding a release, and submit an RT to have the server updated.

  47. Notify Brian Murray to update whoopsie-update-daily-users cronjob in the retracer charm owned by the daisy-pluckers team to include the new release of Ubuntu.
  48. Notify Brian Murray to update the Ubuntu Error Tracker for the new release of Ubuntu e.g. errors change and daisy change

  49. Notify someone with access to the "prod-cnf-extractor" group to enable generation of command-not-found indices (juju config cnf-extractor releases="space separated list of releases")

  50. Update /usr/share/python-apt/templates/Ubuntu.info in python-apt to include the new release name.

  51. Update ubuntu-release-upgrader to handle the new release e.g. https://bazaar.launchpad.net/~ubuntu-core-dev/ubuntu-release-upgrader/trunk/revision/3181 and DistUpgradeQuirks e.g. https://bazaar.launchpad.net/~ubuntu-core-dev/ubuntu-release-upgrader/trunk/revision/3232.

First weeks, after toolchain complete

  1. Notify 'ubuntu-devel' and 'ubuntu-devel-announce' that the release is now open and where to subscribe to the release'-changes' maillist.
  2. Notify the Launchpad team to upload new buildd chroots (which should be done occasionally throughout the cycle, for efficiency).
  3. Merge base-files if necessary and change /etc/issue, /etc/issue.net, /etc/lsb-release, and /etc/os-release to refer to the new release.

  4. Merge debootstrap if necessary and create a bootstrap script for the new release as a copy of the previous one (currently, we can just make the new one a symlink to gutsy, as it hasn't changed since then).

  5. Merge lintian if necessary and update lists of Ubuntu release names to include the new release name.

  6. Merge vim if necessary and update lists of Ubuntu release names to include the new release name.

  7. Merge emacs-goodies-el if necessary and update lists of Ubuntu release names to include the new release name.

  8. Merge clang if necessary and update lists of Ubuntu release names to include the new release name.

  9. Do a no change rebuild of devscripts to pick up the new development release from distro-info.
  10. Do a no change rebuild of app-install-data-partner.
  11. Update meta-release-development to handle the new release.

  12. Merge cdrom-detect, choose-mirror, and iso-scan if necessary and update cdrom/suite and mirror/suite debconf templates to include a choice for the new release and update any previous default.

  13. Merge the rest of the installer in dependency order, ending with debian-installer once all other installer components have built successfully on all architectures.

  14. Notify ubiquity maintainer(s) to run debian/rules update, adjust as necessary to account for changes, and upload.

  15. Ensure that postgresql-common is updated for new release

  16. Update the version number in unity-greeter & unity-control-center logos to refer to the new release

  17. Ensure that the ISO tracker has a daily-build "milestone" for the new release
  18. Create ~ubuntu-archive/wubi/RELEASE/ and symlink ~ubuntu-archive/wubi/RELEASE-1/stable to ~ubuntu-archive/wubi/RELEASE/stable

  19. Edit default_milestone in ~cdimage/.isotracker.conf to match the new milestone name in the ISO tracker

  20. Notify canonical-platform-qa (qa-team@lists.canonical.com) to set up Utah jobs for Ubuntu desktop and server.

  21. Add new series to production/current-triggers on cdimage-builder.internal.

  22. Turn live filesystem and cdimage cron jobs back on.
  23. Add the date of the previous release to calendar.ubuntu in bsdmainutils

  24. Defect Analysts (or team lead for those teams without Defect Analysts) will triage the +nominations bugs list, declining non-SRU candidate nominations and re-target bugs. The release team will evaluate the remaining list, accepting good SRU candidates

  25. Check to make sure the future+1 and future+2 series is available on launchpad to assign bugs to, if not present, then ask webops on #launchpad-ops to add letter-series, with status Future.
  26. Update https://wiki.ubuntu.com/ReleaseTeam/FeatureStatus to new release, and all linked pages.

  27. Two weeks after new release starts, finish EndofLifeProcess for any old releases scheduled now that we've had some migration overlap.

  28. Update ubuntu-release-notes project to know about the new series.
  29. Ask firefox maintainer to update start page for the new release.

  30. Contact flavours to confirm their ongoing participation in the new release, and verify that the contact in the ISO tracker is still accurate.
  31. Continue on MilestoneProcess.


CategoryProcess

NewReleaseCycleProcess (last edited 2023-10-20 14:01:26 by tsimonq2)