Ticket #946 (closed task: fixed)

Opened 20 months ago

Last modified 18 months ago

Fedora 18 Beta freeze readiness: is major functionality in place?

Reported by: notting Owned by:
Priority: major Keywords: meeting
Cc: wwoods, adamw, robatino Blocked By:
Blocking:

Description

As part of the anaconda rewrite for Fedora 18, there will be new code for handling upgrades. This code must work by beta freeze, to meet our beta criteria.

This is intended to be tracked in FESCo one week before beta freeze, tentatively, on 2012-09-26.

Change History

comment:1 Changed 20 months ago by robatino

  • Cc robatino added

comment:2 Changed 20 months ago by robatino

  • Summary changed from Tracking of new upgrade funcitonality for F-18 to Tracking of new upgrade functionality for F-18

comment:3 Changed 19 months ago by jreznik

As the Fedora 18 schedules slipped by another one week, current week before beta freeze is Oct 2. This means this should be discusses on on Oct 3 meeting. Also I'd like to extend this ticket not only to the upgrade functionality but to installer's beta release criteria. This way we could avoid long freeze in case Anaconda is not ready by Beta freeze. I'll talk to QA/Anaconda to provide composes with latest builds to review if the criteria are met.

comment:4 Changed 19 months ago by jwboyer

  • Keywords meeting added

Marking this for next week's meeting. jreznik has asked wwoods to provide some updates.

comment:5 Changed 19 months ago by jreznik

Anaconda Work List with detailed status.

comment:6 follow-up: ↓ 8 Changed 19 months ago by jreznik

From wiki linked above, status of 25-Sep-2012:

"From wwoods: So the code is currently here: https://github.com/wgwoods/fedup. Plan for F-18 Beta is to have an F-17 based upgrade tool that fetches packages *or* sets up an upgrade using a local DVD/USB image.

It might not have any real GUI for beta, but it will for final.

It will involve a special upgrade image, but it's just a dracut-built initramfs. This could be built by lorax, or it could just get built by running 'dracut' inside a mock chroot. This might require rel-eng involvement.

I might need some assistance with: i18n stuff, GUI polish, a special plymouht theme for the upgrade, and network monitoring (e.g. running sshd in dracut, like we do for s390 cmdline installs)."

comment:7 follow-up: ↓ 9 Changed 19 months ago by adamwill

We discussed this topic today at the Fedora QA meeting: we may revisit it next week also.

We agreed to some revisions to the Fedora 18 release criteria which are of obvious interest on this topic. The criteria relating to partitioning that will be enforced for Beta are these:

"The installer must be able to complete an installation using automatic partitioning to any sufficiently large target disk, whether unformatted, empty, or containing any kind of existing data"

"The installer must allow the user to select which of the disks connected to the system will be affected by the installation process. Disks not selected as installation targets must not be affected by the installation process in any way"

"The installer must be able to complete an installation using automatic partitioning to a validly-formatted disk with sufficient empty space, using the empty space and installing a bootloader but leaving the pre-existing partitions and data untouched"

"The installer's custom partitioning mode must be capable of the following:

  1. Creating, destroying and assigning mount points to partitions of any specified size using most commonly-used filesystem types
  2. Creating encrypted partitions
  3. Rejecting obviously invalid operations without crashing"

We also agreed on a revised Beta criterion for upgrading:

"It must be possible to successfully complete an upgrade from a fully updated installation of the previous stable Fedora release with the 'minimal' package set or the package set for a release-blocking desktop, using any officially recommended upgrade mechanism. The upgraded system must meet all release criteria."

Note the use of the word 'or' there is ambiguous and we will improve it: the intent is that it must be possible to upgrade to Fedora 18 Beta from *all three* of those Fedora 17 package sets, not just from *any one* of them. In practice this criterion is not really a change from the status quo prior to F18, it still means 'upgrade has to work at Beta' - it's just better wording for a world in which the upgrade process isn't necessarily part of anaconda.

So with those criteria - plus the other Beta criteria, which so far we believe do not require modification - established as the 'ground rules', our assessment is that Fedora 18 as it stands *right now* is not freeze-able for Beta. Our understanding is that the agreement is that we will not freeze until the basic code relating to all Beta release requirements is in place. The custom partitioning code in 18.10, the latest available anaconda build, does not have code to meet all the requirements above - it does not include 'autopart-into-empty-space' - and the new upgrade tool has not yet seen a testable release (or any release).

However, the anaconda team affirmed at the meeting that they believe the required code will be in place by the freeze date, 10-09. If the upgrade tool is in a testable state and the partitioning code covers the requirements stated above by 10-09, we would consider that to be good enough for the freeze to occur.

comment:8 in reply to: ↑ 6 Changed 19 months ago by mitr

Replying to jreznik:

"From wwoods: So the code is currently here: https://github.com/wgwoods/fedup. Plan for F-18 Beta is to have an F-17 based upgrade tool that fetches packages *or* sets up an upgrade using a local DVD/USB image.

It might not have any real GUI for beta, but it will for final.

Per http://fedoraproject.org/wiki/Fedora_18_Beta_Release_Criteria , beta is supposed to be code complete. Is the above acceptable?

comment:9 in reply to: ↑ 7 Changed 19 months ago by mitr

Replying to adamwill:

"The installer's custom partitioning mode must be capable of the following:

  1. Creating, destroying and assigning mount points to partitions of any specified size using most commonly-used filesystem types

To clarify, does this require that it must be possible to assign a mount point to a partition on RAID/LVM, or to an encrypted partition or not? If it does require RAID/LVM, does it require arbitrary configurations of the storage stack or a limited subset?

comment:10 Changed 19 months ago by adamwill

  • Summary changed from Tracking of new upgrade functionality for F-18 to Fedora 18 Beta freeze readiness: is major functionality in place?

Adjust summary per jwb request. We would like this ticket to be a general review of whether the Fedora 18 codebase is ready to be frozen for Beta, i.e., is code to cover all major F18 Beta requirements in place at the freeze date.

As well as the new upgrade tool, the other currently known area of concern is partitioning. anaconda 18.11 (latest available version at time of comment) does not contain code to cover the Beta partitioning requirements. Other areas of concern may also be discovered.

comment:11 Changed 19 months ago by mmaslano

FESCo: #agreed short special meeting on Monday (2012-10-08) at 17utc. Those that cannot attend will vote in the ticket.

Please keep us post it about state of the code.

comment:12 Changed 19 months ago by jreznik

The current status:

  • fedup (in testable state) will be ready by the end of the week
  • partitioning
    • raid code is in the current build, in testable state
    • luks code under patch review (posted on Thursday)
  • Current blocker/proposed blocker bugs list

comment:13 Changed 19 months ago by jreznik

Agreed: Due to upgrade functionality not being in a testable state, push schedule 1 week to allow it to land before entering beta freeze (+1 6, 0 0, -1 0 for FESCo, +1 for the whole QA, +1 for FPGM)

Info: to use this ticket to revisit the decision, freeze by default next week

comment:14 Changed 18 months ago by adamwill

So, it's a week later, and the upgrade tool still is not available for testing. Additionally, RH has asked its staff on the anaconda team to prioritize work on a pending RHEL release over work on Fedora 18, and that is happening, which may further delay work on the upgrade tool and the current blocker list. Note the blocker tracking app - http://qa.fedoraproject.org/blockerbugs/current - may soon go down for an upgrade; if it's down, you can check Tim's personal dev instance at http://supermegawaffle.com/blockerbugs-devel/ to get the blocker list.

As discussed at this morning's QA meeting, QA still cannot recommend we freeze for Beta with the upgrade tool incomplete and no firm ETA. It's worth noting that, as the upgrade tool would be packaged for Fedora 17 for upgrade to Fedora 18, it does not necessarily technically affect the F18 Beta spin process. However its smooth functioning may require changes to F18 components (e.g. dracut). We would also be very hesitant to freeze for Beta - and, perhaps, eventually ship it - with the upgrade story 'we're writing a brand new upgrade tool which will be the sole recommended upgrade method for F18 Final, but we haven't finished it yet and we don't know when we will. We're hoping it'll show up some time between now and Final, and work. We'll keep you posted. In the meantime, use yum, which we always tell you not to use'.

That would suck.

Aside from the upgrade tool, I'm not aware of any ways in which Beta is significantly code-incomplete at this point, the partitioning code is mostly there, although there are still some big bugs. There are five unfixed confirmed blockers on the list (we have to have all confirmed blockers fixed before we start spinning RCs, by policy) and nine unfixed proposed blockers which we need more data to evaluate and confirm or reject as blockers.

comment:15 Changed 18 months ago by jreznik

Adam, thanks for comment.

Current fedup status (based on wwoods input): it is still not yet in testable state.

What works:

  • Fetching packages from network working
    • GUI is in-progress
  • Media detection works (DVD/USB/etc)
  • Messing with bootloader works
  • Hooks for third-party upgrade "plugins" present
    • Migration stuff can be handled by packagers
  • Running the upgrade transaction works
  • Plymouth progress working

TODOs:

  • boot-swap-o-rama, needs systemd/dracut help
  • build upgrade image (lorax)
  • packaging
  • finish support for installing from media
  • finish GUI

comment:16 follow-up: ↓ 17 Changed 18 months ago by kevin

Sadly, I guess I am for another 1 week slip before freeze here.

That would put release at: 2012-12-11

Could other fesco folks vote or provide counter proposals?

In addition, can we reach out and try and get help for any of the remaining todos? Can we get some time from systemd/dracut folks, help with packaging/review, or any design work needed for the GUI?

comment:17 in reply to: ↑ 16 Changed 18 months ago by jreznik

Replying to kevin:

Sadly, I guess I am for another 1 week slip before freeze here.

That would put release at: 2012-12-11

Could other fesco folks vote or provide counter proposals?

In addition, can we reach out and try and get help for any of the remaining todos? Can we get some time from systemd/dracut folks, help with packaging/review, or any design work needed for the GUI?

I'm going to poke systemd/dracut folks to help with fedup, also some kind of guide how-to test current code in the git would be nice to have.

FESCo members: as the Beta Change Freeze is tomorrow (Tue 2012-10-16), please vote (or propose counterproposal of freeze) in the ticket as soon as possible.

comment:18 Changed 18 months ago by limb

+1 slip.

comment:19 Changed 18 months ago by tmraz

I am +1 to slip the freeze one more week.

comment:20 Changed 18 months ago by notting

+1. Not enthused, but do not see other options.

comment:21 Changed 18 months ago by pjones

+1 to slip for another week.

comment:22 Changed 18 months ago by mitr

+1 to slip.

comment:23 Changed 18 months ago by mmaslano

+1 to slip freeze.

comment:24 Changed 18 months ago by jreznik

+7 to slip and push schedule one week due to missing required features (fedup tool, not yet in testable state). Beta Change Deadline/Features? 100% Complete is now Oct 23, Beta Release is Nov 06.

comment:25 Changed 18 months ago by jreznik

Current fedup status: it should be ready for QA to look on it

  • known issues:
    • systemd issues are worked on, Lennart knows about the needed patch + discussion about better upgrade support ongoing
    • plymouth issue, not a blocker for beta
    • logging to journal does not work, not a blocker
    • lorax patch needed
  • in case of no surprises, Will is ready to start packaging/test documentation

Well, fedup should be in testable state (not internal only this week), with luck also packaged and ready for Beta...

comment:26 Changed 18 months ago by notting

What do we need to get it packaged? In any case, I'm tentatively +1 to *not* preemptively slipping this week.

comment:27 Changed 18 months ago by mmaslano

I will agree with QA. Does it match release criteria for Beta?

comment:28 follow-up: ↓ 31 Changed 18 months ago by kevin

I have a middle road idea. I know we do slips in 1 week increments, but in this case how about we instead do:

As soon as the upgrade tool is packaged and available for testing, we freeze. So, if that happens tomorrow, we freeze tomorrow. If it happens friday, we freeze friday. Then we just jump on our regular freeze schedule after that (go/no-gos on thursdays, release on tuesdays, etc). This may give us a bit more non freeze time than simply slipping a week.

Thoughts?

comment:29 Changed 18 months ago by adamwill

QA discussed this at the meeting this morning. We still can't really recommend freezing with the tool not yet available for testing (jreznik says Will says it's testable, but we haven't been given anything). There are two days before the freeze date, but as things stand, QA still doesn't recommend freezing. However, we note that we considered the issue strictly on the QA merits, and FESCo may take a wider view and decide to freeze. If FESCo does vote to freeze, we would like to see a definite plan for how we can ship Beta even without the upgrade tool being fully ready.

If we receive the upgrade tool for testing by Wednesday and it passes smoke testing, I will update the ticket again.

comment:30 Changed 18 months ago by jreznik

The schedule should be taken into consideration, we are getting close to the Christmas - see ticket 960 - with Red Hat being closed... Kevin's proposal can give us a some time, to avoid one week long slips (as we can freeze once the feature is in testable state). But we still need a backup plan for Beta as we can't wait forever, we can consider alternative upgrade methods to fedup - yum (even we discourage users from using it), preupgrade (if still possible) or relax Beta release criteria in some way.

comment:31 in reply to: ↑ 28 Changed 18 months ago by jwboyer

Replying to kevin:

I have a middle road idea. I know we do slips in 1 week increments, but in this case how about we instead do:

As soon as the upgrade tool is packaged and available for testing, we freeze. So, if that happens tomorrow, we freeze tomorrow. If it happens friday, we freeze friday. Then we just jump on our regular freeze schedule after that (go/no-gos on thursdays, release on tuesdays, etc). This may give us a bit more non freeze time than simply slipping a week.

Thoughts?

I also don't want to preemptively freeze, and I think this is a good proposal.

At the moment, I have no thoughts on a backup plan in regards to the schedule.

comment:32 follow-up: ↓ 35 Changed 18 months ago by pjones

I'm generally for Kevin's proposal, though we may want to make the trigger be a successful test compose, rather than simply having the package exist.

comment:33 Changed 18 months ago by notting

One of the problems we had in the past was confusion about when we were frozen, when it wasn't tied to a particular date. It also leaves the FESCo meeting where we review feature completion (as an example) up in the air.

So while we could do it as a one-off for this feature, I'm not really sure it's the best idea.

comment:34 Changed 18 months ago by adamwill

"But we still need a backup plan for Beta as we can't wait forever, we can consider alternative upgrade methods to fedup - yum (even we discourage users from using it), preupgrade (if still possible) or relax Beta release criteria in some way."

preupgrade is not still possible. preupgrade relied on the upgrade code in anaconda - all preupgrade really did was download all the packages, put them in a directory, download the anaconda kernel/initramfs, write a kickstart which said 'do an upgrade install using the packages in this directory', and write a bootloader entry which ran the installer kernel/initramfs with that kickstart. (simple, eh?) since The New Anaconda has no upgrade code, there is no way preupgrade can possibly work any more, unless someone puts upgrade code back into the installer.

really, I think yum is the only available 'fallback plan'. I'm not personally terribly happy with that idea as it would _really_ screw up our messaging around upgrades - "so wait, Fedora has a new upgrade tool called yum? Oh, that's not it? But I have to use yum now? But I shouldn't ever use yum again? What should I use instead? preupgrade?", oh the hilarity - but it is at least viable. And in purely practical terms, F17 to F18 happens to be a bump which yum handles pretty well, we don't have any crazy bootloader changes or /usr move stuff in F18.

comment:35 in reply to: ↑ 32 ; follow-ups: ↓ 36 ↓ 44 Changed 18 months ago by jreznik

Replying to pjones:

I'm generally for Kevin's proposal, though we may want to make the trigger be a successful test compose, rather than simply having the package exist.

Actually the package is F17 one, in F18 as far as I understand we need only patched systemd. That leads to another question if we could decouple fedup release from Beta release. Fedup has to land to F17 and in case of online update, required stuff for F18 could be pulled from repos even in case it's not ready by Beta. I understand QA won't be happy with this "solution" and we really have to be sure it lands at least shortly after Beta release.

comment:36 in reply to: ↑ 35 ; follow-up: ↓ 37 Changed 18 months ago by mmaslano

Replying to jreznik:

Replying to pjones:

I'm generally for Kevin's proposal, though we may want to make the trigger be a successful test compose, rather than simply having the package exist.

Actually the package is F17 one, in F18 as far as I understand we need only patched systemd. That leads to another question if we could decouple fedup release from Beta release. Fedup has to land to F17 and in case of online update, required stuff for F18 could be pulled from repos even in case it's not ready by Beta. I understand QA won't be happy with this "solution" and we really have to be sure it lands at least shortly after Beta release.

I agree with jreznik. If the fedup is in anaconda before release of F-18, then it's fine with me.

comment:37 in reply to: ↑ 36 Changed 18 months ago by jwboyer

Replying to mmaslano:

Replying to jreznik:

Replying to pjones:

I'm generally for Kevin's proposal, though we may want to make the trigger be a successful test compose, rather than simply having the package exist.

Actually the package is F17 one, in F18 as far as I understand we need only patched systemd. That leads to another question if we could decouple fedup release from Beta release. Fedup has to land to F17 and in case of online update, required stuff for F18 could be pulled from repos even in case it's not ready by Beta. I understand QA won't be happy with this "solution" and we really have to be sure it lands at least shortly after Beta release.

I agree with jreznik. If the fedup is in anaconda before release of F-18, then it's fine with me.

Did you mean "in F17" ? Fedup isn't going to be in anaconda, ever. That's the entire point of this whole exercise of removing upgrade functionality from anaconda.

comment:38 Changed 18 months ago by jreznik

Yep, it's not related to Anaconda.

Well, we need some kind of decision - do FESCo wants to vote on Kevin's proposal? As it's the only I found here or just vote on freeze/not freeze + if freeze, how to proceed with upgrades (see my comments with a few ideas, forget preupgrade ;-).

comment:39 Changed 18 months ago by mmaslano

I'm fine with nirik proposals if it helps. Let's hope fedup will be available for testing soon.

comment:40 Changed 18 months ago by tmraz

+1 to Kevin's proposal from me as well.

comment:41 Changed 18 months ago by limb

+1 for Kevin's idea, it should minimize slippage.

comment:42 follow-up: ↓ 43 Changed 18 months ago by mitr

-1 to Kevin's proposal. beta is supposed to be 100% feature complete, and looking at comment 25, that's apparently not the case.

I think we should decide whether we cut the feature scope (= what kinds of upgrades are supported), or slip; not go ahead with a known incomplete beta and risk having to cut the scope for final.

(Unless I am mistaken, the proposal has +5 already, though.)

comment:43 in reply to: ↑ 42 ; follow-up: ↓ 45 Changed 18 months ago by kevin

Replying to mitr:

-1 to Kevin's proposal. beta is supposed to be 100% feature complete, and looking at comment 25, that's apparently not the case.

I think we should decide whether we cut the feature scope (= what kinds of upgrades are supported), or slip; not go ahead with a known incomplete beta and risk having to cut the scope for final.

I didn't suggest that. I simply said we don't freeze today (since we are not 100% feature complete), but if we are tomorrow, we do so then instead of waiting until next tuesday to decide to freeze. I realize that could confuse folks, but I think it buys us a partial week slip hopefully so it's worth it.

(Unless I am mistaken, the proposal has +5 already, though.)

I think so, it's hard to tell. ;)

comment:44 in reply to: ↑ 35 ; follow-up: ↓ 46 Changed 18 months ago by drago01

Replying to jreznik:

Replying to pjones:

I'm generally for Kevin's proposal, though we may want to make the trigger be a successful test compose, rather than simply having the package exist.

Actually the package is F17 one, in F18 as far as I understand we need only patched systemd. That leads to another question if we could decouple fedup release from Beta release. Fedup has to land to F17 and in case of online update, required stuff for F18 could be pulled from repos even in case it's not ready by Beta. I understand QA won't be happy with this "solution" and we really have to be sure it lands at least shortly after Beta release.

Well that really depends on when is "shortly after beta" ... the whole point of requiring this by beta time is to have sufficient time for testing it. Given the recent track record "shortly after beta" could be "a few days before GA". I still think dismissing #944 as "insane" was a mistake but well it is to late for that.

comment:45 in reply to: ↑ 43 Changed 18 months ago by mmaslano

Replying to kevin:

Replying to mitr:

-1 to Kevin's proposal. beta is supposed to be 100% feature complete, and looking at comment 25, that's apparently not the case.

I think we should decide whether we cut the feature scope (= what kinds of upgrades are supported), or slip; not go ahead with a known incomplete beta and risk having to cut the scope for final.

I didn't suggest that. I simply said we don't freeze today (since we are not 100% feature complete), but if we are tomorrow, we do so then instead of waiting until next tuesday to decide to freeze. I realize that could confuse folks, but I think it buys us a partial week slip hopefully so it's worth it.

(Unless I am mistaken, the proposal has +5 already, though.)

I think so, it's hard to tell. ;)

I agreed with nirik's "As soon as the upgrade tool is packaged and available for testing, we freeze." I didn't say let's freeze no matter what. We should really stated proposals better.

comment:46 in reply to: ↑ 44 Changed 18 months ago by mmaslano

Replying to drago01:

Replying to jreznik:

Replying to pjones:

I'm generally for Kevin's proposal, though we may want to make the trigger be a successful test compose, rather than simply having the package exist.

Actually the package is F17 one, in F18 as far as I understand we need only patched systemd. That leads to another question if we could decouple fedup release from Beta release. Fedup has to land to F17 and in case of online update, required stuff for F18 could be pulled from repos even in case it's not ready by Beta. I understand QA won't be happy with this "solution" and we really have to be sure it lands at least shortly after Beta release.

Well that really depends on when is "shortly after beta" ... the whole point of requiring this by beta time is to have sufficient time for testing it. Given the recent track record "shortly after beta" could be "a few days before GA". I still think dismissing #944 as "insane" was a mistake but well it is to late for that.

It wouldn't help to use old anaconda. I asked before the decision about #944. Old anaconda would also have bugs, because it wouldn't work with F-18 without lot of fixes and we would end up with two broken versions.

For the future FESCo must review contingency plan better and follow on important features more closely. Especially, if there is not any contingency plan.

comment:47 Changed 18 months ago by jreznik

Well, as I don't see freeze notification, I expect the result of this ticket was to go with Kevin's proposal (and mitr pointed it was already +5). Just checking, it would be great to communicate this - I can take care.

comment:48 Changed 18 months ago by kevin

Yeah. So, we are waiting for the word from wwoods. As soon as fedup is packaged and ready for testing, we should freeze. If that doesn't happen before next monday, we should re-evaluate again I suppose, but I'm really hoping for somtime in the next few days. ;)

comment:49 Changed 18 months ago by limb

#agreed Freeze now for Beta, stick to current schedule. (+:5,-:1,0:0)

comment:50 Changed 18 months ago by limb

  • Status changed from new to closed
  • Resolution set to fixed
Note: See TracTickets for help on using tickets.