Ticket #261 (closed task: fixed)

Opened 5 years ago

Last modified 2 years ago

Revise upgrade test case set

Reported by: adamwill Owned by: hongqing
Priority: major Milestone: Fedora 18
Component: Test cases Version:
Keywords: Cc: test@…
Blocked By: Blocking:


Our present upgrade test cases are probably a sub-optimal set. We exercise various rarely-used choices - bootloader action, and text upgrade path - but don't exercise more common variations, like installed package sets, install media, and disk layouts. While we can't commit to supporting absolutely any upgrade, we should cover at least some _more_ cases in order to catch major fails.

Based on the experience in F16 and the recommendations in the retrospective, I'd suggest we could consider dropping or downgrading the importance of some tests, perhaps:

and add test cases for all or some of the following scenarios (please add any other suggestions as comments):

  • upgrade from image written to USB stick (livecd-iso-to-disk)
  • upgrade with KDE (and possibly Xfce and LXDE as 'optional' test cases)
  • upgrade an EFI install (see also https://fedorahosted.org/fedora-qa/ticket/256 for organizational issues here)
  • upgrade with bootloader on first partition (not MBR)
  • upgrade with separate /usr , /home and /var partitions
  • upgrade a RAID install

This should be completed ideally by Fedora 17 Alpha phase, and definitely by Beta phase (when upgrade issues become blockers).

Change History

comment:1 Changed 5 years ago by hongqing

  • Owner set to hongqing
  • Status changed from new to assigned

comment:2 Changed 5 years ago by hongqing

  • Component changed from Proventester Mentor Request to Test Review

comment:3 Changed 5 years ago by kparal

We should discuss the issue of upgrading over more than 1 release (e.g. F14 -> F16 directly). We don't have it in the release criteria, but we have a test case for it. That's not optimal.

The installation guide says a direct upgrade over multiple releases is possible:


But it does not say over how many releases. The best way would probably be to ask FESCo how many releases do we want *officialy* support to upgrade over. And then QA should reflect this approach in its test cases. Anaconda should also warn when user tries to upgrade over more than the officially supported number of releases.

I would love to see only one officially supported release to upgrade from (only F15->F16), but that's probably not gonna happen. Anyway, some clear decision would be highly appreciated.

comment:4 Changed 5 years ago by adamwill

The current situation is how it's meant to be - though obviously, since it's possible to interpret it as a problem like you did, we need to clarify things.

It's actually okay to have tests in the matrix which are associated with no release phase. I call these 'advisory tests'. The idea is that there are things we want to test so we catch and file any bugs in them and so we know if they're broken or buggy and can document the problems, but bugs in them are _not_ release blockers. That's fine.

That's what the 'upgrade two releases' test is: it's not a release blocking issue if it fails, but we do want to know whether there are major problems we should document and at least _try_ to fix.

Obviously, we need to make this more clear somehow, so people don't have the same questions about it that you did!

The current 'official' support is exactly what's in the criteria: upgrade from one release ago is release-blocking, upgrade from more than one release ago is not.

comment:5 Changed 5 years ago by athmane

comment:6 follow-up: ↓ 7 Changed 5 years ago by jdulaney

I would remove the bit about GDM being the display manager in case someone has KDM.

comment:7 in reply to: ↑ 6 Changed 5 years ago by athmane

Replying to jdulaney:

I would remove the bit about GDM being the display manager in case someone has KDM.

Fixed, however it seems that GDM is pulled when XFCE group is selected. see [1]

[1] http://git.fedorahosted.org/git/?p=comps.git;a=blob_plain;f=comps-f16.xml.in;hb=HEAD

comment:8 Changed 5 years ago by adamwill

athmane: that's correct. Fedora's Xfce uses GDM as its default login manager, at least presently.

For the record - by default, in Fedora, GNOME uses GDM, KDE uses KDM, Xfce uses GDM, and LXDE uses LXDM.

comment:9 Changed 5 years ago by athmane

comment:10 Changed 5 years ago by jdulaney

Adamw, apologies. I was thinking of LXDE by mistake.

Need to get these pesky brain farts under control.

comment:11 Changed 5 years ago by adamwill

  • Milestone changed from Fedora 17 to Fedora 18

We haven't completed this yet (though thanks to Athmane for the added test cases). We still should. Bumping out to F18.

comment:12 Changed 5 years ago by adamwill

  • Component changed from Test Review to Test cases

comment:13 Changed 2 years ago by adamwill

  • Cc test@… added
  • Status changed from assigned to closed
  • Resolution set to fixed

This more or less got done - we have a rather different set of test cases now. We probably should consider re-*re*-doing it for the Fedora.next flavor split, so we have tests for upgrading each flavor, but we can have a new ticket for that.

Note: See TracTickets for help on using tickets.