Ticket #260 (closed task: fixed)

Opened 2 years ago

Last modified 23 months ago

Cover USB-written images better in installation validation testing

Reported by: adamwill Owned by: jskladan
Priority: major Milestone: Fedora 17
Component: Wiki Version:
Keywords: Cc: adamwill
Blocked By: Blocking:

Description

It became clear in the Fedora 16 cycle that use of Fedora images (both live and non-live) written to USB sticks is now extensive, and our current installation validation tests don't really test this method very comprehensively. We should extend the installation validation matrix to cover at least boot and installation in the following situations:

  • Live image written to USB stick using livecd-iso-to-disk
  • Live image written to USB stick using liveusb-creator
  • Live image written to USB stick using dd
  • Non-live image written to USB stick using livecd-iso-to-disk
  • Non-live image written to USB stick using dd

We may want to consider different l-i-t-d parameters also (especially the use or non-use of --format could be significant). We could also cover use of unetbootin as an _informational_ (non-blocking) test, but I'm not super sure about that.

Part of this ticket is deciding on the best way to represent this - whether it's by simply creating new test cases and adding them into the matrix, or somehow re-rendering the matrix to cover these cases as 'variants' of existing test cases, or some other method.

The changes from this ticket and https://fedorahosted.org/fedora-qa/ticket/256 should be co-ordinated (possibly handled by the same person or group) as they both involve similar questions of how to revise the matrix to cover more situations.

This should be completed by Fedora 17 Alpha TC1.

Change History

comment:1 Changed 2 years ago by adamwill

Let's consider https://fedorahosted.org/fedora-qa/ticket/236 to be merged into this - that's the "Non-live image written to USB stick using dd" case, specifically, ensuring that the non-live images are actually hybridized.

comment:2 Changed 2 years ago by jskladan

I'm strongly considering to take this ticket, but I don't know anything about EFI, so who of you, gentlemen. will be so kind to cooperate with me on this?

comment:3 Changed 2 years ago by adamwill

  • Cc adamwill added
  • Owner set to jskladan

I can help out, but you don't need to know much about EFI to understand the problem space, really: so far as representing the results is concerned, all you need to know is that for some test cases, we'll need to somehow extend the matrices to cover results from EFI installs as well as BIOS installs. I can provide a rough list of the test cases in question. It's pretty much like we have results for both i686 and x86-64 right now: we could simply add a third column to the table labelled 'EFI', but there may be better ways to do it than that. I'm not sure how far we can go just adding columns to the table until we want to look at a different way of representing the data.

comment:4 Changed 2 years ago by jskladan

Ad USB written images: it would be for the best, if we added a special category for the USB written images, since this seems to be most in line with the current style (as we have categories for DVD/PXE/Live Images etc). I'm still not really sure, about handling EFI - could you provide the list of tests which are EFI-relevant?

comment:5 Changed 2 years ago by adamwill

Okay, this is my take on the cases we need to consider:

QA:Testcase_Boot_Methods_Boot_Iso QA:Testcase_Boot_Methods_Dvd QA:Testcase_Live_Image_Boot QA:Testcase_install_repository_Mirrorlist_default QA:Testcase_install_repository_DVD_default QA:Testcase_install_repository_Live_Image

These are obviously critical: we really kinda need all possible combinations there. We need to test whether the boot.iso, DVD ISO and live ISOs boot when written to CD/DVD *or* USB, and in each case, we need to test both BIOS and EFI. In each case, we need to be sure the image boots and a straight-through installation works.

QA:Testcase_Anaconda_autopart_install

For EFI installs we need to make sure anaconda handles the EFI system partition differently (this differs from a non-EFI install). For USB installs we need to make sure the USB stick you're installing from is correctly filtered out: it's not used as an install target or a potential bootloader location. So this test is significant to both.

QA:Testcase_Anaconda_Upgrade_New_Bootloader QA:Testcase_Anaconda_Upgrade_Skip_Bootloader QA:Testcase_Anaconda_Upgrade_Update_Bootloader

Similar to above - upgrades can behave differently with USB and EFI installs, particularly in the area of bootloader management. So we need to test these.

The simplest thing we can do is just add EFI and USB columns to the table, with boxes in those columns only for the tests in question, I guess. Other organization scheme suggestions welcome!

comment:6 Changed 2 years ago by jskladan

Even though I'm still not sure about the EFI, I believe, that the USB-installs should be special testcases in each category - i.e. testcases "USB stick using livecd-iso-to-disk" and "USB stick using dd" for Boot.iso/Netinst.iso installation & DVD.iso installation; and "USB stick using livecd-iso-to-disk", "USB stick using dd", and "USB stick using liveusb-creator" for the Live.iso installation.

Especially since we need to provide the "how to" part of the testcase for the usb stick creation, and based on the comment#5 we also have different Expected results (e.g. USB stick is not used as install target, etc).

comment:7 Changed 2 years ago by jskladan

Just one more thing: If we dd dvd.iso to USB stick, should the 'expected results' be that Anaconda uses the repository on the USB stick during installation (as if it was an actual DVD)? Or is this required just for the livecd-iso-to-disk?

comment:8 Changed 2 years ago by adamwill

AIUI, no, we expect it to work like a boot.iso if you dd it, though there's a parameter you can add to make it use the stick as a repo. If you use livecd-iso-to-disk, though, it should add the parameter automatically so the stick should be used as a source.

comment:9 Changed 23 months ago by adamwill

  • Resolution set to fixed
  • Status changed from new to closed

So the ticket stalled, but the work got done: Josef wrote up several tests and we added them to the F17 matrix with positive results, F17 is clearly the best-behaving release yet so far as USB writing is concerned. For the record, the tests created are:

Thanks Josef!

Note: See TracTickets for help on using tickets.