Ticket #173 (closed enhancement: fixed)

Opened 3 years ago

Last modified 3 years ago

Improve the install sources tests to adapt F-15 anaconda

Reported by: rhe Owned by: rhe
Priority: major Milestone: Fedora 15
Component: Wiki Version:
Keywords: Cc: jlaska, clumens@…
Blocked By: Blocking:

Description

problem

Now that install.img is included in initrd.img, it can no longer be loaded from somewhere else other than media, therefore Using 'askmethod' is actually the same as 'repo=' which is to direct anaconda using a specific package repo.

analysis

So I propose to obsolete the install source test cases of install sources from http, nfs, ftp, hd, and only test the package repositories of these methods.

enhancement recommendation

The following tests will be kept:

The following will be obsoleted:

Current package repository tests are listed at: Category:Repository

Hard disk and nfs iso repo tests need be added, then they'll cover all package install methods.

clumens, how do you think of these? do the remaining tests(install source+repo) cover the common use cases?

Change History

comment:1 follow-up: ↓ 2 Changed 3 years ago by jlaska

Thanks for raising this topic! We'll need to confirm with clumens of course, but I don't know if there are plans to add graphical repository configuration for NFS ISO, HD and HD ISO installation methods. I don't believe there are plans to remove loader handling for package repository access methods. Which means, Fedora will still support traditional installation methods available by way of askmethod.

Perhaps we need to rename/rethink these different tests?

The two functions that need verification are ...

  1. the loader handling/setup of a given installation repository. I'm thinking these are the operations that dictate how the 'Installation repo' is setup. For some boot methods, this is handled for us (e.g. DVD creates a CDROM repo, boot.iso and pxeboot don't add any 'installation repo')
  2. the graphical repository dialog allowing for additional repos, repo removal, and existing repo edits. These are mostly manual (except for kickstart) functions.

As for loader handling/setup of a given installation repository, we have the following scenarios (most of which are already tests, but perhaps need adjustment/rename):

  1. Installation repository - http
  2. Installation repository - NFS
    • default scenario
      1. Use graphical repository dialog to input NFS Server and PATH
    • variation(s):
      1. Boot with: repo=nfs:<server>:/<path>
  3. Installation repository - NFS ISO
    • default scenario
      1. Boot with 'askmethod' and manually enter NFS ISO server + path
    • variation(s):
      1. Boot with: repo=nfsiso[:options]:<server>:/<path>
  4. Install repository - HD ISO
    • default scenario
      1. Boot with 'askmethod' and manually enter HD device and path
    • variations:
      1. repo=hd:<device>:/<path>
      2. repo=hd:LABEL=<label>:/<path>
      3. repo=hd:UUID=<uuid>:/<path>
  5. Install repository - CD/DVD ISO
    • default scenario
      1. Boot and install from a CD or DVD ISO image
    • variations:
      1. Boot with 'askmethod' and selects CD-ROM/DVD-ROM
      2. Boot with repo=cdrom:<device>

If we include the FTP* tests you list as a variation of the HTTP test, we can remove them altogether.

As for graphical repository edits, we have those covered already right?

https://fedoraproject.org/wiki/Category:Repository

comment:2 in reply to: ↑ 1 Changed 3 years ago by rhe

Replying to jlaska:

Thanks for raising this topic! We'll need to confirm with clumens of course, but I don't know if there are plans to add graphical repository configuration for NFS ISO, HD and HD ISO installation methods. I don't believe there are plans to remove loader handling for package repository access methods. Which means, Fedora will still support traditional installation methods available by way of askmethod.

Ok, if it's still supported, there should be tests covering it.

Perhaps we need to rename/rethink these different tests?

Yeah, and your grouping idea reminds me a way to do it: Combine current Install source(except install source on Media tests) and Package repository together, naming it as 'Installation repository'. In this category, cases are sub-categoried by http, NFS, NFSISO and CD/DVD. How does this sound?

The two functions that need verification are ...

  1. the loader handling/setup of a given installation repository. I'm thinking these are the operations that dictate how the 'Installation repo' is setup. For some boot methods, this is handled for us (e.g. DVD creates a CDROM repo, boot.iso and pxeboot don't add any 'installation repo')
  2. the graphical repository dialog allowing for additional repos, repo removal, and existing repo edits. These are mostly manual (except for kickstart) functions.

As for loader handling/setup of a given installation repository, we have the following scenarios (most of which are already tests, but perhaps need adjustment/rename):

  1. Installation repository - http
    • default scenario
      1. Boot and install using a netinst.iso or pxeboot images

Current test is: InstallSourceBootIso (using default retrieved mirror repo)

  • variation(s):
    1. Boot with: 'askmethod' and manually input HTTP repo

Current test is: InstallSourceHttp, need rename it and modify the content about 'install source' part.

  1. Boot with: repo=http://server/path/to/repo

Current test is: QA:Testcase_Http_Repository

  1. Use ftp://server/path/to/repo
  2. Use ftp://user:pass@server/path/to/repo

If we include FTP tests in HTTP tests, these two can be included in the above two http cases as a choice to test?

  1. Configure proxy=[protocol://][username[:password]@]host[:port]

No test case yet. I think this can be included as an optional step in the http repo tests?

  1. Installation repository - NFS
    • default scenario
      1. Use graphical repository dialog to input NFS Server and PATH

Current test is: QA:Testcase_Additional_NFS_Repository

  • variation(s):
    1. Boot with: repo=nfs:<server>:/<path>

Current test is: QA:Testcase_Nfs_Repository

and 2. Boot with 'askmethod'? InstallSourceNfs is the test to be adjusted.

  1. Installation repository - NFS ISO
    • default scenario
      1. Boot with 'askmethod' and manually enter NFS ISO server + path

Current test to be adjusted: InstallSourceNfsIso

  • variation(s):
    1. Boot with: repo=nfsiso[:options]:<server>:/<path>

No test case covering it yet. Need to create it.

  1. Install repository - HD ISO
    • default scenario
      1. Boot with 'askmethod' and manually enter HD device and path

InstallSourceHardDrive

  • variations:
    1. repo=hd:<device>:/<path>
    2. repo=hd:LABEL=<label>:/<path>
    3. repo=hd:UUID=<uuid>:/<path>

No test case yet. The above three can be combined into one test in my opinion.

  1. Install repository - CD/DVD ISO
    • default scenario
      1. Boot and install from a CD or DVD ISO image

Current test is InstallSourceDvd.

  • variations:
    1. Boot with 'askmethod' and selects CD-ROM/DVD-ROM
    2. Boot with repo=cdrom:<device>

No test case yet. Do we need cover these two? I don't think they are common use cases.

If we include the FTP* tests you list as a variation of the HTTP test, we can remove them altogether.

Can you clarify this? do you mean ftp can be a option in http test cases?

As for graphical repository edits, we have those covered already right?

Yes. They may need to rename then.

comment:3 follow-up: ↓ 4 Changed 3 years ago by jlaska

I like your idea of creating/renaming tests to be Installation repository. In addition to that, some test rewording may be required. I think all the same installation sources are still supported, but the phrasing, and the procedure, has changed slightly.

An attempt to identify the methods for testing the different Installation repository scenarios. This doesn't include the Additional repository tests. As you noted, those may need to be run also :(

Installation Repository Boot media used Default test scenario Additional test variations
DVD DVD.iso Boot DVD with no additional arguments
  • askmethod, and select CDROM/DVD
  • repo=cdrom
  • repo=cdrom:/dev/sr0
Mirrorlist netinst.iso or pxeboot Boot netinst.iso with no additional arguments  
HTTP/FTP netinst.iso, pxeboot or DVD Boot netinst.iso with ''askmethod'', select URL
  • repo=http://host/path
  • repo=https://host/path
  • proxy=[protocol://][username[:password]@]host[:port]
NFS netinst.iso, pxeboot or DVD Boot with ''askmethod'', select NFS
  • repo=nfs:server:path
NFS ISO netinst.iso, pxeboot or DVD Boot with ''askmethod'', select NFS
  • repo=nfs[:options]:server:path
  • repo=nfsiso[:options]:server:path
HD ISO netinst.iso, pxeboot or DVD Boot with ''askmethod'', select ''Hard drive''
  • repo=hd:device:/path
  • repo=hd:LABEL=:/path
  • repo=hd:UUID=:/path

What do you think about the idea of including additional variations inside a test case. My thought here is that the test could still pass using the default test scenario, but would result in a warning if any of the optional test variations fail. Is this confusing, should we just create distinct test cases for each of the optional test variations? I was hoping to avoid that, but not if it's too confusing.

Can you clarify this? do you mean ftp can be a option in http test cases?

Yeah, since pycurl is used now for HTTP and FTP, I'm wondering if we need to explicitly call out FTP in our tests. Should we leave that as a variation?

comment:4 in reply to: ↑ 3 Changed 3 years ago by rhe

Replying to jlaska:

I like your idea of creating/renaming tests to be Installation repository. In addition to that, some test rewording may be required. I think all the same installation sources are still supported, but the phrasing, and the procedure, has changed slightly.

An attempt to identify the methods for testing the different Installation repository scenarios. This doesn't include the Additional repository tests. As you noted, those may need to be run also :(

No worries, then they'll be included in Installation repository.

Installation Repository Boot media used Default test scenario Additional test variations
DVD DVD.iso Boot DVD with no additional arguments
  • askmethod, and select CDROM/DVD
  • repo=cdrom
  • repo=cdrom:/dev/sr0
Mirrorlist netinst.iso or pxeboot Boot netinst.iso with no additional arguments  
HTTP/FTP netinst.iso, pxeboot or DVD Boot netinst.iso with ''askmethod'', select URL
  • repo=http://host/path
  • repo=https://host/path
  • proxy=[protocol://][username[:password]@]host[:port]
NFS netinst.iso, pxeboot or DVD Boot with ''askmethod'', select NFS
  • repo=nfs:server:path
NFS ISO netinst.iso, pxeboot or DVD Boot with ''askmethod'', select NFS
  • repo=nfs[:options]:server:path
  • repo=nfsiso[:options]:server:path
HD ISO netinst.iso, pxeboot or DVD Boot with ''askmethod'', select ''Hard drive''
  • repo=hd:device:/path
  • repo=hd:LABEL=:/path
  • repo=hd:UUID=:/path

Wow, I'm impressed by your table format here, which inspired me to learn trac syntax.:)

What do you think about the idea of including additional variations inside a test case. My thought here is that the test could still pass using the default test scenario, but would result in a warning if any of the optional test variations fail. Is this confusing, should we just create distinct test cases for each of the optional test variations? I was hoping to avoid that, but not if it's too confusing.

It also depends on our test coverage IMO. First, the default test scenario must be covered in every run. For the additional test variations, should all of them or at least one of them be tested for each source? if the latter case, I prefer to write all Additional variations of each source in one case, then each source has two tests(except graphical): default one, and anyone from the variations. How do you think?

Can you clarify this? do you mean ftp can be a option in http test cases?

Yeah, since pycurl is used now for HTTP and FTP, I'm wondering if we need to explicitly call out FTP in our tests. Should we leave that as a variation?

Yeah, since ftp is still a use case, we can leave that as a variation of HTTP/FTP source.

comment:5 follow-up: ↓ 6 Changed 3 years ago by jlaska

Note, while testing, hansg reminded us that many of our tests refer to stage1 and stage2. With Fedora 15, the previous distinction between stage1 and stage2 no longer applies. There is still a difference, but the difference now is between loader (old stage1) and anaconda (old stage2).

Are stage1/stage2 changes to test cases included in this ticket, or should I track this as a separate effort?

comment:6 in reply to: ↑ 5 Changed 3 years ago by rhe

  • Summary changed from Obsoleting the tests for verifying http/nfs/ftp/hard disk install sources to Improve the install sources tests to adapt F-15 anaconda

Replying to jlaska:

Note, while testing, hansg reminded us that many of our tests refer to stage1 and stage2. With Fedora 15, the previous distinction between stage1 and stage2 no longer applies. There is still a difference, but the difference now is between loader (old stage1) and anaconda (old stage2).

Are stage1/stage2 changes to test cases included in this ticket, or should I track this as a separate effort?

Never mind, this is already in part of the plan, I've also renamed the ticket, thanks for reminding.:)

comment:7 Changed 3 years ago by rhe

I've updated the tests based on the table in comment:4 by either creating uncovered new tests or moving/renaming some install source tests to install repo ones with rewordings and adjustments. Now all the tests have been integrated into the install repository category sub-categoried by DVD, HTTP/FTP, Mirrorlist, NFS, NFSISO and HardDrive?. In each subcategory, there are generally three cases: default scenario, additional variation, and Graphical repo dialog. Feel free to review and feedback is welcomed! Thanks.

comment:8 follow-up: ↓ 9 Changed 3 years ago by jlaska

Nice work indeed! Some minor comments ...

  • When you say default you mean the default method for testing that particular function, not the installers default method for handling it?

With the modified tests, and the boot method tests ... do we have the following scenarios covered for each media?

  1. boot media and accept defaults (different behavior based on media used)
  2. boot media with askmethod, and choose appropriate installation source
  3. boot media with appropriate repo= boot parameter
  4. boot media and graphically choose installation source

comment:9 in reply to: ↑ 8 Changed 3 years ago by rhe

Replying to jlaska:

Nice work indeed! Some minor comments ...

  • When you say default you mean the default method for testing that particular function, not the installers default method for handling it?

I mean the default method for testing that function. The 'default', 'variation' was named according to the table in comment:3

With the modified tests, and the boot method tests ... do we have the following scenarios covered for each media?

  1. boot media and accept defaults (different behavior based on media used)

Boot media tests have covered all the media, so I prefer not to modify them a lot but will reword the descriptions about anaconda stage#1&#2.

For install repo tests, the followings are the default install repo cases for each media:

  1. boot media with askmethod, and choose appropriate installation source
  2. boot media with appropriate repo= boot parameter

Now there are two tests which are 'askmethod' and 'repo=' for every method: HTTP/FTP, Harddrive, NFS, NFSISO and DVD(askmethod and repo= are contained in one test for DVD). IMO, they don't depend on any media type and should also be put in the Install Variations & General Tests. What do you think?

  1. boot media and graphically choose installation source

These tests have been integrated to install repository category, expect NFS ISO and Harddrive which are not supported. Previously, graphical tests were put into DVD media tests to verify the network enabling ability at that step. For fedora-16, do you think if these graphical tests should be put to Install Variations & General Tests instead?

In addition, These repos can also be specified in kickstart, is it an alternative way to test 'repo=' method?

comment:12 Changed 3 years ago by rhe

  • Resolution set to fixed
  • Status changed from new to closed

As discussed above, I've improved the install source cases and re-categorized them to Category:Installation_Repository. I also updated Category:Installer_Boot_Methods. I think the previous requirements have been met. Close this ticket for now, but feel free to comment and reopen it.

Note: See TracTickets for help on using tickets.