Ticket #181 (closed task: fixed)

Opened 5 years ago

Last modified 5 years ago

Quick and dirty support for Virtualization

Reported by: kparal Owned by: kparal
Priority: major Milestone: Virtualization
Component: production Keywords:
Cc: Blocked By:


The task is to implement quick and dirty support for virtualization for AutoQA tests. We can already install autotest-client into virt machine, but there are some tests which must not run inside virt environment (tests using virt themselves for example). Therefore we must ensure all our tests are marked properly ("may use virt", "may not use virt"), all our tests clients are marked properly ("is virt", "is bare metal"), and this information are propagated from AutoQA to autotest-server and used properly when selecting appropriate host and scheduling jobs.

This could be quite easy, there is some support for tags in autotest already.

As an enhancement, we may think about preferences. Do we want to mark some test that it is preferred to run it in virt, but may also be run on bare metal? How do we achieve correct behavior for autotest-server for such task? For example: We may want to prefer executing sanity tests in virt env, but it's not really mandatory, just desired. Food for thoughts.

Change History

comment:1 follow-up: ↓ 2 Changed 5 years ago by kparal

Ah, and a good remark - when creating all the tags, we should clearly distinguish the tag "needs hardware with virt support" from tag "needs to be run inside virt system". The same goes for "this machine supports virt systems" and "this machine is a virt system". Those are really different things, so the tags should not be ambiguous. And we may need both.

comment:2 in reply to: ↑ 1 Changed 5 years ago by jlaska

Replying to kparal:

Ah, and a good remark - when creating all the tags, we should clearly distinguish the tag "needs hardware with virt support" from tag "needs to be run inside virt system". The same goes for "this machine supports virt systems" and "this machine is a virt system". Those are really different things, so the tags should not be ambiguous. And we may need both.

Good point. I read this as we have machine tags that indicate what the system is capable of, and then we'll have tests that require all (or a subset) of tags when selecting an applicable test client to use.

Right now we have the following tags (autotest calls them labels):

  • platform tags (architecture)
    • i386
    • x86_64
    • ppc
  • release tags (unused)
    • f11
    • f12
    • f13
  • hardware feature tags (unused)
    • kvm (aka vmx) - indicates VT support for virt guests
    • hypervisor - indicates if it's a virt system

comment:3 Changed 5 years ago by jlaska

I couldn't help but continue playing with this setup. Using the script provided in comment#1, along with 2 patches to dracut (thanks to haraldh for guidance), I have a working setup that uses virtualization and LVM snapshots to provide disposable test systems. The patches are out for review with dracut and can be tracked in bugzilla (see bug#602649 and bug#602784).

With this setup, you just reboot the test system when finished testing (this is the default behavior with autotest). During boot, the system creates and uses new LVM snapshot for the '/' volume on boot. Any changes previously made to the disk are gone and the system is restored to the initial state. If this works, and is accepted, I believe this obsoletes the need for ticket#183 and ticket#185.

Comments/questions/flame encouraged!

comment:4 Changed 5 years ago by jskladan

This looks awesome James! My only concern is this: are there any tests which require reboot during the testrun? (Are these tests even possible to write? this is maybe more of a "i'm curious about it" type of question, than a concern :) ) Because even thought this is a quick'n'dirty temporary solution, my experience tells me, that temporary solutions tend to last :-D

comment:5 Changed 5 years ago by kparal

Jskladan is right, we need to have the possibility to reboot the machine and continue executing the test sometimes. For example for package sanity test we may need to do "yum update" and reboot first if the machine is not fully up-to-date from regular maintenance (some updates were pushed in the meantime) and only then continue executing the test.

We might still use it for other tests, but I think some kind of signaling to the host machine will be necessary in the end.

comment:6 Changed 5 years ago by jlaska

Good thinking gents, keep the ideas coming. I'd like to poke holes in this solution to see how well it stands up and if it's good enough.

With regards to tests that require reboot... in the current dracut+LVM implementation, for any tests that need to reboot, we would just need to remove the boot argument rd_LVM_SNAPSHOT. That will allow dracut to continue using the existing LVM snapshot. When the test is complete, just add the rd_LVM_SNAPSHOT boot argument back to grub.conf.

As an aside, I'm sure it's just a matter of time for tests that require rebooting. However, I'm not sure how that works yet with autotest.

comment:7 Changed 5 years ago by kparal

  • Owner set to kparal

(This comment concerns the original ticket description.)

I'm in time press, so just shortly: I've played with autotest-server and it seems that "atest job create -b labels" behaves exactly like we need it - it looks for hosts that satisfy all the labels at once.

  -b LABELS, --labels=LABELS
                        Comma separated list of labels to get machine list

There are also some dependencies, I haven't still managed to explore what they are good for:

                        Comma separated list of labels this job is dependent

So, when I'm back from PTO I will patch autoqa to transfer required labels to autotest-server. We will also need some standards how individual labels are defined for particular tests. Also we should make a list of labels we use and document them on the wiki. I believe I can do it in a few days and then we will have support for unsafe tests to be executed inside VMs, isn't that cool? :) Looking forward for autotest-server testing instance, where we can try out all this stuff.

comment:8 Changed 5 years ago by kparal

  • Status changed from new to assigned

Initial (and hopefully working) code was pushed as 'labels' branch, read more at

I have documented the new labels,

comment:10 Changed 5 years ago by kparal

  • Resolution set to fixed
  • Status changed from assigned to closed

This has been implemented when my control.autoqa patchset was merged to master: http://git.fedorahosted.org/git/?p=autoqa.git;a=commit;h=d7f91c3e89f173b1b926454c1b47168426e014cd

We can now specify that test needs virt environment and correct machine is chosen.

Note: See TracTickets for help on using tickets.