Ticket #361 (closed task: wontfix)

Opened 3 years ago

Last modified 2 years ago

improve pretty log output for anaconda tests

Reported by: jlaska Owned by:
Priority: minor Milestone: 0.8.0
Component: tests Keywords:
Cc: Blocked By:
Blocking:

Description (last modified by jlaska) (diff)

With the advent of the super awesome html pretty logs, some of the less frequented tests are in need of some care and feeding. While reviewing installer test results with the anaconda team, I noticed that the installer tests don't all look good with pretty logs.

Sample output for existing anaconda test results ...

This ticket is intended to address improving the html result output for the above installer tests.

Change History

comment:1 Changed 3 years ago by jlaska

  • Description modified (diff)

comment:2 Changed 3 years ago by jlaska

For the anaconda_storage test, what I'd probably want to see is the info from the DEBUG output in the html report. Perhaps something like the following ...

Summary

Tests Passed:  10
Tests Failed:  6
Suites Passed: 7
Suites Failed: 6

Highlights

===== Storage Test Results =====
Suite autopart-freespace:  PASSED
Suite ext4-on-partition:  PASSED
Suite raid0:  FAILED
Suite lvm-on-raid0:  FAILED
Suite ext2-on-partition:  PASSED
Suite raid5:  FAILED
Suite raid1:  FAILED
Suite autopart:  PASSED
Suite conflicting-lvm-autopart:  FAILED
Suite raid6:  FAILED
Suite ext3-on-partition:  PASSED
Suite encrypted-autopart:  PASSED
Suite autopart-preexisting-autopart:  PASSED

Details

===== Running Test Suite ext2-on-partition =====
----- Creating Disk Images -----
Formatting '/usr/share/autotest/tmp/tmpwdPq34_anaconda_storage/images/ext2-on-partition-p2QUJL/tmprW7JI0', fmt=qcow2 size=8589934592 encryption=off cluster_size=0 
----- Creating Test Case 1-FSOnPartition -----
I: -input-charset not specified, using utf-8 (detected in locale settings)
Using AUTOP000.PY;1 for  ./cases/AutopartFreespace.py (Autopart.py)
Using AUTOP000.PYO;1 for  ./cases/AutopartFreespace.pyo (Autopart.pyo)
Using AUTOP000.PYC;1 for  ./cases/Autopart.pyc (AutopartFreespace.pyc)
Using AUTOP000.PYC;1 for  ./suites/Autopart.pyc (AutopartPreexistAutopart.pyc)
Using AUTOP000.PYO;1 for  ./suites/AutopartPreexistAutopart.pyo (AutopartFreespace.pyo)
Using AUTOP000.PY;1 for  ./suites/AutopartFreespace.py (AutopartPreexistAutopart.py)
Using AUTOP001.PY;1 for  ./suites/AutopartPreexistAutopart.py (Autopart.py)
Using AUTOP001.PYO;1 for  ./suites/AutopartFreespace.pyo (Autopart.pyo)
Using AUTOP001.PYC;1 for  ./suites/AutopartPreexistAutopart.pyc (AutopartFreespace.pyc)
Total translation table size: 0
Total rockridge attributes bytes: 8415
Total directory bytes: 16516
Path table size(bytes): 38
Max brk space used 1f000
----- Running Test Case 1-FSOnPartition -----
331 extents written (0 MB)
Test ext2-on-partition/1-FSOnPartition: PASSED
===== Running Test Suite ext4-on-partition =====
----- Creating Disk Images -----
Formatting '/usr/share/autotest/tmp/tmpwdPq34_anaconda_storage/images/ext4-on-partition-DOgLB2/tmpNXNuYY', fmt=qcow2 size=8589934592 encryption=off cluster_size=0 
----- Creating Test Case 1-FSOnPartition -----
I: -input-charset not specified, using utf-8 (detected in locale settings)
Using AUTOP000.PY;1 for  ./cases/AutopartFreespace.py (Autopart.py)
Using AUTOP000.PYO;1 for  ./cases/AutopartFreespace.pyo (Autopart.pyo)
Using AUTOP000.PYC;1 for  ./cases/Autopart.pyc (AutopartFreespace.pyc)
Using AUTOP000.PYC;1 for  ./suites/Autopart.pyc (AutopartPreexistAutopart.pyc)
Using AUTOP000.PYO;1 for  ./suites/AutopartPreexistAutopart.pyo (AutopartFreespace.pyo)
Using AUTOP000.PY;1 for  ./suites/AutopartFreespace.py (AutopartPreexistAutopart.py)
Using AUTOP001.PY;1 for  ./suites/AutopartPreexistAutopart.py (Autopart.py)
Using AUTOP001.PYO;1 for  ./suites/AutopartFreespace.pyo (Autopart.pyo)
Using AUTOP001.PYC;1 for  ./suites/AutopartPreexistAutopart.pyc (AutopartFreespace.pyc)
Total translation table size: 0
Total rockridge attributes bytes: 8415
Total directory bytes: 16516
Path table size(bytes): 38
Max brk space used 1f000
----- Running Test Case 1-FSOnPartition -----
331 extents written (0 MB)
Test ext4-on-partition/1-FSOnPartition: PASSED
===== Running Test Suite autopart-freespace =====
----- Creating Disk Images -----
Formatting '/usr/share/autotest/tmp/tmpwdPq34_anaconda_storage/images/autopart-freespace-c6YUhf/tmpa6Z_a0', fmt=qcow2 size=8589934592 encryption=off cluster_size=0 
----- Creating Test Case 1-AutopartFreespace -----
I: -input-charset not specified, using utf-8 (detected in locale settings)
Using AUTOP000.PY;1 for  ./cases/AutopartFreespace.py (Autopart.py)
Using AUTOP000.PYO;1 for  ./cases/AutopartFreespace.pyo (Autopart.pyo)
Using AUTOP000.PYC;1 for  ./cases/Autopart.pyc (AutopartFreespace.pyc)
Using AUTOP000.PYC;1 for  ./suites/Autopart.pyc (AutopartPreexistAutopart.pyc)
Using AUTOP000.PYO;1 for  ./suites/AutopartPreexistAutopart.pyo (AutopartFreespace.pyo)
Using AUTOP000.PY;1 for  ./suites/AutopartFreespace.py (AutopartPreexistAutopart.py)
Using AUTOP001.PY;1 for  ./suites/AutopartPreexistAutopart.py (Autopart.py)
Using AUTOP001.PYO;1 for  ./suites/AutopartFreespace.pyo (Autopart.pyo)
Using AUTOP001.PYC;1 for  ./suites/AutopartPreexistAutopart.pyc (AutopartFreespace.pyc)
Total translation table size: 0
Total rockridge attributes bytes: 8415
Total directory bytes: 16516
----- Running Test Case 1-AutopartFreespace -----
Path table size(bytes): 38
Max brk space used 1f000
331 extents written (0 MB)
Test autopart-freespace/1-AutopartFreespace: PASSED
===== Running Test Suite ext3-on-partition =====
----- Creating Disk Images -----
Formatting '/usr/share/autotest/tmp/tmpwdPq34_anaconda_storage/images/ext3-on-partition-VRPO5x/tmppfYdKp', fmt=qcow2 size=8589934592 encryption=off cluster_size=0 
----- Creating Test Case 1-FSOnPartition -----
I: -input-charset not specified, using utf-8 (detected in locale settings)
Using AUTOP000.PY;1 for  ./cases/AutopartFreespace.py (Autopart.py)
Using AUTOP000.PYO;1 for  ./cases/AutopartFreespace.pyo (Autopart.pyo)
Using AUTOP000.PYC;1 for  ./cases/Autopart.pyc (AutopartFreespace.pyc)
Using AUTOP000.PYC;1 for  ./suites/Autopart.pyc (AutopartPreexistAutopart.pyc)
Using AUTOP000.PYO;1 for  ./suites/AutopartPreexistAutopart.pyo (AutopartFreespace.pyo)
Using AUTOP000.PY;1 for  ./suites/AutopartFreespace.py (AutopartPreexistAutopart.py)
Using AUTOP001.PY;1 for  ./suites/AutopartPreexistAutopart.py (Autopart.py)
Using AUTOP001.PYO;1 for  ./suites/AutopartFreespace.pyo (Autopart.pyo)
Using AUTOP001.PYC;1 for  ./suites/AutopartPreexistAutopart.pyc (AutopartFreespace.pyc)
Total translation table size: 0
Total rockridge attributes bytes: 8415
Total directory bytes: 16516
Path table size(bytes): 38
----- Running Test Case 1-FSOnPartition -----
Max brk space used 1f000
331 extents written (0 MB)
Test ext3-on-partition/1-FSOnPartition: PASSED
===== Running Test Suite raid5 =====
----- Creating Disk Images -----
Formatting '/usr/share/autotest/tmp/tmpwdPq34_anaconda_storage/images/raid5-3QKkfy/tmpgApV1E', fmt=qcow2 size=10737418240 encryption=off cluster_size=0 
----- Creating Test Case 1-RAID5 -----
I: -input-charset not specified, using utf-8 (detected in locale settings)
Using AUTOP000.PY;1 for  ./cases/AutopartFreespace.py (Autopart.py)
Using AUTOP000.PYO;1 for  ./cases/AutopartFreespace.pyo (Autopart.pyo)
Using AUTOP000.PYC;1 for  ./cases/Autopart.pyc (AutopartFreespace.pyc)
Using AUTOP000.PYC;1 for  ./suites/Autopart.pyc (AutopartPreexistAutopart.pyc)
Using AUTOP000.PYO;1 for  ./suites/AutopartPreexistAutopart.pyo (AutopartFreespace.pyo)
Using AUTOP000.PY;1 for  ./suites/AutopartFreespace.py (AutopartPreexistAutopart.py)
Using AUTOP001.PY;1 for  ./suites/AutopartPreexistAutopart.py (Autopart.py)
Using AUTOP001.PYO;1 for  ./suites/AutopartFreespace.pyo (Autopart.pyo)
Using AUTOP001.PYC;1 for  ./suites/AutopartPreexistAutopart.pyc (AutopartFreespace.pyc)
Total translation table size: 0
Total rockridge attributes bytes: 8415
Total directory bytes: 16516
----- Running Test Case 1-RAID5 -----
Path table size(bytes): 38
Max brk space used 1f000
331 extents written (0 MB)
Test raid5/1-RAID5: FAILED
===== Running Test Suite encrypted-autopart =====
----- Creating Disk Images -----
Formatting '/usr/share/autotest/tmp/tmpwdPq34_anaconda_storage/images/encrypted-autopart-kUZqj3/tmpUDu2sX', fmt=qcow2 size=8589934592 encryption=off cluster_size=0 
----- Creating Test Case 1-EncryptedAutopart -----
I: -input-charset not specified, using utf-8 (detected in locale settings)
Using AUTOP000.PY;1 for  ./cases/AutopartFreespace.py (Autopart.py)
Using AUTOP000.PYO;1 for  ./cases/AutopartFreespace.pyo (Autopart.pyo)
Using AUTOP000.PYC;1 for  ./cases/Autopart.pyc (AutopartFreespace.pyc)
Using AUTOP000.PYC;1 for  ./suites/Autopart.pyc (AutopartPreexistAutopart.pyc)
Using AUTOP000.PYO;1 for  ./suites/AutopartPreexistAutopart.pyo (AutopartFreespace.pyo)
Using AUTOP000.PY;1 for  ./suites/AutopartFreespace.py (AutopartPreexistAutopart.py)
Using AUTOP001.PY;1 for  ./suites/AutopartPreexistAutopart.py (Autopart.py)
Using AUTOP001.PYO;1 for  ./suites/AutopartFreespace.pyo (Autopart.pyo)
Using AUTOP001.PYC;1 for  ./suites/AutopartPreexistAutopart.pyc (AutopartFreespace.pyc)
Total translation table size: 0
Total rockridge attributes bytes: 8415
Total directory bytes: 16516
Path table size(bytes): 38
Max brk space used 1f000
----- Running Test Case 1-EncryptedAutopart -----
331 extents written (0 MB)
Test encrypted-autopart/1-EncryptedAutopart: PASSED
===== Running Test Suite raid1 =====
----- Creating Disk Images -----
Formatting '/usr/share/autotest/tmp/tmpwdPq34_anaconda_storage/images/raid1-PW0jTd/tmpgzAx8t', fmt=qcow2 size=8589934592 encryption=off cluster_size=0 
----- Creating Test Case 1-RAID1 -----
I: -input-charset not specified, using utf-8 (detected in locale settings)
Using AUTOP000.PY;1 for  ./cases/AutopartFreespace.py (Autopart.py)
Using AUTOP000.PYO;1 for  ./cases/AutopartFreespace.pyo (Autopart.pyo)
Using AUTOP000.PYC;1 for  ./cases/Autopart.pyc (AutopartFreespace.pyc)
Using AUTOP000.PYC;1 for  ./suites/Autopart.pyc (AutopartPreexistAutopart.pyc)
Using AUTOP000.PYO;1 for  ./suites/AutopartPreexistAutopart.pyo (AutopartFreespace.pyo)
Using AUTOP000.PY;1 for  ./suites/AutopartFreespace.py (AutopartPreexistAutopart.py)
Using AUTOP001.PY;1 for  ./suites/AutopartPreexistAutopart.py (Autopart.py)
Using AUTOP001.PYO;1 for  ./suites/AutopartFreespace.pyo (Autopart.pyo)
Using AUTOP001.PYC;1 for  ./suites/AutopartPreexistAutopart.pyc (AutopartFreespace.pyc)
Total translation table size: 0
Total rockridge attributes bytes: 8415
Total directory bytes: 16516
----- Running Test Case 1-RAID1 -----
Path table size(bytes): 38
Max brk space used 1f000
331 extents written (0 MB)
Test raid1/1-RAID1: FAILED
===== Running Test Suite autopart-preexisting-autopart =====
----- Creating Disk Images -----
Formatting '/usr/share/autotest/tmp/tmpwdPq34_anaconda_storage/images/autopart-preexisting-autopart-FH1_Ss/tmpxnFlF1', fmt=qcow2 size=8589934592 encryption=off cluster_size=0 
----- Creating Test Case 1-Autopart -----
I: -input-charset not specified, using utf-8 (detected in locale settings)
Using AUTOP000.PY;1 for  ./cases/AutopartFreespace.py (Autopart.py)
Using AUTOP000.PYO;1 for  ./cases/AutopartFreespace.pyo (Autopart.pyo)
Using AUTOP000.PYC;1 for  ./cases/Autopart.pyc (AutopartFreespace.pyc)
Using AUTOP000.PYC;1 for  ./suites/Autopart.pyc (AutopartPreexistAutopart.pyc)
Using AUTOP000.PYO;1 for  ./suites/AutopartPreexistAutopart.pyo (AutopartFreespace.pyo)
Using AUTOP000.PY;1 for  ./suites/AutopartFreespace.py (AutopartPreexistAutopart.py)
Using AUTOP001.PY;1 for  ./suites/AutopartPreexistAutopart.py (Autopart.py)
Using AUTOP001.PYO;1 for  ./suites/AutopartFreespace.pyo (Autopart.pyo)
Using AUTOP001.PYC;1 for  ./suites/AutopartPreexistAutopart.pyc (AutopartFreespace.pyc)
Total translation table size: 0
Total rockridge attributes bytes: 8415
Total directory bytes: 16516
Path table size(bytes): 38
----- Running Test Case 1-Autopart -----
Max brk space used 1f000
331 extents written (0 MB)
Test autopart-preexisting-autopart/1-Autopart: PASSED
----- Creating Test Case 2-Autopart -----
I: -input-charset not specified, using utf-8 (detected in locale settings)
Using AUTOP000.PY;1 for  ./cases/AutopartFreespace.py (Autopart.py)
Using AUTOP000.PYO;1 for  ./cases/AutopartFreespace.pyo (Autopart.pyo)
Using AUTOP000.PYC;1 for  ./cases/Autopart.pyc (AutopartFreespace.pyc)
Using AUTOP000.PYC;1 for  ./suites/Autopart.pyc (AutopartPreexistAutopart.pyc)
Using AUTOP000.PYO;1 for  ./suites/AutopartPreexistAutopart.pyo (AutopartFreespace.pyo)
Using AUTOP000.PY;1 for  ./suites/AutopartFreespace.py (AutopartPreexistAutopart.py)
Using AUTOP001.PY;1 for  ./suites/AutopartPreexistAutopart.py (Autopart.py)
Using AUTOP001.PYO;1 for  ./suites/AutopartFreespace.pyo (Autopart.pyo)
Using AUTOP001.PYC;1 for  ./suites/AutopartPreexistAutopart.pyc (AutopartFreespace.pyc)
Total translation table size: 0
Total rockridge attributes bytes: 8415
Total directory bytes: 16516
Path table size(bytes): 38
Max brk space used 1f000
331 extents written (0 MB)
----- Running Test Case 2-Autopart -----
Test autopart-preexisting-autopart/2-Autopart: PASSED
===== Running Test Suite no-swap-lvm =====
----- Creating Disk Images -----
Formatting '/usr/share/autotest/tmp/tmpwdPq34_anaconda_storage/images/no-swap-lvm-9N5PR3/tmp9ciub9', fmt=qcow2 size=8589934592 encryption=off cluster_size=0 
----- Creating Test Case 1-NoSwapLVM -----
I: -input-charset not specified, using utf-8 (detected in locale settings)
Using AUTOP000.PY;1 for  ./cases/AutopartFreespace.py (Autopart.py)
Using AUTOP000.PYO;1 for  ./cases/AutopartFreespace.pyo (Autopart.pyo)
Using AUTOP000.PYC;1 for  ./cases/Autopart.pyc (AutopartFreespace.pyc)
Using AUTOP000.PYC;1 for  ./suites/Autopart.pyc (AutopartPreexistAutopart.pyc)
Using AUTOP000.PYO;1 for  ./suites/AutopartPreexistAutopart.pyo (AutopartFreespace.pyo)
Using AUTOP000.PY;1 for  ./suites/AutopartFreespace.py (AutopartPreexistAutopart.py)
Using AUTOP001.PY;1 for  ./suites/AutopartPreexistAutopart.py (Autopart.py)
Using AUTOP001.PYO;1 for  ./suites/AutopartFreespace.pyo (Autopart.pyo)
Using AUTOP001.PYC;1 for  ./suites/AutopartPreexistAutopart.pyc (AutopartFreespace.pyc)
Total translation table size: 0
Total rockridge attributes bytes: 8415
Total directory bytes: 16516
Path table size(bytes): 38
----- Running Test Case 1-NoSwapLVM -----
Max brk space used 1f000
331 extents written (0 MB)
===== Running Test Suite conflicting-lvm-autopart =====
----- Creating Disk Images -----
Formatting '/usr/share/autotest/tmp/tmpwdPq34_anaconda_storage/images/conflicting-lvm-autopart-HwiXPf/tmpro80jK', fmt=qcow2 size=8589934592 encryption=off cluster_size=0 
Formatting '/usr/share/autotest/tmp/tmpwdPq34_anaconda_storage/images/conflicting-lvm-autopart-HwiXPf/tmphsKhIa', fmt=qcow2 size=8589934592 encryption=off cluster_size=0 
----- Creating Test Case 1-Autopart -----
I: -input-charset not specified, using utf-8 (detected in locale settings)
Using AUTOP000.PY;1 for  ./cases/AutopartFreespace.py (Autopart.py)
Using AUTOP000.PYO;1 for  ./cases/AutopartFreespace.pyo (Autopart.pyo)
Using AUTOP000.PYC;1 for  ./cases/Autopart.pyc (AutopartFreespace.pyc)
Using AUTOP000.PYC;1 for  ./suites/Autopart.pyc (AutopartPreexistAutopart.pyc)
Using AUTOP000.PYO;1 for  ./suites/AutopartPreexistAutopart.pyo (AutopartFreespace.pyo)
Using AUTOP000.PY;1 for  ./suites/AutopartFreespace.py (AutopartPreexistAutopart.py)
Using AUTOP001.PY;1 for  ./suites/AutopartPreexistAutopart.py (Autopart.py)
Using AUTOP001.PYO;1 for  ./suites/AutopartFreespace.pyo (Autopart.pyo)
Using AUTOP001.PYC;1 for  ./suites/AutopartPreexistAutopart.pyc (AutopartFreespace.pyc)
Total translation table size: 0
Total rockridge attributes bytes: 8415
Total directory bytes: 16516
----- Running Test Case 1-Autopart -----
Path table size(bytes): 38
Max brk space used 1f000
331 extents written (0 MB)
Test conflicting-lvm-autopart/1-Autopart: PASSED
----- Creating Test Case 2-Autopart -----
I: -input-charset not specified, using utf-8 (detected in locale settings)
Using AUTOP000.PY;1 for  ./cases/AutopartFreespace.py (Autopart.py)
Using AUTOP000.PYO;1 for  ./cases/AutopartFreespace.pyo (Autopart.pyo)
Using AUTOP000.PYC;1 for  ./cases/Autopart.pyc (AutopartFreespace.pyc)
Using AUTOP000.PYC;1 for  ./suites/Autopart.pyc (AutopartPreexistAutopart.pyc)
Using AUTOP000.PYO;1 for  ./suites/AutopartPreexistAutopart.pyo (AutopartFreespace.pyo)
Using AUTOP000.PY;1 for  ./suites/AutopartFreespace.py (AutopartPreexistAutopart.py)
Using AUTOP001.PY;1 for  ./suites/AutopartPreexistAutopart.py (Autopart.py)
Using AUTOP001.PYO;1 for  ./suites/AutopartFreespace.pyo (Autopart.pyo)
Using AUTOP001.PYC;1 for  ./suites/AutopartPreexistAutopart.pyc (AutopartFreespace.pyc)
Total translation table size: 0
Total rockridge attributes bytes: 8415
Total directory bytes: 16516
Path table size(bytes): 38
Max brk space used 1f000
331 extents written (0 MB)
----- Running Test Case 2-Autopart -----
Test conflicting-lvm-autopart/2-Autopart: PASSED
----- Creating Test Case 3-Autopart -----
I: -input-charset not specified, using utf-8 (detected in locale settings)
Using AUTOP000.PY;1 for  ./cases/AutopartFreespace.py (Autopart.py)
Using AUTOP000.PYO;1 for  ./cases/AutopartFreespace.pyo (Autopart.pyo)
Using AUTOP000.PYC;1 for  ./cases/Autopart.pyc (AutopartFreespace.pyc)
Using AUTOP000.PYC;1 for  ./suites/Autopart.pyc (AutopartPreexistAutopart.pyc)
Using AUTOP000.PYO;1 for  ./suites/AutopartPreexistAutopart.pyo (AutopartFreespace.pyo)
Using AUTOP000.PY;1 for  ./suites/AutopartFreespace.py (AutopartPreexistAutopart.py)
Using AUTOP001.PY;1 for  ./suites/AutopartPreexistAutopart.py (Autopart.py)
Using AUTOP001.PYO;1 for  ./suites/AutopartFreespace.pyo (Autopart.pyo)
Using AUTOP001.PYC;1 for  ./suites/AutopartPreexistAutopart.pyc (AutopartFreespace.pyc)
Total translation table size: 0
Total rockridge attributes bytes: 8415
Total directory bytes: 16516
Path table size(bytes): 38
Max brk space used 1f000
331 extents written (0 MB)
----- Running Test Case 3-Autopart -----
Timer expired, killing test suite conflicting-lvm-autopart
===== Running Test Suite autopart =====
----- Creating Disk Images -----
Formatting '/usr/share/autotest/tmp/tmpwdPq34_anaconda_storage/images/autopart-H6vaIB/tmpBhiA00', fmt=qcow2 size=8589934592 encryption=off cluster_size=0 
----- Creating Test Case 1-Autopart -----
I: -input-charset not specified, using utf-8 (detected in locale settings)
Using AUTOP000.PY;1 for  ./cases/AutopartFreespace.py (Autopart.py)
Using AUTOP000.PYO;1 for  ./cases/AutopartFreespace.pyo (Autopart.pyo)
Using AUTOP000.PYC;1 for  ./cases/Autopart.pyc (AutopartFreespace.pyc)
Using AUTOP000.PYC;1 for  ./suites/Autopart.pyc (AutopartPreexistAutopart.pyc)
Using AUTOP000.PYO;1 for  ./suites/AutopartPreexistAutopart.pyo (AutopartFreespace.pyo)
Using AUTOP000.PY;1 for  ./suites/AutopartFreespace.py (AutopartPreexistAutopart.py)
Using AUTOP001.PY;1 for  ./suites/AutopartPreexistAutopart.py (Autopart.py)
Using AUTOP001.PYO;1 for  ./suites/AutopartFreespace.pyo (Autopart.pyo)
Using AUTOP001.PYC;1 for  ./suites/AutopartPreexistAutopart.pyc (AutopartFreespace.pyc)
Total translation table size: 0
Total rockridge attributes bytes: 8415
Total directory bytes: 16516
Path table size(bytes): 38
Max brk space used 1f000
----- Running Test Case 1-Autopart -----
331 extents written (0 MB)
Test autopart/1-Autopart: PASSED
===== Running Test Suite raid0 =====
----- Creating Disk Images -----
Formatting '/usr/share/autotest/tmp/tmpwdPq34_anaconda_storage/images/raid0-bSSJ1g/tmpXOVcUq', fmt=qcow2 size=8589934592 encryption=off cluster_size=0 
----- Creating Test Case 1-RAID0 -----
I: -input-charset not specified, using utf-8 (detected in locale settings)
Using AUTOP000.PY;1 for  ./cases/AutopartFreespace.py (Autopart.py)
Using AUTOP000.PYO;1 for  ./cases/AutopartFreespace.pyo (Autopart.pyo)
Using AUTOP000.PYC;1 for  ./cases/Autopart.pyc (AutopartFreespace.pyc)
Using AUTOP000.PYC;1 for  ./suites/Autopart.pyc (AutopartPreexistAutopart.pyc)
Using AUTOP000.PYO;1 for  ./suites/AutopartPreexistAutopart.pyo (AutopartFreespace.pyo)
Using AUTOP000.PY;1 for  ./suites/AutopartFreespace.py (AutopartPreexistAutopart.py)
Using AUTOP001.PY;1 for  ./suites/AutopartPreexistAutopart.py (Autopart.py)
Using AUTOP001.PYO;1 for  ./suites/AutopartFreespace.pyo (Autopart.pyo)
Using AUTOP001.PYC;1 for  ./suites/AutopartPreexistAutopart.pyc (AutopartFreespace.pyc)
Total translation table size: 0
Total rockridge attributes bytes: 8415
Total directory bytes: 16516
Path table size(bytes): 38
----- Running Test Case 1-RAID0 -----
Max brk space used 1f000
331 extents written (0 MB)
Test raid0/1-RAID0: FAILED
===== Running Test Suite raid6 =====
----- Creating Disk Images -----
Formatting '/usr/share/autotest/tmp/tmpwdPq34_anaconda_storage/images/raid6-4V4TWX/tmpi96sEP', fmt=qcow2 size=10737418240 encryption=off cluster_size=0 
----- Creating Test Case 1-RAID6 -----
I: -input-charset not specified, using utf-8 (detected in locale settings)
Using AUTOP000.PY;1 for  ./cases/AutopartFreespace.py (Autopart.py)
Using AUTOP000.PYO;1 for  ./cases/AutopartFreespace.pyo (Autopart.pyo)
Using AUTOP000.PYC;1 for  ./cases/Autopart.pyc (AutopartFreespace.pyc)
Using AUTOP000.PYC;1 for  ./suites/Autopart.pyc (AutopartPreexistAutopart.pyc)
Using AUTOP000.PYO;1 for  ./suites/AutopartPreexistAutopart.pyo (AutopartFreespace.pyo)
Using AUTOP000.PY;1 for  ./suites/AutopartFreespace.py (AutopartPreexistAutopart.py)
Using AUTOP001.PY;1 for  ./suites/AutopartPreexistAutopart.py (Autopart.py)
Using AUTOP001.PYO;1 for  ./suites/AutopartFreespace.pyo (Autopart.pyo)
Using AUTOP001.PYC;1 for  ./suites/AutopartPreexistAutopart.pyc (AutopartFreespace.pyc)
Total translation table size: 0
Total rockridge attributes bytes: 8415
Total directory bytes: 16516
Path table size(bytes): 38
Max brk space used 1f000
----- Running Test Case 1-RAID6 -----
331 extents written (0 MB)
Test raid6/1-RAID6: FAILED
===== Running Test Suite lvm-on-raid0 =====
----- Creating Disk Images -----
Formatting '/usr/share/autotest/tmp/tmpwdPq34_anaconda_storage/images/lvm-on-raid0-NiepA8/tmpw4Q5yC', fmt=qcow2 size=8589934592 encryption=off cluster_size=0 
----- Creating Test Case 1-LVMonRAID0 -----
I: -input-charset not specified, using utf-8 (detected in locale settings)
Using AUTOP000.PY;1 for  ./cases/AutopartFreespace.py (Autopart.py)
Using AUTOP000.PYO;1 for  ./cases/AutopartFreespace.pyo (Autopart.pyo)
Using AUTOP000.PYC;1 for  ./cases/Autopart.pyc (AutopartFreespace.pyc)
Using AUTOP000.PYC;1 for  ./suites/Autopart.pyc (AutopartPreexistAutopart.pyc)
Using AUTOP000.PYO;1 for  ./suites/AutopartPreexistAutopart.pyo (AutopartFreespace.pyo)
Using AUTOP000.PY;1 for  ./suites/AutopartFreespace.py (AutopartPreexistAutopart.py)
Using AUTOP001.PY;1 for  ./suites/AutopartPreexistAutopart.py (Autopart.py)
Using AUTOP001.PYO;1 for  ./suites/AutopartFreespace.pyo (Autopart.pyo)
Using AUTOP001.PYC;1 for  ./suites/AutopartPreexistAutopart.pyc (AutopartFreespace.pyc)
Total translation table size: 0
Total rockridge attributes bytes: 8415
Total directory bytes: 16516
Path table size(bytes): 38
Max brk space used 1f000
----- Running Test Case 1-LVMonRAID0 -----
331 extents written (0 MB)
Test lvm-on-raid0/1-LVMonRAID0: FAILED

comment:3 Changed 3 years ago by jlaska

For the compose_tree test, what I'd probably want to see is the true result. For example, the test run linked above actually failed due to a traceback (which is a valid bug). I'm not sure why that information isn't in the log at all. The test script may not be properly capturing and returning this failure from mock.

DEBUG util.py:250:  Traceback (most recent call last):
DEBUG util.py:250:    File "/usr/bin/pungi", line 222, in <module>
DEBUG util.py:250:      main()
DEBUG util.py:250:    File "/usr/bin/pungi", line 120, in main
DEBUG util.py:250:      mypungi.doCreaterepo()
DEBUG util.py:250:    File "/usr/lib/python2.7/site-packages/pypungi/__init__.py", line 807, in doCreaterepo
DEBUG util.py:250:      self._makeMetadata(self.topdir, cachedir, compsfile, repoview=True, repoviewtitle=repoviewtitle)
DEBUG util.py:250:    File "/usr/lib/python2.7/site-packages/pypungi/__init__.py", line 764, in _makeMetadata
DEBUG util.py:250:      repomatic.doPkgMetadata()
DEBUG util.py:250:    File "/usr/lib/python2.7/site-packages/createrepo/__init__.py", line 383, in doPkgMetadata
DEBUG util.py:250:      self._setup_old_metadata_lookup()
DEBUG util.py:250:    File "/usr/lib/python2.7/site-packages/createrepo/__init__.py", line 369, in _setup_old_metadata_lookup
DEBUG util.py:250:      self.oldData = readMetadata.MetadataIndex(old_repo_path, opts)
DEBUG util.py:250:    File "/usr/lib/python2.7/site-packages/createrepo/readMetadata.py", line 57, in __init__
DEBUG util.py:250:      self.scan()
DEBUG util.py:250:    File "/usr/lib/python2.7/site-packages/createrepo/readMetadata.py", line 64, in scan
DEBUG util.py:250:      self._repo.sack.populate(self._repo, 'all', None, False)
DEBUG util.py:250:    File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 164, in populate
DEBUG util.py:250:      if self._check_db_version(repo, mydbtype):
DEBUG util.py:250:    File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 222, in _check_db_version
DEBUG util.py:250:      return repo._check_db_version(mdtype)
DEBUG util.py:250:    File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 1264, in _check_db_version
DEBUG util.py:250:      repoXML = self.repoXML
DEBUG util.py:250:    File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 1463, in <lambda>
DEBUG util.py:250:      repoXML = property(fget=lambda self: self._getRepoXML(),
DEBUG util.py:250:    File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 1459, in _getRepoXML
DEBUG util.py:250:      raise Errors.RepoError, msg
DEBUG util.py:250:  yum.Errors.RepoError: Cannot retrieve repository metadata (repomd.xml) for repository: garbageid. Please verify its path and try again

As for the Summary, I think all we need to know is whether the images were built successfully.

For Details, I hesitate to suggest adding the *full* test script output ... but that might be an option.

comment:4 follow-up: ↓ 5 Changed 3 years ago by kparal

When the new html log architecture was introduced, I re-wrote all our tests, but I didn't re-write anaconda tests. I just made sure they still execute. I supposed that anaconda developers would rewrite them themselves, if they hadn't been satisfied with reading the full log or debug log.

So my question is simple - do we want to maintain these tests, or just provide consulting to anaconda developers and ask them to adjust it? I see following challenges for the first approach:

  1. We have no insight in the tests. Now that James provided expected output, it's much easier to decide what should go into the html log. But still if we encounter any obstacle (something crashing, working weird, etc) we are not able to work on it.
  1. These tests require bare-metal machine to execute and we don't have it. When I had to work on them in the past, I needed to ask James to reserve a machine for me (he probably used one of our autoqa client machines). Not convenient.
  1. We could spend the time better by working on our internal code (library, tests).

I don't object adjusting the current output as suggested above, this time. But I am interested in our approach in the future. Let's face it, support for third-party tests will take a long time until it's implemented. And managing tests we don't really understand (welcome, depcheck example) can be really time-consuming. Do we want to take that route?

comment:5 in reply to: ↑ 4 Changed 3 years ago by jlaska

Replying to kparal:

So my question is simple - do we want to maintain these tests, or just provide consulting to anaconda developers and ask them to adjust it? I see following challenges for the first approach:

Simple answer ... yes! But it's never that simple, is it :)

We'll continue to make an exception for these anaconda tests, they provide important results that immediately translate into code fixes. My idea is that autoqa-devel@ would be responsible for maintaining the test wrapper layer and anaconda-devel@ will continue to maintain the tests themselves. I'll need to confirm with clumens, but I think this is a fair division, and places the expertise in the right area. Also, this aligns with how I'd expect this to work when we have a more *official* third-party test workflow.

  1. We have no insight in the tests. Now that James provided expected output, it's much easier to decide what should go into the html log. But still if we encounter any obstacle (something crashing, working weird, etc) we are not able to work on it.

Sure, that's understandable. I have insight into the tests, as does clumens. It's unreasonable to expect one person to know everything about all tests.

  1. These tests require bare-metal machine to execute and we don't have it. When I had to work on them in the past, I needed to ask James to reserve a machine for me (he probably used one of our autoqa client machines). Not convenient.

This is always a challenge ... when I get around to installing the staging server, I understand this will help alleviate some of the problems. Also, only anaconda_storage requires bare-metal.

  1. We could spend the time better by working on our internal code (library, tests).

We could, but we can also make an exception here :) This is an important test suite that needs to be run, and it's also forcing us to think about 3rd party test support. I'm not worried that this is spreading the group too thin.

I don't object adjusting the current output as suggested above, this time. But I am interested in our approach in the future. Let's face it, support for third-party tests will take a long time until it's implemented. And managing tests we don't really understand (welcome, depcheck example) can be really time-consuming. Do we want to take that route?

Yes for now and in the future. We need to be running these tests.

The approach you took when updating tests for pretty_log support was fine, I was happy with it then, and still think it was the right approach. When any test/library changes occur, I'd expect autoqa-devel@ to be responsible for ensuring that these tests at *least* run (no horrible autoqa-caused failures). Anything else is best-effort (aka nice-to-have).

Outside of that, I'd expect me or clumens to raise concerns and ask for guidance for resolving. For this ticket, before digging deep, I assumed it was a simple pretty_log fix. However, I suspect a bit more test wrapper change may be needed. Since I created + maintain the test wrappers, I'll update them. But let's use this ticket to tell me how to make the changes ... since I have no idea right now :)

Sound good?

comment:6 Changed 3 years ago by kparal

If the tests are important for us and if we go the best-effort way, I don't have any more concerns. Let's have it this way.

As for the asignee of this ticket, you don't have to step up, James, just because you wrote the test, that's not mandatory :-) We can use this ticket e.g. for Peter to gain some knowledge in output logging. As far as only AutoQA integration code is involved and you are able to answer prospective questions (that's very valuable), it should be doable also for him or anyone else. You can also dig deep into AutoQA logging of course if you want. In that case I'll check our documentation whether it contains everything necessary and link to it.

comment:7 Changed 3 years ago by jlaska

Thanks Kamil! I don't mind learning how to align stuff with the new logging structure at all, in fact, it'd be good for me to come up to speed there. I might take a crack at the anaconda_storage test, and maybe Petr can grab compose_tree?

comment:9 follow-up: ↓ 10 Changed 3 years ago by jlaska

I still need to look at making sure 'anaconda_storage' does the right thing. If we can resolve that quickly, let's move this over to 0.6.0?

comment:10 in reply to: ↑ 9 ; follow-up: ↓ 11 Changed 3 years ago by kparal

Replying to jlaska:

I still need to look at making sure 'anaconda_storage' does the right thing. If we can resolve that quickly, let's move this over to 0.6.0?

Definitely. If we can fix this up before 0.6 release, then it will be part of that release. If we can't, then it will be part of the next release :-) Or are you asking to hold on 0.6 release until this ticket is completed? I'll send some 0.6-planning email into autoqa-devel soon, we can discuss it there.

comment:11 in reply to: ↑ 10 Changed 3 years ago by jlaska

Replying to kparal:

Or are you asking to hold on 0.6 release until this ticket is completed? I'll send some 0.6-planning email into autoqa-devel soon, we can discuss it there.

No, I don't think I would hold the release for this change. Seems like a good nice-to-have candidate. Thanks!

comment:12 Changed 2 years ago by kparal

  • Status changed from new to closed
  • Resolution set to wontfix
  • Milestone changed from Finger Food to 0.8.0

We have temporarily removed anaconda tests from master. We want to provide infrastructure to allow real test authors to maintain the test, including their desired output. After that we should include the tests again.

I'm closing this ticket. Partly it has been fixed and partly it's obsolete.

Note: See TracTickets for help on using tickets.