#6173 Same volume-id on all F22 ISOs
Closed: Fixed 5 years ago Opened 8 years ago by zeenix.

I was informed that this is where I need for file these bugs so I'll be filing issues for each of the "distribution" bugs I've filed on Fedora bz. This one if against:

https://bugzilla.redhat.com/show_bug.cgi?id=1219851

While I was crying about same volume-id on full DVD and netinst (bug#1180269) in F21, it seems things have turned towards worse for me than better since all F22 ISOs share the same volume-id. While we now have means to differentiate ISOs based on size of the volume in libosinfo (and therefore Boxes/virt-manager), I can't already use that mechanism since the volume-size of final ISOs will most certainly be different than Beta ones.

Fortunately it seems the release is happening soon but I would have prefered to add support for F22 before its released, not after.


To be clear, what you are asking for is that, when generating the ISO container file from the composed tree, we should be producing a unique volume-id for each of the editions and spins, correct?

What format would you want this string to take?

This seems reasonable, but I'm afraid we've probably missed the boat for Fedora 22 here. I know it's not ideal, but as https://bugzilla.redhat.com/show_bug.cgi?id=1180269#c4 notes, it's not an easy fix.

I'm going to target this at F23. And again referring back to the not-quite-an-easy-fix situation, if you have patches for Anaconda (or can convince an Anaconda developer of the importance) that'd probably significantly accelerate things.

this request is way too late to do anything about for F22. Tooling chnages now risk us having to slip the release. It may not be something that is changeable. you do not provide enopugh information as to what you are seeing. for one the boot.iso and install DVD have to have the same ID. the ID's on livecds should already be different and nothing changed in them in F22 from F21. as to the install media, we have a boot,iso for each of the 3 products, but only a install DVD for Server. I honestly am not 100% sure what Issues you are seeing. So I for one would appreciate some extra information.

In F21, we had at least different volume-id on different variants, server, workstation and cloud. The https://bugzilla.redhat.com/show_bug.cgi?id=1180269#c4 is about netiso vs full installer DVD so while I'd understand if things had remained the same, I'm puzzled as to why this problem is expanding to all ISOs now. Even microsoft ensures that there is a unique volume-id on each of their ISO.

Replying to [comment:4 ausil]:

you do not provide enopugh information as to what you are seeing.

I guess a look into F21 media info in our OS database might help you understand:

https://git.fedorahosted.org/cgit/libosinfo.git/tree/data/oses/fedora.xml.in#n3925
https://git.fedorahosted.org/cgit/libosinfo.git/tree/data/oses/fedora.xml.in#n3950
https://git.fedorahosted.org/cgit/libosinfo.git/tree/data/oses/fedora.xml.in#n4017

Notice the different volume-id pattern for each of the ISO (except netiso and full installer)? That's what is missing in F22 ISOs.

Replying to [comment:5 zeenix]:

In F21, we had at least different volume-id on different variants, server, workstation and cloud. The https://bugzilla.redhat.com/show_bug.cgi?id=1180269#c4 is about netiso vs full installer DVD so while I'd understand if things had remained the same, I'm puzzled as to why this problem is expanding to all ISOs now. Even microsoft ensures that there is a unique volume-id on each of their ISO.

Workstation had no install isos in F21. they only had a live.

{{{
[ausil@compose-x86-01 ~]$ blkid /mnt/fedora_koji/compose/22_TC3/22_TC3//x86_64/iso/Fedora-iso
/mnt/fedora_koji/compose/22_TC3/22_TC3/Cloud_Atomic/x86_64/iso/Fedora-Cloud_Atomic-x86_64-22_TC3.iso: UUID="2015-05-08-02-24-09-00" LABEL="Fedora-22-x86_64" TYPE="iso9660" PTUUID="53968acb" PTTYPE="dos"
/mnt/fedora_koji/compose/22_TC3/22_TC3/Cloud/x86_64/iso/Fedora-Cloud-netinst-x86_64-22_TC3.iso: UUID="2015-05-08-07-11-56-00" LABEL="Fedora-22_T3-x86_64" TYPE="iso9660" PTUUID="2e553da7" PTTYPE="dos"
/mnt/fedora_koji/compose/22_TC3/22_TC3/Server/x86_64/iso/Fedora-Server-DVD-x86_64-22_TC3.iso: UUID="2015-05-08-06-59-49-00" LABEL="Fedora-22_T3-x86_64" TYPE="iso9660" PTUUID="11906394" PTTYPE="dos"
/mnt/fedora_koji/compose/22_TC3/22_TC3/Server/x86_64/iso/Fedora-Server-netinst-x86_64-22_TC3.iso: UUID="2015-05-08-06-59-12-00" LABEL="Fedora-22_T3-x86_64" TYPE="iso9660" PTUUID="36ef7148" PTTYPE="dos"
/mnt/fedora_koji/compose/22_TC3/22_TC3/Workstation/x86_64/iso/Fedora-Live-Workstation-x86_64-22-TC3.iso: UUID="2015-05-08-02-27-29-00" LABEL="Fedora-Live-WS-x86_64-22-T3" TYPE="iso9660" PTUUID="7ec4ab46" PTTYPE="dos"
/mnt/fedora_koji/compose/22_TC3/22_TC3/Workstation/x86_64/iso/Fedora-Workstation-netinst-x86_64-22_TC3.iso: UUID="2015-05-08-06-49-14-00" LABEL="Fedora-22_T3-x86_64" TYPE="iso9660" PTUUID="3bf61992" PTTYPE="dos"
[ausil@compose-x86-01 ~]$ blkid /mnt/fedora_koji/compose/22_TC3/22_TC3//x86_64/Fedora-iso
/mnt/fedora_koji/compose/22_TC3/22_TC3/Live/x86_64/Fedora-Live-KDE-x86_64-22-TC3.iso: UUID="2015-05-08-02-22-48-00" LABEL="Fedora-Live-KDE-x86_64-22-T3" TYPE="iso9660" PTUUID="6ae4427b" PTTYPE="dos"
/mnt/fedora_koji/compose/22_TC3/22_TC3/Live/x86_64/Fedora-Live-LXDE-x86_64-22-TC3.iso: UUID="2015-05-08-02-15-07-00" LABEL="Fedora-Live-LXDE-x86_64-22-T3" TYPE="iso9660" PTUUID="7bea4056" PTTYPE="dos"
/mnt/fedora_koji/compose/22_TC3/22_TC3/Live/x86_64/Fedora-Live-MATE_Compiz-x86_64-22-TC3.iso: UUID="2015-05-08-02-17-58-00" LABEL="Fedora-Live-MATE-x86_64-22-T3" TYPE="iso9660" PTUUID="20776222" PTTYPE="dos"
/mnt/fedora_koji/compose/22_TC3/22_TC3/Live/x86_64/Fedora-Live-SoaS-x86_64-22-TC3.iso: UUID="2015-05-08-02-13-20-00" LABEL="Fedora-Live-SoaS-x86_64-22-T3" TYPE="iso9660" PTUUID="2c4d9655" PTTYPE="dos"
/mnt/fedora_koji/compose/22_TC3/22_TC3/Live/x86_64/Fedora-Live-Xfce-x86_64-22-TC3.iso: UUID="2015-05-08-02-19-24-00" LABEL="Fedora-Live-Xfce-x86_64-22-T3" TYPE="iso9660" PTUUID="247c7cf7" PTTYPE="dos"
/mnt/fedora_koji/compose/22_TC3/22_TC3/Spins/x86_64/Fedora-Live-Design_suite-x86_64-22-TC3.iso: UUID="2015-05-08-02-27-39-00" LABEL="Fedora-Live-Dsgn-x86_64-22-T3" TYPE="iso9660" PTUUID="53e83304" PTTYPE="dos"
/mnt/fedora_koji/compose/22_TC3/22_TC3/Spins/x86_64/Fedora-Live-Games-x86_64-22-TC3.iso: UUID="2015-05-08-02-55-19-00" LABEL="Fedora-Live-Game-x86_64-22-T3" TYPE="iso9660" PTUUID="78f3b5ed" PTTYPE="dos"
/mnt/fedora_koji/compose/22_TC3/22_TC3/Spins/x86_64/Fedora-Live-Jam_KDE-x86_64-22-TC3.iso: UUID="2015-05-08-02-39-41-00" LABEL="Fedora-Live-Jam-x86_64-22-T3" TYPE="iso9660" PTUUID="663b35ef" PTTYPE="dos"
/mnt/fedora_koji/compose/22_TC3/22_TC3/Spins/x86_64/Fedora-Live-Robotics-x86_64-22-TC3.iso: UUID="2015-05-08-02-28-57-00" LABEL="Fedora-Live-Robo-x86_64-22-T3" TYPE="iso9660" PTUUID="148bab0b" PTTYPE="dos"
/mnt/fedora_koji/compose/22_TC3/22_TC3/Spins/x86_64/Fedora-Live-Scientific_KDE-x86_64-22-TC3.iso: UUID="2015-05-08-02-48-53-00" LABEL="Fedora-Live-SciK-x86_64-22-T3" TYPE="iso9660" PTUUID="73c0f803" PTTYPE="dos"
/mnt/fedora_koji/compose/22_TC3/22_TC3/Spins/x86_64/Fedora-Live-Security-x86_64-22-TC3.iso: UUID="2015-05-08-02-18-32-00" LABEL="Fedora-Live-Sec-x86_64-22-T3" TYPE="iso9660" PTUUID="156b3ce6" PTTYPE="dos"
}}}

the installer trees I guess all have the same label. however it is too late to make changes to the tooling for f22. note that workstation and cloud only have a boot.iso so if it is a dvd then it is Server being installed. for Cloud and Workstation you have to do a network install using the boot.iso you are probably better off actually just grabbing the kernel and initrd and pointing it at the right install tree for the product you are installing.

So in F22 we'll have workstation and server netiso separate and they seem to have exactly same volume-id:

Fedora-Workstation-netinst-x86_64-22_TC4.iso: Fedora-22_T4-x86_64
Fedora-Server-netinst-x86_64-22_Beta.iso: Fedora-22_B-x86_64

So the release status seems to be included as part of volume-id (which is quite irrelevant to anyone needing to rely on volume-id for anything AFAIK) but important info (server/WS and installer/netiso) is missing.

that the netinst and install dvd have the same volumue-id is intentinal and part of how everything has been designed to work. I have made some changes to pungi in rawhide to get product info into the volume-id. but changing the netinst and install dvd to have different volume-ids means completely redoing how we build the media. It is not likely to happen. realistically you should not be using the volume-id for what you are using it for.

there is already well defined entry points to get all the information. at least for the installation trees.

the .composeinfo file points to the locations for the installation media
http://dl.fedoraproject.org/pub/fedora/linux/releases/22/Workstation/.composeinfo is the f22 workstation version http://dl.fedoraproject.org/pub/fedora/linux/releases/22/Workstation/x86_64/os/.treeinfo is the .treeinfo file that points at the boot media. though it does not point at the install dvd in the Server version and we do not have anything for the livecds.

I think it would be helpful to know what you really need and what you do with it, then we can work out how to best bridge the gap. To me it seems that libosinfo is perhaps not doing things in the best possible way. At the least we never considered such use cases when designing and building the tools. So it is not all unexpected that you do not get the info that you think you should be getting.

I don't understand this insistence of justifying a bug in Fedora's process of creating ISOs. libosinfo isn't doing anything weird. volume-id has been a very reliable easy method to differential ISOs and as the name suggests, it's an "ID" (short for identification[1]). As I mentioned before, every OS out there sets unique IDs on different ISOs they distribute. When it's not the case, we file bugs on the distro and this is the first time distro is telling us that we are doing it wrong.

We now have means to also differentiate ISOs through volume-size (that I specifically added for Fedora) but that is only useful for releases (not Alpha, beta nightlies etc).

Having said all that, if you have ideas on what other methods we could utilize to differentiate ISOs, I'm all ears (I really mean it).

Regarding having not considered libosinfo's use case, I totally understand that you can't possibly think of all possible use cases but surely we can agree that it should have been kept in mind that volume-id are IDs and hence by definition it has to be unique.

[1] http://en.wikipedia.org/wiki/Identification_%28information%29

Still the same in F23. I guess i'm being ignored now cause that's probably easier then to fix this issue. :(

Still problematic with f24,

Fedora-S-dvd-{i386,x86_64}-24 is used both regular and netinst iso.

Any change requires massive rewriting and change in how we do things, we will have to run some processes twice. It's likely not going to happen any time soon. You really should not be relying on the volume id to differenciate between install DVD and boot.i

Replying to [comment:13 ausil]:

Any change requires massive rewriting and change in how we do things, we will have to run some processes twice. It's likely not going to happen any time soon. You really should not be relying on the volume id to differenciate between install DVD and boot.i

The problems isn't just that but the fact that volume ID naming seems to change from release to release and that's what elmarco was talking about. The differentiate between DVD and netiso isn't critical but we really really need the volume IDs to adhere to a specific pattern over the releases and different stages of the release.

The only other means to detect the ISO is volume-size and I've already spent quite many days in getting libosinfo to be able to do that. However, the volume size is not finalised until the release day and therefore we are unable to rely on that either until Fedora release. i-e no detection for Alpha, Beta or rawhide ISOs.

I trust that this is not easily fixed and that there are higher priorities. But that argument is not at all related to the appropriateness, and conformance aspect, of intentionally using the same Volume Identifier for two obviously very different volumes.

The ISO 9660 spec makes it really clear that the Volume Identifier is expected to be reasonably unique specifically for the purpose of volume differentiation and therefore file location. The Volume Set Identifier can be the same among volumes. Setting the Volume Identifier field to the same value for two different volumes thwarts the entire purpose of the Primary Volume Descriptor. I agree with zeenix that the current behavior is flawed, but further I think it arguably makes these ISOs not in conformance with ISO 9660's intent to enable the user to identify and differentiate what are in fact different volumes.

If it's OK to do this with two ISOs, then why not do it to all ISOs and just stick "fedora" as the Volume Identifier?

{{{

Fedora-Server-netinst-x86_64-24-1.2.iso
/dev/loop1: UUID="2016-06-14-15-55-58-00" LABEL="Fedora-S-dvd-x86_64-24" TYPE="iso9660" PTUUID="041a2601" PTTYPE="dos"
Volume Identifier=Fedora-S-dvd-x86_64-24
Volume Creation Date and Time=2016061415555800
ISO MD5SUM = 81ffacb48528f35651fe01eaa892060d
}}}

By filename, this is the net installer, not the dvd, but the Volume Identifier and therefore the LABEL provided by libblkid says it is the dvd.

{{{
Fedora-Server-dvd-x86_64-24-1.2.iso
/dev/loop0: UUID="2016-06-14-16-21-47-00" LABEL="Fedora-S-dvd-x86_64-24" TYPE="iso9660" PTUUID="26760f17" PTTYPE="dos"
Volume Identifier=Fedora-S-dvd-x86_64-24
Volume Creation Date and Time=2016061416214700
ISO MD5SUM = 9e37b58618c7f598417c620f92de9d15
}}}

The ISO MD5SUMs are different because they two ISOs contain completely different contents, and of course are intended for different installation use cases. So saying these two volumes are sufficiently similar to give them the same name is not compelling.

ISO 9660 has no field for UUID. It looks like libblkid is producing a UUID based on the Volume Creation Date and Time field in the Primary Volume Descriptor, which while it supports 1/100s precision is only populated with 1s precision. It might be sufficiently unique anyway, but there is also the MD5SUM field. So there are a couple of less obvious ways to know that two identically named volumes are in fact not identical. But both suffer from the same problem is differentiation on file size, which is that these values aren't known until the ISO is already created.

This may be fixed now with the new pungi version we use if not @lsedlar could you open an upstream issue on it please

Metadata Update from @ausil:
- Issue assigned to lsedlar
- Issue close_status updated to: None

6 years ago

I checked and current status is that there indeed are two different ISO images with identical volume ids. The netinst produced by lorax shares the same value with isos that Pungi create in createiso phase (which is basically the netinst with the yum repo).

This commit added the ability to customize the dvd substring of the volume id, but it uses the same value for for both the above mentioned ones.

Maybe it should use netinst instead as for the lorax ones? I'm not sure if it would break something.

https://pagure.io/pungi/pull-request/636 would change the netinst isos to actually say netinst in the volume id.

@mohanboddu thinks this might be fixed and is checking.

Metadata Update from @syeghiay:
- Issue close_status updated to: Fixed
- Issue status updated to: Closed (was: Open)

5 years ago

Login to comment on this ticket.

Metadata