#6144 Document the semantics of "comps"
Closed: Duplicate 5 years ago Opened 8 years ago by rholy.

We, as the developers of DNF, miss a document that describes what "comps" mean and how should package managers work with them. Until it is documented, we cannot handle DNF bugs related to "comps" properly since we don't know what is the expected behaviour.

E.g. what does "mandatory" packages mean? Could a "comp" be installed without a mandatory package (since it seems that there is a common practice to do that in kickstart files; e.g. RHBZ 1131969 and RHBZ 1185408)? What should happen if a "comp" is installed and its mandatory package gets removed? What if multiple comps share the same mandatory packages? And so on... And please don't focus only on mandatory packages. If there is a piece of software that has to work with another one, the dependency has to be documented.

Also another issue related to the mandatory packages comes to my mind. Since I understand "comps" as something that specifies which packages are the Fedora default (is true? can you document it?), what is the recommended approach to replace a default package (YUM) with a new default package (DNF) on all user systems? That is something that should be possible to be specified on the comps level IIUIC.

Also it would be nice if there will be a way how to specify which packages are preferred over which ones since there are people that are convinced that Fedora e.g. prefers MariaDB over community-mysql so Fedora has to express it somehow.

Anyway, we can discuss it once you finish the documentation. I just wanted to let you know about some use cases in case it would help you to design the semantics. It would be nice if you could fix it as soon as possible.

(Actually, DNF uses "libcomps" to work with "comps" so in fact, we don't mind if it is documented on "comps.xml" level or on "libcomps" level but I assume that "libcomps" cannot be documented until "comps.xml" is documented anyway.)


Since this is a critical thing for us and it probably needs cooperation between multiple teams, let me CC representatives of Anaconda and RH rel-engs.

I'd like to start a broader discussion on this topic. I had conversations with several people in the past where we agreed that the semantics of comps, never mind the lack of their documentation and common understanding, might not be sufficient for the future.

Could we use this opportunity to start a discussion about what are our teams' expectations of comps? In my opinion we represent the most important stakeholders here so if we agree on something, it shouldn't be too difficult to come to FESCo and get it approved.

It really would be nice to have the comps specification documented somewhere. Most of the functionality that is understood or expected of comps comes from historical research. Not great, but that's what we have. comps do not fit in to the standard RPM metaphor we have because they describe RPMs outside of the context of a single RPM. Development and maintenance around comps has always been outside of the standard package maintenance model.

For installation-time behavior, I recommend the kickstart %packages documentation:

https://github.com/rhinstaller/pykickstart/blob/master/docs/kickstart-docs.rst#chapter-3-package-selection

That explains how the installer has treated mandatory, default, and optional attributes on packages defined in comps. In brief:

1) Mandatory packages are always installed if a user asks for a comp (a group in kickstart or installer language).

2) Default packages are installed, but the user can exclude them. In the graphical installer, these are the packages that would appear checked by default but the user could uncheck them.

3) Optional packages are not installed by default, but the user can select them. The opposite of #2.

So there are really just two types of packages: mandatory and user selectable. "default" means the user selectable is flagged for installation by default. "optional" means the package is not flagged for installation by default.

All of this language really only matters when users are working with comps definitions. That is, package groups. Installing @desktop or @development triggers the definitions from comps. Users can, and sometimes do, list individual packages by name in the %packages section.

As for overlapping comps definitions....this shouldn't happen in comps directly, but does happen via RPM dependencies. A comps definition is typically short (except for the translation data) because we know that RPM dependencies will pull in a lot of packages. Comps definitions don't work without RPM dependencies. The appearance of overlapping comps/group membership in an RPM is (or should be) the result of RPM dependency resolution and not duplicate listing in the comps file.

Any changes to how comps works with DNF should take in to account the established behavior in %packages in kickstart files. We can change things, but we should avoid breaking existing behavior for users.

Replying to [comment:4 dcantrel]:

It really would be nice to have the comps specification documented somewhere. Most of the functionality that is understood or expected of comps comes from historical research. Not great, but that's what we have. comps do not fit in to the standard RPM metaphor we have because they describe RPMs outside of the context of a single RPM. Development and maintenance around comps has always been outside of the standard package maintenance model.

For installation-time behavior, I recommend the kickstart %packages documentation:

https://github.com/rhinstaller/pykickstart/blob/master/docs/kickstart-docs.rst#chapter-3-package-selection

That explains how the installer has treated mandatory, default, and optional attributes on packages defined in comps. In brief:

1) Mandatory packages are always installed if a user asks for a comp (a group in kickstart or installer language).

there is a * if the package is not available it is skipped. which we use for things like bootloaders. If dnf was to refuse to install a group if a package in the group was not available we would need a way to specify in comps that the package was manadatory on some subset of arches.

2) Default packages are installed, but the user can exclude them. In the graphical installer, these are the packages that would appear checked by default but the user could uncheck them.

3) Optional packages are not installed by default, but the user can select them. The opposite of #2.

So there are really just two types of packages: mandatory and user selectable. "default" means the user selectable is flagged for installation by default. "optional" means the package is not flagged for installation by default.

All of this language really only matters when users are working with comps definitions. That is, package groups. Installing @desktop or @development triggers the definitions from comps. Users can, and sometimes do, list individual packages by name in the %packages section.

As for overlapping comps definitions....this shouldn't happen in comps directly, but does happen via RPM dependencies. A comps definition is typically short (except for the translation data) because we know that RPM dependencies will pull in a lot of packages. Comps definitions don't work without RPM dependencies. The appearance of overlapping comps/group membership in an RPM is (or should be) the result of RPM dependency resolution and not duplicate listing in the comps file.

Any changes to how comps works with DNF should take in to account the established behavior in %packages in kickstart files. We can change things, but we should avoid breaking existing behavior for users.

we definitely should not break any long term expected behaviours.

Metadata Update from @ausil:
- Issue assigned to ausil

6 years ago

Metadata Update from @syeghiay:
- Issue close_status updated to: None

6 years ago

Metadata Update from @syeghiay:
- Issue tagged with: review

6 years ago

Metadata Update from @syeghiay:
- Issue assigned to syeghiay (was: ausil)

6 years ago

Metadata Update from @syeghiay:
- Assignee reset

6 years ago

Metadata Update from @kellin:
- Issue assigned to kellin

6 years ago

This ticket needs a deep dive with Rob and Mohan. Maybe involve mattdm later.

Metadata Update from @syeghiay:
- Issue close_status updated to: Duplicate
- Issue status updated to: Closed (was: Open)

5 years ago

Login to comment on this ticket.

Metadata