Hello

I would like to bring up an issue that might have been overseen during the
process of introduction and adoption of "rich dependencies" within RPM packages
and the covering package managers around it.

Unfortunately english isn't my native language, so please excuse in advance, if
sentences don't sound as they should.

I know this issue can be seen as sensetive, but I really try to be as
objectively as possible. I try to explain the case of problems with our own
situation that we are currently trapped in. We have no way getting out of the
conflict here and therefore see the requirement to bring this up to fesco.

= Short summary of the issue: =
1. RPM supports "rich dependencies".
2. DNF supports resolving packages with "rich dependencies".
3. YUM does not support resolving packages with "rich dependencies".

Issue 3) is causing our infrastructure to break apart and we are out of service
now.

= The preface story: =
We are a team of people who are enjoying the work coming from the community
people and Red Hat related to Fedora. Our main objectives are systems
integration, software engineering, cross platform building of software, some
administrative tasks, process automation and semi automation and other work
which is heavily related with RPM packages, DNF, YUM and package management.

We've been using and supporting Fedora, CentOS, RHEL because we believe in the
work done by the community, by Red Hat and all the volunteers over the world.
In the past years we wrote an infrastructure, that covered the use cases that
YUM offered us. The infrastructure was an ongoing process. It started from
scratch, got improved, stabilized, tested for hours, months, years.

= The challenges we had: =
With the change to Fedora 22 YUM got replaced by DNF and YUM got marked as
deprecated. And here is, where all our problems started. Initially we thought,
that DNF may be an inplace replacement of YUM. A lot of news sites made it look
like this. Even comments on the Fedora mailing lists tend to make us believe,
that this might be the case. And until Fedora 22, we didn't saw a reason to go
into conversation with the DNF people. DNF wasn't finished until then and we
were quite confident, that the work on DNF will go on for yet another while.

With Fedora 22 and DNF as main package management system we quickly figured
out, that we will be running into deep problems because DNF is not an inplace
replacement as we believed - even though it offers similar command line
arguments. The problems we saw can be described as following:

• Exit and Return codes got changed. Causing our infrastructure to run in
  trouble when checking for error and exit codes.

• Removal of verbosity and debug output in a level, that we couldn't process
  the data anymore. The verbosity and debug output is simply missing. You can
  see this, if you issue yum -d 3 <somecommand> vs. dnf -d 3 <somecommand>.

• Missing or removed command line options (and use cases), that has been part
  in YUM for many years and that we urgently rely on.

• --skip-broken. I don't need to explain this. It has been covered a lot on the
  devel mailing list. We still don't know howto deal with that missing
  situation because it will render our work flow to operate differently. But
  this has our least priority for now.

• Ongoing and frustrating communication with the developers around DNF on
  Bugzilla. In all the years, we've never seen opened reports shut down faster
  than we were able to log off from Bugzilla. Rendering most - but not all - of
  our reports staying unrecognized. A really frustrating experience ontop of
  the challenges we were presented with.

The above listed items caused us really hard and frustrating times changing
huge chunks of code, that worked flawlessly for years. Over the months we had
to cover the new exit and return codes, introduce a different way in getting
necessary information during the DNF process. Tasks that we did, by grepping
the verbose and debug output of YUM had to be re-written entirely for DNF.

The transition from YUM to DNF has nearly completed - with one exception. We
can't switch one of our main use cases to DNF yet because we are waiting for
the missing parts to show up again. Please note that we are no python
developers. We don't know the internals of YUM or DNF, to cover the missing
bits.

We also learned that the developers around DNF behave quite sensible when it
comes to DNF related things. I don't know why and what but some of them seem
to be treating everything related to it, in an unprofessional way. A bunch of
our reports and explained use cases has been closed, even before we were able
to explain ourselves correctly. Sometimes we've been having the feeling that
our use cases are not really understood. This gave us another level of
frustration ontop of that and we ended up avoiding dealing with the developers.
We hoped that if we keep quiet, that things will go better in our direction, by
giving them the time needed to cover the use cases. We were also told that old
options will be re-introduced into DNF as time goes by.

= The challenge that makes everthing break now: =
One of our use cases can be described as following.

We use DNF / YUM to deal with all kind of packages for Fedora. We for example
run Fedora 22 and use the YUM / DNF that comes shipped with it (including
updates) to deal with packages. We deal not only with Fedora 22 packages but
also with Fedora 23, 24 and rawhide packages. E.g. using a lower version Fedora
release to cover packages of a future Fedora release.

We are not doing any magic here. We only download a subset of packages and
metapackages (groups), have YUM resolve the dependencies and have them stored
in a specific directory. YUM offers the possibility to download packages in a
specific directory. By this it offers a global option named:

--downloaddir

The YUM process is being started a couple of times per day, grabbing all newly
updated packages from the repositories. Because of the --downloaddir option,
the already downloaded packages are being skipped and only the subset of new
packages are being downloaded.

DNF is lacking this global option but it offers a slightly different command
named:

dnf download <packages> --destdir --resolve

Unfortunately this command deals with packages only. It doesn't understand
metapackages (groups). We tried covering our use case by xpath'ing the
comps.xml files from Fedora git and parsing the metapackages (groups), to
receive the list of packages. Sadly DNF can't process large chunks of packages
in one go within the command line. Not only that! It also makes our own code
look like a hackfest.

Initially we thought about changing a couple of lines of code in our
infrastructure. Maybe the one or other command line. But we never ever imagined
that we ended up basicly rewriting everything.

Right now we need to stay with YUM for this one particular use case because DNF
is not covering it. Bugzilla has plenty of reports coming from different people
that request this to show up.

= And now the current situation: =
Sadly we are now facing a problem that we can not solve anymore. We can't go
further nor backwards anymore.

Since a couple of days (let's say 1.0 - 1.5 weeks) ago, packages are adopting
"rich dependencies". Sadly YUM the system we still depend on, can't deal with
rich dependencies and causes a lot of errors when pulling and resolving
packages from the repositories. In short: It aborts!

= And here is the problem: =
There is no alternative available!

The use of "rich dependencies" is something that is only covered by RPM and
DNF. People who are still forced to use YUM (because of current limitations of
DNF) are left out of this.

Please note, that this has nothing to do with "religious" preferences like YUM
vs. DNF or so. We really like to switch to DNF today rather than anytime else.
We can't because we were unwilling, we can't because there is no alternatives.

Where possible, we were able to convince package maintainers with the current
challenges we have. Most of the maintainers have been very helpful, understood
our issues, understood the "do not break" policy and solved the issues. Some
other maintainers sadly not - and we do understand it as well because it's
basicly not a "bug". It's just an uncovered situation.

Sadly our infrastructure (and our automated workflow) broke apart and is out of
service for a couple of days now.

We really like to switch to DNF to complete our transition from YUM to DNF.
Sadly the features we were expecting are still missing. Please note that we
are not talking about new features. We are talking about features (options)
that has been part of YUM for years.

We don't know what to do anymore and we also don't understand how a core
element such as DNF / YUM got altered in a way, that breaks everything related
and around it. It's equal like breaking an API but an API break usually offers
an alternative. And here this is not the case.

Anyways! My explaination is not the best and english isn't my native language.
So pointing to Bugzilla where I tried to explain the situation (and possible
solutions) might be more of a help:

https://bugzilla.redhat.com/show_bug.cgi?id=1317481

In this bugreport, which covers some errors our system reported, I had a
conversation with a maintainer, where I explained the problem case. Said
maintainer was very helpful and solved the issues. He also had some
conversation with different people. During his conversation he mentioned that
some other 3rd party programs also rely on YUM and that they might break as
well because of unsupported "rich dependencies". You will also find a much
better explainations of the trap we are in.

The problem is, that there are still people outside who have written and fully
working stuff around YUM that they depend on. Be it for private usage, be it in
a corporate environment, be it for infrastructure related things and so on.
They rely on using YUM not because they are huge fans of YUM. No! Because there
is no alternative for it. Sadly YUM reached a point where it can't deal with
"rich dependencies" anymore. This renders a lot of said infrastructures to get
in trouble. No alternatives offered! (Not to mention all the general DNF
problems that still exists).

The situation is not just about "software development". There are probably a
bunch of small or larger companies around here that face themselves running
into trouble sooner or later because things that has been there - and there
forever - got changed in a way where even exit and return codes are not working
as expected anymore and features they rely on are simply stripped away or not
carried over to the new system.

I also like to mention that even RHEL will probably switch from YUM to DNF one
day (maybe the next release ?). How about the people using RHEL facing
themselves in a position where their stuff breaks apart because no alternative
has been offered.

I used to work for a large IT-Service-Corporate a few years ago (Degree in
upper management and projectmanagement). Had the same enthusiasm in "coding"
and "developing" new things. Developers usually tend to forget (or ignore or
simply don't have the view) of real life demands of administrators, engineers
and so on. They are caught in their own world with "development" - forgetting
the chain around it. The chain of people that rely on the work done before.
Which - in worst cases - can cost them their bread and butter - if things stop
working from one day to another.

We also had competency groups like "juniors", "professionals", "seniors",
"leading professionals", "principals" and so on. Those competency groups
usually describe the maturity or the people that are working for a company. It
also describes the amount of customer relations, communications and other
things tied with it.

= Possible '''optional''' solutions that comes to my mind: =
1. Have yum support the relevant new stuff
2. Have dnf support a general --downloaddir option
3. Have dnf download groupname --destdir --resolve become able dealing with metapackages (groups)

Please note, that I will not be responding to this ticket anymore (if
possible). We do have a good relationship with most members of the Fedora
community and would like to have it be like this. We really like to continue
using Fedora, we like to continue introducing Fedora to others and therefore
like to ask for help. It would be nice if this situation can be cleared in an
acceptable and respective way. This is the first time that I ever contacted
fesco.

With regards,

Ali Akcaagac