#943 Yum repository format and its effects on Fedora infrastructure
Closed None Opened 11 years ago by djasa.

= phenomenon =
from my POV, yum repository format doesn't scale well with ever-growing repository size and this causes adverse effects on fedora infrastructure (up to tens of % of total traffic) which in turn means lack of desirable features like rollback - because making repos any bigger than absolutely necessary makes metadata overhead even larger

= reason =

= recommendation =
if bug 850896 [1] gets implemented sooner than later, metadata traffic would drop by one-and-half to two orders of magnitude, making it negligible compared to actual data traffic

[1] https://bugzilla.redhat.com/show_bug.cgi?id=850896


I've asked some yum folks to comment in the attached bug.

Personally, I think agreement should be reached by maintainers who's packages consume repodata before any changes are made, with input from rel-eng and infrastructure.

I don't feel fesco should bypass that process.

I would personally also be very much for shipping N and N-1 packages if we can come up with a nice way to do so. Always having the ability to 'yum downgrade' would be quite valuable if we can get buy in from mirrors and rel-eng folks for tooling.

I don't feel there's much for fesco to do here until there's a consensus on how to change the repodata.

Right, I don't think there's much for FESCo to do here other than get interested people working together on solving the problem.

I think I won't be able to attend the meeting so I'll try to summarize all my arguments in this comment:

  • for people with slow (anything below low hundreds of kB/s), FUP'd (capped at anything above low GB's [1]) or paid-per-traffic lines
  • this may be annoying to the point of not choosing Fedora for systems using such lines predominantly or exclusively
  • this is the reason why I personally started to dig into this issue
  • IMHO this is less significant than other reasons below

  • effective inability of rollback hurts most Rawhide IMHO because of prolonged periods, when:

  • you cannot install system equivalent to the one you'd get with {{{yum --skip-broken update}}} in any other way then by cherry-picking older versions of desired package and all of its dependencies from koji
  • you cannot install system at all if there are broken dependencies in package set by anaconda (not sure about this one, I didn't need this recently)

  • significant portion of total traffic from mirrors taken by metada (my very rough guess is 10-50 % - Infrastructure team could give a better figure here) is impolite to outright rude to the mirror donors

  • total package count seems growing almost constantly linearly from fc3 to fc17 [2] while in my experience, internet connection over here (CZE) didn't improve in general in last two years

All of these effects do some '''harm''' to Fedora project '''as whole'''.

IMHO FESCo can get the actual figures and assess the real impact to the Fedora Project better than me (I may tend to exaggerate it) or yum/dnf people (who IMHO underestimate its total severity) - and if it is judged as important/severe, I'm sure yum/dnf people will give it higher priority. This could be a reply to [comment:4 kevin] and [comment:5 notting]:

I don't feel there's much for fesco to do here until there's a consensus on how to change the repodata.

Right, I don't think there's much for FESCo to do here other than get interested people working together on solving the problem.


Replying to [comment:4 kevin]:

I've asked some yum folks to comment in the attached bug.

Thanks for that.


[1] [https://en.wikipedia.org/wiki/Bandwidth_cap#Download_quota Wikipedia: Bandwidth Cap#Download Quota][[BR]]
[2] [https://admin.fedoraproject.org/community/#statistics/packages Fedora Package Number Statistics]

From 2012-09-05 FESCo meeting:

FESCo suggests the requesters come up with a consensus/concrete proposal for changes with stakeholders and re-open this ticket when necessary. Resolving as 'wontfix' for now.

Login to comment on this ticket.

Metadata