Was there a previous ticket, or did you just unbundle them by default?

I assume these are going to be permanent bundling?

If you could fill out the std. bundling questions, that might help:

https://fedoraproject.org/wiki/Packaging:No_Bundled_Libraries#Standard_questions

Replying to [comment:3 james]:

Was there a previous ticket, or did you just unbundle them by default?

No previous ticket, we just unbundled them.

I assume these are going to be permanent bundling?

Yes.

If you could fill out the std. bundling questions, that might help:

You might want to number the questions .... but anyway:

Has the library behaviour been modified? If so, how has it been modified? If the library has been modified in ways that change the API or behaviour then there may be a case for copying. Note that fixing bugs is not grounds to copy. If the library has not been modified (ie: it can be used verbatim in the distro) there's little chance of an exception.
Why haven't the changes been pushed to the upstream library? If no attempt has been made to push the changes upstream, we shouldn't be supporting people forking out of laziness.

   Have the changes been proposed to the Fedora package maintainer for the library? In some cases it may make sense for our package to take the changes despite upstream not taking them (for instance, if upstream for the library is dead).

Yes, the library was modified. The namespaces are modified, in Bundler they are prepended by Bundler:: and similarly in RubyGems, they are prepended by Gem::. But the biggest modification is that they have modified all load paths. E.g. if the original library had line "require 'molinillo/gem_metadata'", in the bundled version it looks like "require 'rubygems/resolver/molinillo/lib/molinillo/gem_metadata'". There is import script [1] which does these changes for RubyGems and there is similar for Bundler.

Could we make the forked version the canonical version within Fedora? For instance, if upstream for the library is dead, is the package we're working on that bundles willing to make their fork a library that others can link against?

No. Although if there were used require_relative instead of require, it could make the possible unbundling easier, but I don't thing usage of require_relative is encouraged practice.

Are the changes useful to consumers other than the bundling application? If so why aren't we proposing that the library be released as a fork of the upstream library?

There are no functional changes in the bundled library, although it might be updated to some prerelease version in some cases or modified if needed. But hard to say if this will happen in future.

Is upstream keeping the base library updated or are they continuously one or more versions behind the latest upstream release?

https://github.com/rubygems/rubygems/pull/1189#issuecomment-123434653

What is the attitude of upstream towards bundling? (Are they eager to remove the bundled version? are they engaged with the upstream for the library? Do they have a history of bundling? Are they argumentative?)

They love bundling, thats why they develop "Bundler". But to be honest, RubyGems as well as Bundler are bit exceptional libraries, since you probably want to restrict package manage's dependencies. Moreover, the previous way of lightweight bundling caused some conflicts between bundled version of library with possibly different version of library installed on system [2].

Overview of the security ramifications of bundling

Molinillo is dependency resolver. So I don't expect any security issues. net-http-persistent is quite stable already for some time, I don't expect there to be issues. From thor, there is used just some small subset of functionality for managing CLI input I believe. I can't imagine there should be any serious issues.

Does the maintainer of the Fedora package of the library being bundled have any comments about this?

I sad already enough on Fedora as well as on upstream side ...

Is there a plan for unbundling the library at a later time? Include things like what features would need to be added to the upstream library, a timeline for when those features would be merged, how we're helping to meet those goals, etc.

No, I don't think this will ever happen. And we tried to unbundle the libraries for quite some time already, but this become unmaintainable.

Please include any relevant documentation -- mailing list links, bug reports for upstream or the bundled library, etc. Include also a list of the bundled files (or a link to a document containing them) and if possible links to the actual files so that they may be easily inspected.

They were in my initial request. Otherwise feel free to search my nick 'voxik' in upstream repositories for my comments:

https://github.com/drbrain/net-http-persistent [[BR]]
https://github.com/erikhuda/thor [[BR]]
https://github.com/CocoaPods/Molinillo [[BR]]
https://github.com/rubygems/rubygems [[BR]]
https://github.com/bundler/bundler/ [[BR]]

[1] https://github.com/rubygems/rubygems/blob/master/Rakefile#L111 [[BR]]
[2] https://github.com/bundler/bundler/issues/3492

We discussed this at this weeks meeting (http://meetbot.fedoraproject.org/fedora-meeting-1/2015-08-27/fpc.2015-08-27-16.05.txt):

• 560 Bundling exception for

  
  rubygem{molinillo,thor,net-http-persistent} (geppetto, 16:46:03)
• LINK: https://fedorahosted.org/fpc/ticket/560 (geppetto, 16:46:04)
• ACTION: Allow rubygems to bundle molinillo, because software
  engineering is hard (+1:5, 0:0, -1:1) (geppetto, 17:28:55)
• Everyone seems to be leaning on just dropping "Bundler", upstream
  doesn't seem to want anyone to ship it and presumably rumbygems can
  get the mess if devs. want it. The deps. on it from vagrant and
  rails seem bad though … can they be removed? (geppetto, 17:30:23)

The deps. on it from vagrant and rails seem bad though … can they be removed?

Not from Vagrant since that would mean not being able to install any plugins. Also just Bundler on its own is one of the key components in the Ruby world. If anyone is interesting in anything, it's often Ruby (with RubyGems) + Bundler. And we, as Fedora, should offer that. I would be strongly against the removal even though I am also not happy about the upstream attitude.

Here are some answers on some random points somebody of you made during the meeting.

16:55:38 <orionp> Does anything use rubygem-bundler in Fedora? Perhaps just not packaging it....

This was common theme during the meeting, but I am strongly against it. Yes, RoR and Vagrant uses Bundler, but if you take a look what is shipped in RHEL, what OpenShift is using, what is contained in RHSCL, it is recently just Ruby (yes, they bundle RubyGems by default) and Bundler. That is todays typical starting point for every Ruby application (lets take a look on Redmine's installation instructions [1] for example). Unfortunately, I can't imagine Fedora without Bundler.

16:57:36 <tibbs|w> Why not package the two libraries separately?

That makes no sense. ATM, the bundled Mollinilo, Thor or net-http-persistent are in regards of the functionality equivalent to the independent rubygem- packages. But the thing is the namespace and resolution. The biggest change what they did was not only the namespace thing, which is cosmetic, but they change how the internal parts of libraries is loaded, e.g. [2, 3]. Previously, the load path was modified [4] and hence it might happen that different than expected version of Thor could be loaded.

Please note that this is typically not issue for Fedora if you exclusively use gems shipped via RPM, but this is well possible in pure RubyGems environment or if you use mixture of RPM and non RPM gems.

17:03:40 <orionp> Upstream states: Bundler and Rubygems can't have dynamically resolved dependencies.[[BR]]
17:04:13 <orionp> Which I can kind of parse, at least in the rubygems case

Bundler builds on top of RubyGems, but it uses its own resolver, which was historically more capable. Now, the same resolver is the same, but still, Bundler uses just part of RubyGems functionality and replaces the other parts by its own.

Also note, that since beginning, Bundler was more or less intended to be integrated into RubyGems. I am not sure if this is still the plan, but there were certainly made significant steps to make these two projects closer and the same resolver is definitely one of the steps. Unfortunately upstream of Bundler still keeps backward compatibility with older RubyGems, that is why they refuse to simply drop their own version of resolver.

17:18:26 <tibbs|w> I can't understand why anything would want to call bundler, though.

If you have more versions of gem installed on the system, bundler assures that the right one, compatible with other gems is loaded. This is actually very good feature, RubyGems itself does not provide. RubyGems would load the most recent version of some gem just to find out later it is not compatible with some other version of gem it is trying to load later.

17:22:04 <Rathann> that we as distro make sure that this "Bundler and Rails may need need to load different versions of Thor." doesn't happen

As long as we support installation of gems directly from RubyGems.org (and NO we don't want to drop this functionality), neither we can be sure, since you can have compatible version of Thor installed via RPM, but later pulled in some incompatible version via RubyGem/Bundler from RubyGems.org.

17:24:59 <orionp> If I hear "only guaranteed to work with version X" one more time....

This is more or less just urban legend IMO. We are executing the same test suite as they are, but admittedly, we failed at times, mainly since their usage of some git snapshot version.

17:26:28 <orionp> Well it seems upstream doesn't want bundler packaged, so there [[BR]]
17:26:59 <orionp> The Bundler team does not provide support for installing or using Bundler as an OS package

I don't think this is fair from upstream, but unfortunately, I don't have any proof in my hand to prove how many people is using bundler via RPM, so it is hard to change their mind :/

[1] http://www.redmine.org/projects/redmine/wiki/RedmineInstall#Step-4-Dependencies-installation [[BR]]
[2] https://github.com/erikhuda/thor/blob/master/lib/thor.rb#L2 [[BR]]
[3] https://github.com/bundler/bundler/blob/master/lib/bundler/vendor/thor/lib/thor.rb#L2 [[BR]]
[4] https://github.com/bundler/bundler/blob/0f2adb28025911ed9ef19b4d67bd9431ac254eef/lib/bundler/vendored_thor.rb

Ping ... any chance you could grant the exception soon?

As FPC is no longer in the business of approving or disapproving bundling requests outside of the critical path, you are now free to bundle as you wish. Please do add the proper Provides: bundled(...) tags to your packages.