Ticket #134 (closed enhancement: fixed)

Opened 2 years ago

Last modified 2 years ago

New packaging guidelines for Ruby

Reported by: bkabrda Owned by: spot
Priority: major Milestone:
Component: Guideline Draft Version:
Keywords: Cc: vondruch@…, mmaslano
Blocked By: Blocking:

Description

Hi, we would like to have have our new Ruby packaging guidelines reviewed and approved as a part of migration to Ruby 1.9.3 [1], which will incorporate some changes in the way of Ruby packaging. The proposed draft is at [2].

Regards, Bohuslav.

[1] http://fedoraproject.org/wiki/Features/Ruby_1.9.3 [2] https://fedoraproject.org/wiki/PackagingDrafts/Ruby

Change History

comment:1 Changed 2 years ago by spot

I think you have a mistake in the definition of %{gem_instdir}, it says "%{gem_dir}/gems/%{gem_name}-%{version}", but that would eval out to "/usr/share/gems/gems/%{gem_name}-%{version}". Perhaps it should be "%{gem_dir}/%{gem_name}-%{version}" ?

Also, %{gem_cache} is odd as %{gem_dir}/cache/%{gem_name}-%{version}.gem ?

Perhaps a better top level directory would be something like /usr/share/ruby-1.9.3/

comment:2 Changed 2 years ago by james

Also, why isn't gem_cache called gem_cachedir?

comment:3 Changed 2 years ago by bkabrda

To give some background info: RubyGems? are generally placed under a directory with following subdirectories:

  • cache - a directory with cached versions of Gems (a single file named %{gem_name}-%{version}.gem for each Gem) - these are only used for "gem pristine" command, which restores the Gem to its original state (if user modifies it and does something wrong, he can use this) - obviously, this can be replaced by "yum reinstall" in RPM versions of Gems.
  • doc - directory with documentation, each Gem has its own subdirectory called %{gem_name}-%{version}
  • gems - directory containing the actual source code of each Gem under the proper subdirectory - %{gem_name}-%{version}.
  • specification - directory containing the Gem specifications, used by RubyGems? library (again, named %{gem_name}-%{version}.gemspec).

Now to answer you questions:

  • The top directory of the Gem structure is /usr/share/gems - this is done because we are planning to introduce multiple Ruby runtimes (JRuby, Rubinius) and the Gems should be available independently on the Ruby implementation installed on user's computer (therefore /usr/share/ruby-1.9.3/ would not be a good name for the top level directory).
  • Since the top directory of the Gem structure is /usr/share/gems and the directory structure under it reflects a standard Gem directory (as explained above), the source code of each Gem is placed under /usr/share/gems/gems/%{gem_name}-%{version} == %{gem_dir}/gems/%{gem_name}-%{version}.
  • Also, as explained above, the %{gem_cache} expands into /usr/share/gems/cache/%{gem_name}-%{version}.gem, which is a single file - the cached Gem.

comment:4 follow-up: ↓ 10 Changed 2 years ago by spot

/usr/share/gems/gems/ just seems wrong to me. I still think that first "gems/" should be renamed to avoid confusion. Even if it was /usr/share/rubygems/.

comment:5 follow-up: ↓ 11 Changed 2 years ago by toshio

Some questions that come to mind immediately:

  • Why are we deprecating packaging gems for non-gem use? One of the things we looked at originally was how to not have to force packagers and end users to patch upstream ruby application and script code in order to use libraries which we had only packaged as gems. Does ruby no longer have the former methods of importing libraries? Everything must be done via gems now?
  • I don't see %{ruby_vendorlibdir} defined anywhere in the draft. Is that a %{_libdir} directory or a /usr/lib directory?
  • It looks like you have some instances of %{gemdir} ; are those supposed to be %{gem_dir} ?
  • A good addition to this guideline would be how to deal with having to apply patches to a gem. If I'm reading this right, for the pure ruby case, the patch can be applied directly to the source that was created with gem install in %prep. It's unclear what to do when patching something that has C extensions.

comment:6 follow-ups: ↓ 8 ↓ 12 Changed 2 years ago by spot

One more item: It looks like the gem_install block (for both noarch and arch gems) should be fine to call in %build (where it would be more appropriate) instead of %prep. Is there any reason that this is impossible or impractical?

comment:7 follow-up: ↓ 13 Changed 2 years ago by toshio

Also, it might help us see what the effect of the new guidelines are if you could post a sample srpm that complies. Then we can poke at it and see how it works.

it seems like there are several cases:

  • rubygem -- pure ruby
  • rubygem -- with cextension
  • rubygem -- with normal ruby library (pure ruby)
  • rubygem -- with normal ruby library (with c extension)

We don't need a srpm for each case, but it would be helpful if we have enough srpms that we can tell what happens in each of these cases.

comment:8 in reply to: ↑ 6 ; follow-up: ↓ 14 Changed 2 years ago by toshio

Replying to spot:

One more item: It looks like the gem_install block (for both noarch and arch gems) should be fine to call in %build (where it would be more appropriate) instead of %prep. Is there any reason that this is impossible or impractical?

We also analyzed this a bit further and realized that the %setup line isn't extracting the sources. The source isn't extracted until the "gem install" command is run. However, we may come back to this once we know what gem commands need to be run to patch rubygems that have C extensions/ if there's some alternate commands that allow us to extract the gem in %prep and then build it/install it to the proper looking hierarchy in a later step.

comment:9 follow-up: ↓ 15 Changed 2 years ago by rathann

All the tutorials for gems talk about gem build command. Why isn't it used here?

comment:10 in reply to: ↑ 4 ; follow-up: ↓ 16 Changed 2 years ago by bkabrda

Replying to spot:

/usr/share/gems/gems/ just seems wrong to me. I still think that first "gems/" should be renamed to avoid confusion. Even if it was /usr/share/rubygems/.

Well, that is really just a matter of opinion. We think it's perfectly ok. BTW, /usr/share/rubygems contains the actual Rubygems library.

comment:11 in reply to: ↑ 5 Changed 2 years ago by bkabrda

Replying to toshio:

Some questions that come to mind immediately:

  • Why are we deprecating packaging gems for non-gem use? One of the things we looked at originally was how to not have to force packagers and end users to patch upstream ruby application and script code in order to use libraries which we had only packaged as gems. Does ruby no longer have the former methods of importing libraries? Everything must be done via gems now?

Since Ruby now requires Rubygems library automatically, there is no need for non-gem subpackages. Moreover, Ruby core programmers want to gemify standard library [1], so the Gems are pretty much a standard in Ruby world and no one uses Ruby without Rubygems.

  • I don't see %{ruby_vendorlibdir} defined anywhere in the draft. Is that a %{_libdir} directory or a /usr/lib directory?

It is defined there, see the table in [2].

  • It looks like you have some instances of %{gemdir} ; are those supposed to be %{gem_dir} ?

You are right, I fixed this.

  • A good addition to this guideline would be how to deal with having to apply patches to a gem. If I'm reading this right, for the pure ruby case, the patch can be applied directly to the source that was created with gem install in %prep. It's unclear what to do when patching something that has C extensions.

For the pure Ruby case, you are right, for binary extensions, you need to use gem install, apply the patch and then recompile. You are probably right, that we should create an example that would show how to do this.

[1] https://bugs.ruby-lang.org/projects/ruby/wiki/StdlibGem [2] https://fedoraproject.org/wiki/PackagingDrafts/Ruby#Build_Architecture_and_File_Placement

comment:12 in reply to: ↑ 6 ; follow-up: ↓ 17 Changed 2 years ago by bkabrda

Replying to spot:

One more item: It looks like the gem_install block (for both noarch and arch gems) should be fine to call in %build (where it would be more appropriate) instead of %prep. Is there any reason that this is impossible or impractical?

AFAIK the %patch macros only work in the %prep section, so we would need to apply the patches manually in %build - so it would be impractical.

comment:13 in reply to: ↑ 7 Changed 2 years ago by bkabrda

Replying to toshio:

Also, it might help us see what the effect of the new guidelines are if you could post a sample srpm that complies. Then we can poke at it and see how it works.

it seems like there are several cases:

  • rubygem -- pure ruby
  • rubygem -- with cextension
  • rubygem -- with normal ruby library (pure ruby)
  • rubygem -- with normal ruby library (with c extension)

We don't need a srpm for each case, but it would be helpful if we have enough srpms that we can tell what happens in each of these cases.

We have plenty of SRPMs already built in our koji target f17-ruby. You can list them all at [1]. Good packages to look at for the cases you mention are:

  • rubygem-metaclass for pure Ruby gem
  • rubygem-json for Ruby gem with c extension
  • rubygem-gettext for Ruby gem with normal library
  • I'm not aware of any gem with c extension and a normal library.

[1] http://koji.fedoraproject.org/koji/builds?inherited=0&tagID=199&order=-build_id&latest=1

comment:14 in reply to: ↑ 8 Changed 2 years ago by bkabrda

Replying to toshio:

Replying to spot:

One more item: It looks like the gem_install block (for both noarch and arch gems) should be fine to call in %build (where it would be more appropriate) instead of %prep. Is there any reason that this is impossible or impractical?

We also analyzed this a bit further and realized that the %setup line isn't extracting the sources. The source isn't extracted until the "gem install" command is run.However, we may come back to this once we know what gem commands need to be run to patch rubygems that have C extensions/ if there's some alternate commands that allow us to extract the gem in %prep and then build it/install it to the proper looking hierarchy in a later step.

Basically, the steps to do that I already wrote in comment:11

comment:15 in reply to: ↑ 9 Changed 2 years ago by bkabrda

Replying to rathann:

All the tutorials for gems talk about gem build command. Why isn't it used here?

Gem build is used for creating Gems (as packages) from the source code files, so it's nothing that we need to cover.

comment:16 in reply to: ↑ 10 Changed 2 years ago by vondruch

Replying to bkabrda:

Replying to spot:

/usr/share/gems/gems/ just seems wrong to me. I still think that first "gems/" should be renamed to avoid confusion. Even if it was /usr/share/rubygems/.

Well, that is really just a matter of opinion. We think it's perfectly ok. BTW, /usr/share/rubygems contains the actual Rubygems library.

Well it used to be /usr/ruby/gems/1.8/gems, we just removed the version, so it is more obvious that the "gems" is repeated twice. I see no point to rename it.

comment:17 in reply to: ↑ 12 Changed 2 years ago by vondruch

Replying to bkabrda:

Replying to spot:

One more item: It looks like the gem_install block (for both noarch and arch gems) should be fine to call in %build (where it would be more appropriate) instead of %prep. Is there any reason that this is impossible or impractical?

AFAIK the %patch macros only work in the %prep section, so we would need to apply the patches manually in %build - so it would be impractical.

Or you can call it in %install section ;). We chosen %prep, so in the case you need to apply patch later, you don't have to change "whole" spec.

comment:18 Changed 2 years ago by spot

Okay, after much discussion, we need you to add a section to the draft which discusses how to patch ruby packages, specifically covering:

  • Patching architecture-independent packages in %prep after %gem_install.
  • Patching architecture dependent C code, then rebuilding that C code, in %prep, after %gem_install

This section must contain explicit examples of how to accomplish these tasks, because while we understand the logic here, we have no idea how to tell a gem to rebuild the patched C code.

Once that is done, we should be able to proceed on approving this draft.

comment:19 Changed 2 years ago by spot

Ignore Comment 18. I don't know how I missed that section. If it was a snake, it would have bit me.

comment:20 follow-up: ↓ 21 Changed 2 years ago by vondruch

It makes much sense, so we added it later on your request and that might confused you.

To be honest, there is still missing one scenario: if you cannot install the gem, because the binary extension is incompatible with the Ruby. And that situation is not well supported. I tried to ask upstream once and I got this response [1]. On the other hand, after rebuild of almost all RubyGems? because of Ruby 1.9.3, I am not aware yet of any gem which would need this treatment (although there are signs that rubygem-idn might be first candidate).

[1] http://help.rubygems.org/discussions/problems/584-how-to-install-binary-gem-without-the-build-step

comment:21 in reply to: ↑ 20 Changed 2 years ago by vondruch

Replying to vondruch:

It makes much sense, so we added it later on your request and that might confused you.

To be honest, there is still missing one scenario: if you cannot install the gem, because the binary extension is incompatible with the Ruby. And that situation is not well supported. I tried to ask upstream once and I got this response [1]. On the other hand, after rebuild of almost all RubyGems? because of Ruby 1.9.3, I am not aware yet of any gem which would need this treatment (although there are signs that rubygem-idn might be first candidate).

[1] http://help.rubygems.org/discussions/problems/584-how-to-install-binary-gem-without-the-build-step

I have updated a bit the structure of the "patching" guidelines and I also added the last case mentioned above. So now it should be quite exhaustive.

comment:22 Changed 2 years ago by toshio

Since none of the current FPC are ruby programmers, this is going to be hard to update. Can someone who is familiar with ruby and the guideline update come to our next meeting?

The current thing I'm looking at that needs a ruby programmer is that we tried to clarify the deprecation of gem and non-gem packaging here:

https://fedoraproject.org/wiki/PackagingDrafts/Ruby#Packaging_for_Gem_and_non-Gem_use

Unfortunately, this is not correct in that the syntax require ('rubygem') does not work. Also, I'm not sure from some of what I've read in the above comments if what's being said in the comments above is that rubygem is being shipped with ruby now or rubygem is being require'd now (and thus, the ruby path is being setup transparently).

I'm going to go through and reorganize the draft now to match other guidelines.

It's a shame that we couldn't have looked at normalizing the site/vendor macros with other languages as well. python, perl, etc have LANG_sitearch -- only ruby has LANG_sitearchdir.

comment:23 Changed 2 years ago by toshio

Is there a reason to include instructions for Packaging for Gem and non-Gem use at all? If you can do a simple require 'ruby package that may or may not be a gem' now then it seems like we don't have any reason to package old packages in non-gem format now. We can strip that section to just the disclaimer.

comment:24 Changed 2 years ago by toshio

Question: It looks like this change in guidelines will only affect the ruby-1.9.x. Does that mean we need to keep a copy of the old guidelines around for EPEL and Fedora < 17?

comment:25 Changed 2 years ago by toshio

The https://fedoraproject.org/wiki/PackagingDrafts/Ruby#Binary_Extension_Fails_to_Build section that was recently added looks like it's actually the procedure that we want to be following all the time. So the rules for building gems in all cases would then look like this:

%prep
%setup -q -c -T
pushd ..
gem unpack %{SOURCE0}

pushd %{gem_name}-%{version}
gem spec %{SOURCE0} -l --ruby > %{gem_name}.gemspec

# Optional
%patch0 -p1

gem build %{gem_name}.gemspec
popd
popd

%build
mkdir -p ./%{gem_dir}
export CONFIGURE_ARGS="--with-cflags='%{optflags}'"
gem install -V --local --install-dir ./%{gem_dir} \
	--force	--rdoc ../%{gem_name}-%{version}/%{gem_name}-%{version}.gem

%install
mkdir -p %{buildroot}%{gem_dir}
mkdir -p %{buildroot}%{gem_extdir}/foo
cp -a .%{gem_dir}/* %{buildroot}%{gem_dir}/

mv %{buildroot}%{gem_instdir}/foo/shared_object.so %{buildroot}%{gem_extdir}/foo/
  • This simplifies the guidelines for packagers and reviewers as there's only one path to follow (no decisions for binary vs noarch; patching vs non-patching.)
  • This properly breaks up the steps for building the package into %prep, %build, and %install which has been a longstanding issue with the guidelines. One benefit of breaking it up like this is rpmbuild [-bi | -bc] --short-circuit will finally work as expected with rubygems

comment:26 follow-up: ↓ 27 Changed 2 years ago by toshio

Why was the rubygems package moved from {site,vendor}libdir into /usr/share/rubygems?

comment:27 in reply to: ↑ 26 Changed 2 years ago by toshio

Replying to toshio:

Why was the rubygems package moved from {site,vendor}libdir into /usr/share/rubygems?

Okay... I see the note in the spec file for this which says that you want the rubygem module to be shared between different implementations of the ruby interpreter (for instance, jruby -- which currently bundles its own copy of rubygems since rubygems is also in the ruby stdlib). IIUC, though, with the new rubygem guidelines, the idea is that rubygems can be shared between different ruby implementations. If that's correct, then shouldn't the same be done with vendorlib rather than special-casing just rubygems?

comment:28 Changed 2 years ago by toshio

I've been working on the guideline this morning -- structural changes, I just went ahead and did. Many of the changes I mentioned as being possibly desirable here, I marked in the new page with an admon/question template to either confirm or to think about replacing some section with another that I've written.

I've only gotten to the gem building section so far. Need to merge the rest of the gem building and patching sections with it and then see if we can replace all those other sections with the one I wrote.

I'm abadger1999 on irc.freenode.net if anyone wants to give me guidance on this stuff. I'll be in and out this weekend (I'm out of time for today) but the packaging meeting is on Wednesday so it would be good to catch me before the meeting next week.

comment:29 follow-up: ↓ 30 Changed 2 years ago by vondruch

Ugh, a lot of feedback. Thanks Toshio. I'll try to to answer here.

  • Packaging for Gem and non-Gem use - There were suggestions to completely remove the section. However, there are a few packages which provides also nongem packages for Gems. Therefore, I think we should have it in guidelines mentioned as deprecated practice and prohibit it for new reviewed packages, but if you believe that it is just confusing, then I am fine with removal of this chapter.
  • Site/vendor macros - Ruby itself is using sitelibdir, sitearchdir and similarly for vendor, also, in previous guidelines there were typically used gemdir and quite common, there was used macro gemsintdir. That is why we went with the dir suffix. Not sure if and how to normalize it with other languages. Please note that we also prefer vendor instead of site, which I believe is preferred by other languages, although it is wrong IMO.
  • Guidelines for R1.9 - You are right that the new guidelines target just the Ruby 1.9. If we could keep copy of the old guidelines somewhere around, this would be nice. On the other hand, for example the macros we are using now, which are provided by rubygems-devel were backported by mtasaka also for Fedoras < F17, so they may be used also in older Fedoras. You can take look on rubygem-nokogiri's spec file [1] for example.
  • Binary Extension Fails to Build - I strongly disagree to follow this path always. As I said, from more than 200 gems, I know just about one which needs this procedure. Moreover, it is not as straight forward as it might look. Just for the one gem I know, I needed to tweak this approach [2]. So suggest to do something what doesn't work universally and is rarely used makes no sense to me. Previously, a lot of gems were using just %install section, now we suggest to always use also the %prep section, because it was widely used and if not in time of review, then some time later, but the %build section is typically useless.
  • /usr/share/rubygems - Firs of all, you are right that the idea is to share RubyGems? library and the Gems between implementation. On the other hand, it is in theoretical level yet. Nevertheless JRuby currently breaks packaging guidelines and I doubt there were approved exception. Regarding the vendorlib, there are several things I'd like to point out: (1) We try to limit usage of vendorlib as much as possible. I don't like to see any new packages to be reviewed which goes to vendor lib. That is not the way which Ruby development goes, at least since the RubyGems? become standard part of Ruby. There is just a few, old and typically unmaintained libraries which are not provided as a Gem. (2) Rubygems provides mechanisms, how to install more Gems of some version for different platforms in parallel, e.g. if you install nokogiri, it ends up in nokogiri-1.5.0 for x86 and nokogiri-1.5.0-java for jruby. There is no mechanism how to achieve it for vendorlib. This might work for noarch packages, but will bring us issues for arch dependent libraries.

[1] http://pkgs.fedoraproject.org/gitweb/?p=rubygem-nokogiri.git;a=blob;f=rubygem-nokogiri.spec;h=15b9856a2945b8ee06d5abc71b0f74e389ca9eaa;hb=HEAD

[2] http://vondruch.fedorapeople.org/rubygem-idn-0.0.2-4.fc18.src.rpm

comment:30 in reply to: ↑ 29 ; follow-up: ↓ 31 Changed 2 years ago by toshio

Replying to vondruch:

  • Packaging for Gem and non-Gem use

Committed

  • Site/vendor macros - Ruby itself is using sitelibdir, sitearchdir and similarly for vendor>

python doesn't have the concept of vendorlib. perl is using perl_vendorlib. What do you mean by RUby itself is using sitelibdir and sitearchdir? Are those variables available as public API to ruby somehow? If so, I would agree that that would be a valid thing to sync against rather than other language's macros.

  • Guidelines for R1.9

I've added an admonition link to the top of the new guidelines for older Fedora and RHEL. If you also want to work on updating the old guidelines, you can... but my recommendation is that there'd be little benefit for the work and you'd have to check whether the changes work on both older Fedora and RHEL. So I probably wouldn;t invest the time myself.

  • Binary Extension Fails to Build - I strongly disagree to follow this path always. As I said, from more than 200 gems, I know just about one which needs this procedure. Moreover, it is not as straight forward as it might look. Just for the one gem I know, I needed to tweak this approach [2].

I've looked at that package and I don't see any functional differences between what is in that srpm and what is in https://fedoraproject.org/wiki/PackagingDrafts/Ruby#Building_gems

Can you point out what I'm missing?

  • /usr/share/rubygems - Firs of all, you are right that the idea is to share RubyGems? library and the Gems between implementation. On the other hand, it is in theoretical level yet. Nevertheless JRuby currently breaks packaging guidelines and I doubt there were approved exception. Regarding the vendorlib, there are several things I'd like to point out: (1) We try to limit usage of vendorlib as much as possible. I don't like to see any new packages to be reviewed which goes to vendor lib. That is not the way which Ruby development goes, at least since the RubyGems? become standard part of Ruby. There is just a few, old and typically unmaintained libraries which are not provided as a Gem. (2) Rubygems provides mechanisms, how to install more Gems of some version for different platforms in parallel, e.g. if you install nokogiri, it ends up in nokogiri-1.5.0 for x86 and nokogiri-1.5.0-java for jruby. There is no mechanism how to achieve it for vendorlib. This might work for noarch packages, but will bring us issues for arch dependent libraries.

I don't see an actual argument for why vendorlib/sitelib shouldn't be interpreter independent if gems are independent in the first half (through #1). For #2, I'm not sure what you're driving at. This is what I'm getting out of it: nokogiri builds on all archs. It creates a nokogiri-1.5.0.x86 w/ nokogiri-jruby-1.5.0.noarch and a nokogiri-1.5.0.x86_64 w/ nokogiri-jruby-1.5.0.noarch. I'm assuming that the next step is the problem?

The problem that I'm seeing is that the rubygems package itself is currently being special-cased for no apparent reason. It doesn't seem like there's any reason that all of vendorlib, not just rubygems, can't be shared between ruby interpreters if gems are already shared and rubygems itself can be shared.

[1] http://pkgs.fedoraproject.org/gitweb/?p=rubygem-nokogiri.git;a=blob;f=rubygem-nokogiri.spec;h=15b9856a2945b8ee06d5abc71b0f74e389ca9eaa;hb=HEAD

[2] http://vondruch.fedorapeople.org/rubygem-idn-0.0.2-4.fc18.src.rpm

comment:31 in reply to: ↑ 30 ; follow-up: ↓ 32 Changed 2 years ago by vondruch

Replying to toshio:

Replying to vondruch:

  • Site/vendor macros - Ruby itself is using sitelibdir, sitearchdir and similarly for vendor>

python doesn't have the concept of vendorlib. perl is using perl_vendorlib. What do you mean by RUby itself is using sitelibdir and sitearchdir? Are those variables available as public API to ruby somehow? If so, I would agree that that would be a valid thing to sync against rather than other language's macros.

See the previous version of guidelines [1], e.g. "puts Config::CONFIG[ 'sitelibdir' ]"

  • Guidelines for R1.9

I've added an admonition link to the top of the new guidelines for older Fedora and RHEL. If you also want to work on updating the old guidelines, you can... but my recommendation is that there'd be little benefit for the work and you'd have to check whether the changes work on both older Fedora and RHEL. So I probably wouldn;t invest the time myself.

No, I don't want to :)

  • Binary Extension Fails to Build - I strongly disagree to follow this path always. As I said, from more than 200 gems, I know just about one which needs this procedure. Moreover, it is not as straight forward as it might look. Just for the one gem I know, I needed to tweak this approach [2].

I've looked at that package and I don't see any functional differences between what is in that srpm and what is in https://fedoraproject.org/wiki/PackagingDrafts/Ruby#Building_gems

Can you point out what I'm missing?

You are missing the "sed" line. This is what may bite somebody. I had to find solution how to make the "gem build" actually work. I really don't like to fight with this kind of issues. Moreover, if you are packaging such gem, it should warn you, that something is really bad. Why it does not work? The rubygem-idn does not work, because it is not compatible with Ruby 1.9.3 nor maintained by upstream. Do we really want to support such unmaintained libraries? My answer is NO, with some exceptions. You are asking for more troubles than you solve. This is workaround if you are in troubles, not a standard supported solution.

  • /usr/share/rubygems - Firs of all, you are right that the idea is to share RubyGems? library and the Gems between implementation. On the other hand, it is in theoretical level yet. Nevertheless JRuby currently breaks packaging guidelines and I doubt there were approved exception. Regarding the vendorlib, there are several things I'd like to point out: (1) We try to limit usage of vendorlib as much as possible. I don't like to see any new packages to be reviewed which goes to vendor lib. That is not the way which Ruby development goes, at least since the RubyGems? become standard part of Ruby. There is just a few, old and typically unmaintained libraries which are not provided as a Gem. (2) Rubygems provides mechanisms, how to install more Gems of some version for different platforms in parallel, e.g. if you install nokogiri, it ends up in nokogiri-1.5.0 for x86 and nokogiri-1.5.0-java for jruby. There is no mechanism how to achieve it for vendorlib. This might work for noarch packages, but will bring us issues for arch dependent libraries.

I don't see an actual argument for why vendorlib/sitelib shouldn't be interpreter independent if gems are independent in the first half (through #1). For #2, I'm not sure what you're driving at. This is what I'm getting out of it: nokogiri builds on all archs. It creates a nokogiri-1.5.0.x86 w/ nokogiri-jruby-1.5.0.noarch and a nokogiri-1.5.0.x86_64 w/ nokogiri-jruby-1.5.0.noarch. I'm assuming that the next step is the problem?

The problem that I'm seeing is that the rubygems package itself is currently being special-cased for no apparent reason. It doesn't seem like there's any reason that all of vendorlib, not just rubygems, can't be shared between ruby interpreters if gems are already shared and rubygems itself can be shared.

Example might help here. If you are using plain gems, you can use "--platform" parameter to specify the gem for which platform you want to install. In this case RubyGems? will use following structure to prevent some collisions:

$ tree -d -L 2
.
|-- bin
|-- cache
|-- doc
|   |-- nokogiri-1.5.0
|   |-- nokogiri-1.5.0-java
|   |-- nokogiri-1.5.0-x86-mingw32
|   `-- nokogiri-1.5.0-x86-mswin32-60
|-- gems
|   |-- nokogiri-1.5.0
|   |-- nokogiri-1.5.0-java
|   |-- nokogiri-1.5.0-x86-mingw32
|   `-- nokogiri-1.5.0-x86-mswin32-60
`-- specifications

Therefore you can install jruby packages together with mri packages into one gems tree. RubyGems? later pickup the proper gem for the interpreter, depending on the platform. You are not able to achieve this in vendorlib, because it was never thought of.

Moreover, at these days, there really is not plethora of libraries which are not provided as gems. Take a look on ruby- packages in Fedora and you'll see that there is typically available its gem equivalent. I am working with Ruby more than 6 years now and I don't remember any Ruby library I would ever need to install into site/vendor libdir (except RubyGems? itsefl).

  • ruby vs rubygems provides - I like the idea to have just ruby() provide, however I am afraid that it is too late for F17 and I am not sure about all possible implications. There are some for sure (1) There is no way how to prioritize rubygem- over ruby- packages. (2) There are other ruby libraries, which can take your gems and vendorize them into application you are developing, which is definitely not possible if the you have ruby- library instead of rubygem- (3) Rubygems has metadata, if you require one gem, it loads also the gem's dependencies. (4) You proposed to have provide like ruby(requiredfilename), however there are libraries, where you need to require them using "require 'foo/bar'". Typical example is the rubygem-net-* family. There might be others (may ber rubygem-rack-* ?). And I could continue. I propose to revert the provides to ruby(libraryname)/rubygems(gem_name) and postpone it, lets say for F19?

[1] https://fedoraproject.org/wiki/Packaging:Ruby#Build_Architecture_and_File_Placement

comment:32 in reply to: ↑ 31 ; follow-ups: ↓ 33 ↓ 34 Changed 2 years ago by toshio

Replying to vondruch:

  • Site/vendor macros - Ruby itself is using sitelibdir, sitearchdir and similarly for vendor>

See the previous version of guidelines [1], e.g. "puts Config::CONFIG[ 'sitelibdir' ]"

Okay, taken out.

  • Binary Extension Fails to Build - I strongly disagree to follow this path always. As I said, from more than 200 gems, I know just about one which needs this procedure. Moreover, it is not as straight forward as it might look. Just for the one gem I know, I needed to tweak this approach [2].

I've looked at that package and I don't see any functional differences between what is in that srpm and what is in https://fedoraproject.org/wiki/PackagingDrafts/Ruby#Building_gems

Can you point out what I'm missing?

You are missing the "sed" line. This is what may bite somebody. I had to find solution how to make the "gem build" actually work. I really don't like to fight with this kind of issues. Moreover, if you are packaging such gem, it should warn you, that something is really bad. Why it does not work? The rubygem-idn does not work, because it is not compatible with Ruby 1.9.3 nor maintained by upstream. Do we really want to support such unmaintained libraries? My answer is NO, with some exceptions. You are asking for more troubles than you solve. This is workaround if you are in troubles, not a standard supported solution.

Okay -- looking at this, it seems like a case where upstream's code is wrong. So fixing it from our spec file and submitting a bug report upstream should upstream be active looks like exactly the right thing to do.

  • /usr/share/rubygems

Your example tree is making me even more cautious... You're thinking that that tree is going to show up in %gem_dir, not in %gem_extdir correct? And it's only going to show up for ruby gems that have extensions (not pure ruby)?

Moreover, at these days, there really is not plethora of libraries which are not provided as gems. Take a look on ruby- packages in Fedora and you'll see that there is typically available its gem equivalent. I am working with Ruby more than 6 years now and I don't remember any Ruby library I would ever need to install into site/vendor libdir (except RubyGems? itsefl).

This portion doesn't really have any bearing on the discussion. Or if it does... maybe it's that vendorlib should be fine to treat this way since the only thing that hsould be in it is rubygems itself.

  • ruby vs rubygems provides - I like the idea to have just ruby() provide, however I am afraid that it is too late for F17

First: are you talking below about my proposal for changing the usage of ruby() vs rubygem() or about having ruby() only? I'm assuming the former so some of these won't apply if you mean the latter.

If it's too late for F17 then you'll need to push back your ruby-1.9.3 Feature until F18 as this is one of the things that the Feature says needs to be done for the Feature to go ahead. I'd rather see this finished but there does seem to be a lot of work.

and I am not sure about all possible implications. There are some for sure (1) There is no way how to prioritize rubygem- over ruby- packages.

Requires: rubygem(rubylibrary)

(2) There are other ruby libraries, which can take your gems and vendorize them into application you are developing, which is definitely not possible if the you have ruby- library instead of rubygem-

Not that we should be encouraging it, Requires: rubygem(rubylibrary) in that package would work as well.

(3) Rubygems has metadata, if you require one gem, it loads also the gem's dependencies.

and that won't change no matter what the spec file Requires: look like.

(4) You proposed to have provide like ruby(requiredfilename), however there are libraries, where you need to require them using "require 'foo/bar'". Typical example is the rubygem-net-* family. There might be others (may ber rubygem-rack-* ?).

I don't see how changing when rubygem() and when ruby() is used would cause these issues. Which makes me think that maybe when I changed when ruby() and when rubygem() is used, it introduced something unintended that's not clear. The thrust of the change to requires and provides that I want is that:

Provides: rubygems provide both a ruby() and a rubygem() symbol. non-gems provide only a ruby()

Requires: Requires can always use ruby() if the package does not require anything that can only be found via gem packaging (which as far as I know means, require'rubylibrary')

Requires may use rubygem() if the package needs to use rubygem metadata (for instance, gem 'rubylibrary' -- since there's more metadata here, there's likely other things that a package could be doing that requires that the rub library has gem metadata. Those would also fall here).

And I could continue. I propose to revert the provides to ruby(libraryname)/rubygems(gem_name) and postpone it, lets say for F19?

The ruby provides and requires are one of the three things (building and file placement are the others) that I think need to be fixed in this draft before it can be approved. So you can't push them back without pushing back the entire guidelines which also means pushing back the Ruby-1.9.3 feature.

comment:33 in reply to: ↑ 32 ; follow-up: ↓ 37 Changed 2 years ago by vondruch

Replying to toshio:

Replying to vondruch:

  • /usr/share/rubygems

Your example tree is making me even more cautious... You're thinking that that tree is going to show up in %gem_dir, not in %gem_extdir correct? And it's only going to show up for ruby gems that have extensions (not pure ruby)?

The tree is going to show up also in %gem_extdir, but only for gems which has binary extension.

Moreover, at these days, there really is not plethora of libraries which are not provided as gems. Take a look on ruby- packages in Fedora and you'll see that there is typically available its gem equivalent. I am working with Ruby more than 6 years now and I don't remember any Ruby library I would ever need to install into site/vendor libdir (except RubyGems? itsefl).

This portion doesn't really have any bearing on the discussion. Or if it does... maybe it's that vendorlib should be fine to treat this way since the only thing that hsould be in it is rubygems itself.

My point is that ideally, (1) vendorlib will not be used anymore, therefore it does not matter too much where it is placed (2) how would you treat library that has different implementation for JRuby and MRI when it is in single vendorlib? If you are using one vendorlib for all implementations, you have to assume that every library will run with every implementation, but that is true just as long as you are speaking about noarch libraries (and also there might be exceptions, since JRuby does not support fork for example). How will you deal with implementation specific libraries?

However, back to RubyGems?. If we would not care and go with upstream, the RubyGems? would be placed into /usr/share/ruby/rubygems, as they are originally bundled. We though also about moving them into vendorlib, but based on the (2), we decided to move them completely out of the Ruby's tree and also we decided to place the gems into separate tree.

  • ruby vs rubygems provides - I like the idea to have just ruby() provide, however I am afraid that it is too late for F17

First: are you talking below about my proposal for changing the usage of ruby() vs rubygem() or about having ruby() only? I'm assuming the former so some of these won't apply if you mean the latter. If it's too late for F17 then you'll need to push back your ruby-1.9.3 Feature until F18 as this is one of the things that the Feature says needs to be done for the Feature to go ahead. I'd rather see this finished but there does seem to be a lot of work.

and I am not sure about all possible implications. There are some for sure (1) There is no way how to prioritize rubygem- over ruby- packages.

Requires: rubygem(rubylibrary)

(2) There are other ruby libraries, which can take your gems and vendorize them into application you are developing, which is definitely not possible if the you have ruby- library instead of rubygem-

Not that we should be encouraging it, Requires: rubygem(rubylibrary) in that package would work as well.

(3) Rubygems has metadata, if you require one gem, it loads also the gem's dependencies.

and that won't change no matter what the spec file Requires: look like.

(4) You proposed to have provide like ruby(requiredfilename), however there are libraries, where you need to require them using "require 'foo/bar'". Typical example is the rubygem-net-* family. There might be others (may ber rubygem-rack-* ?).

I don't see how changing when rubygem() and when ruby() is used would cause these issues. Which makes me think that maybe when I changed when ruby() and when rubygem() is used, it introduced something unintended that's not clear. The thrust of the change to requires and provides that I want is that:

Provides: rubygems provide both a ruby() and a rubygem() symbol. non-gems provide only a ruby()

Requires: Requires can always use ruby() if the package does not require anything that can only be found via gem packaging (which as far as I know means, require'rubylibrary')

Requires may use rubygem() if the package needs to use rubygem metadata (for instance, gem 'rubylibrary' -- since there's more metadata here, there's likely other things that a package could be doing that requires that the rub library has gem metadata. Those would also fall here).

And I could continue. I propose to revert the provides to ruby(libraryname)/rubygems(gem_name) and postpone it, lets say for F19?

The ruby provides and requires are one of the three things (building and file placement are the others) that I think need to be fixed in this draft before it can be approved. So you can't push them back without pushing back the entire guidelines which also means pushing back the Ruby-1.9.3 feature.

Sorry, re-reading it once more and reading your comment, I was wrong. You proposed that Gems should have ruby() as well as rubygems() provides and I did not get that originally. However, there are implications I'd like to point out.

(1) If you require ruby(), it means that library is directly accessible in Ruby's load path and RubyGems? are out of the game.

(2) If you are requiring rubygem(), it means that RubyGems? are in play and therefore the library is not in Ruby's load path by default and also that RubyGems? might pick up different version of library which you have installed locally.

I don't find these two approaches directly interchangeable.

comment:34 in reply to: ↑ 32 ; follow-ups: ↓ 35 ↓ 36 ↓ 38 Changed 2 years ago by vondruch

Replying to toshio:

Replying to vondruch:

  • Binary Extension Fails to Build - I strongly disagree to follow this path always. As I said, from more than 200 gems, I know just about one which needs this procedure. Moreover, it is not as straight forward as it might look. Just for the one gem I know, I needed to tweak this approach [2].

I've looked at that package and I don't see any functional differences between what is in that srpm and what is in https://fedoraproject.org/wiki/PackagingDrafts/Ruby#Building_gems

Can you point out what I'm missing?

You are missing the "sed" line. This is what may bite somebody. I had to find solution how to make the "gem build" actually work. I really don't like to fight with this kind of issues. Moreover, if you are packaging such gem, it should warn you, that something is really bad. Why it does not work? The rubygem-idn does not work, because it is not compatible with Ruby 1.9.3 nor maintained by upstream. Do we really want to support such unmaintained libraries? My answer is NO, with some exceptions. You are asking for more troubles than you solve. This is workaround if you are in troubles, not a standard supported solution.

Okay -- looking at this, it seems like a case where upstream's code is wrong. So fixing it from our spec file and submitting a bug report upstream should upstream be active looks like exactly the right thing to do.

Toshio, I really want to convince you, that repackaging gem in %prep is bad practice, workaround, unsupported black magic etc. It introduces more problems then problems solves. Please don't make it mandatory. It's like requesting Fedora's users to always rebuild each RPM locally, because there might be sometimes broken dependency or other error.

I'm willing to propose upstream some nice "gem install" flag, which would allow to install gem even though the binary extension build was not successful, to solve this issue, but repackaging gems is real overkill.

comment:35 in reply to: ↑ 34 Changed 2 years ago by mtasaka

Replying to vondruch:

Toshio, I really want to convince you, that repackaging gem in %prep is bad practice, workaround, unsupported black magic etc. It introduces more problems then problems solves. Please don't make it mandatory. It's like requesting Fedora's users to always rebuild each RPM locally, because there might be sometimes broken dependency or other error.

I'm willing to propose upstream some nice "gem install" flag, which would allow to install gem even though the binary extension build was not successful, to solve this issue, but repackaging gems is real overkill.

Exactly. Please never think (currently) that repackaging gems like this way is supported and will work always.

comment:36 in reply to: ↑ 34 ; follow-up: ↓ 39 Changed 2 years ago by vondruch

Replying to vondruch:

Replying to toshio:

Replying to vondruch:

Please note that I asked Ruby-SIG about their opinions about this matter: http://lists.fedoraproject.org/pipermail/ruby-sig/2012-February/000886.html

There are also some general comments to the Guidelines in current state.

comment:37 in reply to: ↑ 33 Changed 2 years ago by toshio

Replying to vondruch:

Your example tree is making me even more cautious... You're thinking that that tree is going to show up in %gem_dir, not in %gem_extdir correct? And it's only going to show up for ruby gems that have extensions (not pure ruby)?

The tree is going to show up also in %gem_extdir, but only for gems which has binary extension.

So... that leaves a lot of questions about just what's in those trees.

  • There's an FHS concern: The directories themselves have platform names in them (nokogiri-1.5.0-x86-mingw32) which leads me to believe that they contain platform specific code. I think that what that means is that the code in there is only used by some platforms but that the rubygem mechanism is able to select the appropriate directory of code to use for the platform that it's currently running on. I think that this is contrary to the letter of the FHS ("/usr/share is for read-only architecture independent data files") but I'm not sure that it violates the spirit of the FHS or if that will matter. The full FPC will need to talk about that. Could you give more information about all the files that end up in these trees?
  • The second one is about having gems for multiple interpreters. But I'll keep it merged with the next section:

This portion doesn't really have any bearing on the discussion. Or if it does... maybe it's that vendorlib should be fine to treat this way since the only thing that hsould be in it is rubygems itself.

My point is that ideally, (1) vendorlib will not be used anymore, therefore it does not matter too much where it is placed

This is my point too. If no one's using vendorlib except for rubygems, then it seems like it's fine to put rubygems in it and move vendorlib to an interpreter-neutral location if your goal is to share the rubygems module among different interpreters.

(2) how would you treat library that has different implementation for JRuby and MRI when it is in single vendorlib? If you are using one vendorlib for all implementations, you have to assume that every library will run with every implementation, but that is true just as long as you are speaking about noarch libraries (and also there might be exceptions, since JRuby does not support fork for example). How will you deal with implementation specific libraries?

Right. So this is where it ties into the above section's question. How are gems and gem rpm packaging handling this? When does having the location be the same for different interpreters and platforms make sense and when does it not make sense (How does a shared location benefit gems when there will be different versions of modules anyway [I'm thinking the answer would be -- there's different versions only in a few instances but I'm seeking confirmation])?

However, back to RubyGems?. If we would not care and go with upstream, the RubyGems? would be placed into /usr/share/ruby/rubygems, as they are originally bundled. We though also about moving them into vendorlib, but based on the (2), we decided to move them completely out of the Ruby's tree and also we decided to place the gems into separate tree.

Is /usr/share/ruby interpreter independent or interpreter dependent?

  • ruby vs rubygems provides - [...]

Sorry, re-reading it once more and reading your comment, I was wrong. You proposed that Gems should have ruby() as well as rubygems() provides and I did not get that originally. However, there are implications I'd like to point out.

(1) If you require ruby(), it means that library is directly accessible in Ruby's load path and RubyGems? are out of the game.

Yes to the first half of that but no too the second half of that. At least, the understanding the FPC was given of the situation is that this is not needed as seen in the present contents of "Packaging for Gem and Non-gem usage" https://fedoraproject.org/wiki/PackagingDrafts/Ruby#Packaging_for_Gem_and_non-Gem_use

So everyone is clear, here's what I tested:

$ irb
irb(main):001:0> require 'json'
LoadError: no such file to load -- json
        from (irb):1:in `require'
        from (irb):1
        from :0
irb(main):002:0> require 'rubygems'
=> true
irb(main):003:0> require 'json'
=> true

The packaging as Gem/Non?-gem portion of the guidelines existed so that require 'json' worked. The argument that the draft guidelines seemed to be making was that separate non-gem subpackages were no longer needed to make that work as long as the ubiquitous requirement, rubygems, was required by this particular script/application/library. Adding a patch, if necessary, that simply did <code>require 'rubygems'</code> seems fine.

Is there anything I'm missing?

(2) If you are requiring rubygem(), it means that RubyGems? are in play and therefore the library is not in Ruby's load path by default and also that RubyGems? might pick up different version of library which you have installed locally.

The first part of this is related to the above explanation about Requires: ruby(). The second part is true. Requires: rubygem() can be seen as a superset of the functionality provided by Requires: ruby().

  • Requires: ruby() -- The module will be available when require 'rubylibrary' is used.
  • Requires: rubygem() -- In addition, the module may have specific version requirements or be used in other ways that require the extra metadata that rubygems provide.

comment:38 in reply to: ↑ 34 Changed 2 years ago by toshio

Replying to vondruch:

Toshio, I really want to convince you, that repackaging gem in %prep is bad practice, workaround, unsupported black magic etc. It introduces more problems then problems solves. Please don't make it mandatory. It's like requesting Fedora's users to always rebuild each RPM locally, because there might be sometimes broken dependency or other error.

You'll need to actually provide us with examples of it not working that we can analyze then. At the moment, there's only one example and that one has a problem that it is entirely appropriate for Fedora to fix and submit upstream if Fedora wants to be giving back to the upstream communities.

The current guidelines about gems are a *very* ugly hack that lutter came up with for the FPC because there seemed to be no other way to get gems into packages. FPC was reluctant to pass them but with the assurance that no better way existed, we did. I believe we even asked lutter to keep looking into other ways to make it work after we passed them. Even at the last FPC meeting where we discussed the Guidelines in general, the way that gems apparently required non-standard methods was contentious. Now that a way exists that actually conforms to the way that rpms are supposed to be built, we're very likely to switch to it even if there are minor difficulties with it. You'll need to show us examples of all the varieties of what can go wrong for us to evaluate whether there is, in fact, unaddressable problems with the strategy.

I'm willing to propose upstream some nice "gem install" flag, which would allow to install gem even though the binary extension build was not successful, to solve this issue, but repackaging gems is real overkill.

So, here's pseudo-code for the general way gem commands should look in order to be properly built as rpms:

%prep
%setup -Tqn %{gem_name}-%{gem_version}
gem UNARCHIVE

%patch0

%build
gem BUILDONLY --cflags='%{optflags}' --bindir=%{_bindir} [...]

%install
gem INSTALLONLY --destdir=%{buildroot}

The thing is, we now have non-optimal forms of those commands. UNARCHIVE is roughly equivalent to gem unpack. BUILDONLY can be simulated with gem install and the proper flags and environment variables. INSTALLONLY can be made to work by copying the directories created in the previous step to the proper places in the %{buildroot}.

Your proposed gem install flag totally misses the point of the changes and wouldn't be any closer to achieving this version of the spec file instructions than what we're building now. In fact, a flag that does not fail if a C extension fails to build is more likely to be something that FPC says SHOULD or MUST not be used than something that would be useful.

comment:39 in reply to: ↑ 36 Changed 2 years ago by toshio

Please note that I asked Ruby-SIG about their opinions about this matter: http://lists.fedoraproject.org/pipermail/ruby-sig/2012-February/000886.html

There are also some general comments to the Guidelines in current state.

If you could get this conversation onto the https://lists.fedoraproject.org/mailman/listinfo/packaging mailing list that would be the proper mailing list both in terms of topic and because it will allow everyone interested in Guidelines work to comment as well.

I'm also available on IRC as stated earlier. The way we're using this ticket right now is making it pretty hard to read.

comment:40 Changed 2 years ago by mmaslano

  • Cc mmaslano added

comment:41 follow-up: ↓ 43 Changed 2 years ago by toshio

Draft updated: https://fedoraproject.org/wiki/User:Toshio/RubyPackagingDraft

Since no one was willing to give us packages that don't work with the new guidelines, I just selected a few packages and updated them to use the gem unpack method of building: http://toshio.fedorapeople.org/rubygems/.

The common case is to add three lines, modify two others, and move what gets done in %build vs %prep. I had to further modify one package's spec file which had a find that was catching directories when it should only be operating on files (added -type f). Thanks to macros for gem_name and other package info, these changes don't even vary from package to package. The one package that had patches applied went from 11 lines of commands down to 9 for the build step. Reorganizing that section also made it obvious that the unittests were being installed in the binary rpm under the old code, a problem that the new code did not share. The application example also became simpler by two lines.

I did not modify the Requires/Provides? with the test builds I did. Those changes don't seem to be controversial as to whether they break building; only if they break code on the user's systems.

In giving the guidelines the last bit of testing and polish I did find some more unanswered questions with interpreter independence.

  • It seems that the ruby(abi) Provides is only in ruby-libs; not in jruby{,-libs}. That means that an end user wanting to use jruby and having only rubygems installed still needs to have ruby-libs installed (because of the dep on ruby(abi).) Is this intentional?
  • There needs to be more information about jruby and other alternate interpreters. In particular, the path macros for the jruby non-gem packages need to be added.
  • I took a guess that non-gem packages should have separate subpackages for each interpreter since the thought is that rubygems can handle multiple interpreters but non-gem requires cannot. Further, I guessed that they should be named INTERPRETER-module, for instance, jruby-rpm.

comment:42 Changed 2 years ago by spot

Toshio's latest draft will be considered at the next FPC meeting.

comment:43 in reply to: ↑ 41 Changed 2 years ago by bkabrda

Replying to toshio:

Draft updated: https://fedoraproject.org/wiki/User:Toshio/RubyPackagingDraft

Since no one was willing to give us packages that don't work with the new guidelines, I just selected a few packages and updated them to use the gem unpack method of building: http://toshio.fedorapeople.org/rubygems/.

The common case is to add three lines, modify two others, and move what gets done in %build vs %prep. I had to further modify one package's spec file which had a find that was catching directories when it should only be operating on files (added -type f). Thanks to macros for gem_name and other package info, these changes don't even vary from package to package. The one package that had patches applied went from 11 lines of commands down to 9 for the build step. Reorganizing that section also made it obvious that the unittests were being installed in the binary rpm under the old code, a problem that the new code did not share. The application example also became simpler by two lines.

Well, it's not that they wouldn't work. It's just that you have to actually patch the spec (that you get from gem spec foo.gem) with some sedding (sometimes), which doesn't seem to be a very clean way. Also, there are 250 gems, so it would be quite hard to apply you approach to all of them to see if everything works. I tried handful of them and your approach worked with some sedding, as I said. I still don't like the new approach very much, because it will make it harder to get Ruby people to RPM packaging, but I understand your strict packaging point of view. The Ruby SIG has always looked at that from a half packaging/half Ruby perspective, which is the main cause of our reasoning. So from my point of view, ok, do it this way.

I did not modify the Requires/Provides? with the test builds I did. Those changes don't seem to be controversial as to whether they break building; only if they break code on the user's systems.

As for the Provides, I still have to insist on rubygems providing only rubygem(foo) and not ruby(foo). If I looked correctly, you deleted the section about mandatory creating non-gem subpackages - thanks for that. Now, we have to distinguish between non-gem library and rubygem. Here is my opinion:

The non-gem package ruby-foo should have Provides: ruby(foo)

The Gem package rubygem-bar should have Provides: rubygem(bar), but shouldn't have Provides: ruby(bar).

It even makes more sense now, that the non-gem subpackages of gems won't exist. The provide will let you know, if the package is requirable (in the Ruby terms, not packaging terms) without having rubygems enabled, which is an important feature. (E.g. if the user runs "yum install 'ruby(foo)'", then he knows, that this package will be requirable even with rubygems disabled. Also, we can get to a situation (however unlikely) of having a library with a same name both as a gem and as a non-gem, but with different functionality. Then having Provides: ruby(foo) in the gem version would obviously break things.

In giving the guidelines the last bit of testing and polish I did find some more unanswered questions with interpreter independence.

  • It seems that the ruby(abi) Provides is only in ruby-libs; not in jruby{,-libs}. That means that an end user wanting to use jruby and having only rubygems installed still needs to have ruby-libs installed (because of the dep on ruby(abi).) Is this intentional?

The ultimate plan is for JRuby to provide ruby(abi) and use the system rubygems, but we are still facing some issues: JRuby ships with it's own modified version of rubygems (not currently compatible with the Rubygems upstream; please note that this is not a case of bundling, but rather a fork), and we are currently trying to solve this situation. I am working intensively with the JRuby upstream trying to help them find a way to push their alterations to Rubygems upstream, so that we can really have only one universal system Rubygems (this involves some hooks here and there, alterations of gem building mechanism etc.). It seems doable, but it will probably get to F18 (if everything goes well). So we are currently sticking with the modified rubygems inside JRuby, that operate on their own via jgem command.

  • There needs to be more information about jruby and other alternate interpreters. In particular, the path macros for the jruby non-gem packages need to be added.

For the time being, we only have one alternative interpreter - JRuby. It's path's are gonna be a little problem, but I am discussing this with the upstream as well, so it should get fixed and macroed to F18. For F17, there are no JRuby-specific packages anyway, and everyone can still use the old approach that we used in the previous ruby guidelines (e.g. running jruby interactively to get the path from them). JRuby has been working like this since it's initial packaging into Fedora, so I think we can postpone it for one more release.

  • I took a guess that non-gem packages should have separate subpackages for each interpreter since the thought is that rubygems can handle multiple interpreters but non-gem requires cannot. Further, I guessed that they should be named INTERPRETER-module, for instance, jruby-rpm.

Yep, that should probably be the ideal state - but there should also be an explicit note, that the package _must_ be tested with JRuby too, because these packages are often only created for the MRI Ruby and might not work with JRuby, so the packager should pay a special attention to this.

comment:44 Changed 2 years ago by spot

  • Owner set to spot
  • Status changed from new to assigned

#action Draft approved (+1:6, 0:0, -1:0)

comment:45 Changed 2 years ago by spot

  • Status changed from assigned to closed
  • Resolution set to fixed

Announce text:

The Ruby Packaging Guidelines were almost completely rewritten. If you maintain ruby packages in Fedora, we advise that you review the new guidelines.

https://fedoraproject.org/wiki/Packaging:Ruby

Note: See TracTickets for help on using tickets.