Ticket #76 (closed defect: fixed)

Opened 3 years ago

Last modified 13 months ago

Mention new filtering mechanisms in AutoProvidesAndRequiresFiltering

Reported by: tibbs Owned by:
Priority: minor Milestone:
Component: Guideline Draft Version:
Keywords: meeting Cc: mmaslano@…, alexlan, ppisar
Blocked By: Blocking:

Description

rpm 4.9.0 has new (and I presume better) mechanisms for filtering dependencies. My understanding is that these don't suffer from the problems our existing macros have with respect to disabling the internal dependency generator and the issues surrounding that.

However, I can't find any documentation at all on this besides this message: http://lists.fedoraproject.org/pipermail/devel/2011-February/148317.html

Since F15 has a sufficiently new RPM, it may be worth mentioning these on https://fedoraproject.org/wiki/Packaging:AutoProvidesAndRequiresFiltering

It also occurs to me that at least %perl_default_filter could be updated in F15+ to use the new-style stuff.

Change History

comment:1 Changed 3 years ago by tibbs

  • Keywords meeting added

comment:2 follow-up: ↓ 7 Changed 3 years ago by tibbs

Figured I should mention that the current draft is at https://fedoraproject.org/wiki/User:Tibbs/AutoProvidesAndRequiresFiltering. Open questions that need answers are prefixed in the draft with "XXX".

comment:3 follow-up: ↓ 5 Changed 3 years ago by mmaslano

  • Cc mmaslano@… added

Could you please add examples of filtering into documentation and mention that the old one (sometimes) doesn't work? If filters for more requires are used, then it's needed |. Also regexp can't use perl regexps, but only extended regexps.

For example Perl: /etc/rpm/macros.perl must be changed to something like:

%perl_default_filter %{expand: \
%global __provides_exclude_from %{perl_vendorarch}/auto/.*\.so|%{perl_archlib}/.*\.so|%{_docdir}
%global __requires_exclude_from %{_docdir}
%global __provides_exclude perl\\\\(VMS|perl\\\\(Win32|perl\\\\(BSD::|perl\\\\(DB\\\\)|perl\\\\(UNIVERSAL\\\\)
%global __requires_exclude perl\\\\(VMS|perl\\\\(BSD::|perl\\\\(Win32
}

If there is needed another filtering in module, it must be used in this way:

%perl_default_filter
%global __provides_exclude %{?__provides_exclude}|perl\\(DBI\\)
%global __requires_exclude %{?__requires_exclude}|perl\\(RPC::

All requires or provides must be on one line, otherwise it won't work.

comment:4 Changed 3 years ago by jjames

After playing around with this for a bit, I've discovered that literal parentheses need 3 backslashes, not 4 or 2 as in the above examples; e.g.:

%global requires_exclude ocaml
\(Sig
\)

This is going to mean trouble for macros that expand to strings containing non-literal POSIX extended regexp characters, like +, (, ), ., etc. A "raw string" format would both solve that problem and make the example above more readable. The language does appear to be extended POSIX regexps; I see references to the symbols "regcomp" and "regexec" in librpm, FWIW.

Also, either %global or %define can be used, so %global should be preferred.

comment:5 in reply to: ↑ 3 Changed 3 years ago by iarnell

Replying to mmaslano:

If there is needed another filtering in module, it must be used in this way:

%perl_default_filter
%global __provides_exclude %{?__provides_exclude}|perl\\(DBI\\)
%global __requires_exclude %{?__requires_exclude}|perl\\(RPC::

I think this is a potentially dangerous recommendation. If %__provides_exclude (or requires, of course) isn't already defined, this expands to

%global __provides_exclude |perl\\(DBI\\)

And due to the | operator, ends up matching (and excluding) everything. To avoid this happening, I would recommend being more explicit in the spec file

%global __provides_exclude %{?__provides_exclude:%__provides_exclude|}perl\\(DBI\\)

All requires or provides must be on one line, otherwise it won't work.

After initially defining (globalling?) __provides_exclude, it can be extended, e.g.

%global __provides_exclude %{?__provides_exclude:%__provides_exclude|}perl\\(DBI\\)
%global __provides_exclude %__provides_exclude|perl\\(foo\\)

comment:6 Changed 3 years ago by ppisar

Current perl_default_filter ignores pre-defined __*_exclude* macros, so one needs to call %perl_default_filter first and then append values to __*_exclude* macros in package spec file. Otherwise the values defined in package spec file will get lost.

comment:7 in reply to: ↑ 2 Changed 3 years ago by iarnell

Replying to tibbs:

Figured I should mention that the current draft is at https://fedoraproject.org/wiki/User:Tibbs/AutoProvidesAndRequiresFiltering. Open questions that need answers are prefixed in the draft with "XXX".

  • What regexp language is in use?

Modern POSIX.2 regular expressions - regex(7)

  • Is there any means of or need to escape characters from macros before using them in these patterns?

Yes, the literal characters "^.[$()|*+?{" should be escaped with a double backslash. A literal "\" is represented by four(!) backslashes.

  • Is "macro" the proper term for these things?

As far as I'm aware, yes, but I'm no expert.

  • Include example of a compound pattern. Depends on the regexp language in use.
%global __provides_exclude perl\\((UNIVERSAL|DB)\\)

Will match both 'perl(UNIVERSAL)' and 'perl(DB)'. Note that the literal parentheses are escaped: "
(" and "
)".

  • Can the perl_default_filter bit be converted?

perl_default_filter has already been converted for f16, but not in an automatically backward compatible fashion. All affected perl-* spec files have been updated to work with both f14-f15 style macros and the f16+ macros. The recommended incantation goes something like

%{?filter_setup:
%filter_from_requires /perl(some_bad_require)/d
%filter_from_provides /perl(some_bad_provide)/d
}
%?perl_default_filter
%global __requires_exclude %{?__requires_exclude:%__requires_exclude|}perl\\(some_bad_require\\)
%global __provides_exclude %{?__provides_exclude:%__provides_exclude|}perl\\(some_bad_provide\\)
  • %define or %global to set these?

I guess %global, but I'm not an expert.

  • Is it possible to cleanly add a pattern to one of these macros, so, for example, perl_default_filter could tack on an additional pattern and not simply override whatever the user had set.

It's not particularly clean. If you can't be sure that the macros is already set to something, you need to do something like

%global __requires_exclude %{?__requires_exclude:%__requires_exclude|}some_other_pattern

If you know that the macro is already set (i.e. you've already included the previous incantation in your spec file), the macro can be further extended by

%global __requires_exclude %{__requires_exclude}|yet_another_pattern

And yes, perl_default_filter can and should be updated in the same way to avoid clobbering any existing definitions.

comment:8 follow-up: ↓ 10 Changed 3 years ago by corsepiu

The new macros don't work with "fedpkg srpm" when using fedora-14 as development environment:

Real world example (Package rt3 - Apologies for the size):

# fedpkg srpm
error: Macro % has illegal name (%define)
error: Recursion depth(17) greater than max(16)
 15<                               (empty)
 14<                             (empty)
 13<                           (empty)
 12<                         (empty)
 11<                       (empty)
 10<                     (empty)
  9<                   (empty)
  8<                 (empty)
  7<               (empty)
  6<             (empty)
  5<           (empty)
  4<         (empty)
  3<       (empty)
  2<     (empty)
  1<   (empty)
error: Macro %__requires_exclude failed to expand
  0< (empty)
error: Recursion depth(17) greater than max(16)
 15<                               (empty)
 14<                             (empty)
 13<                           (empty)
 12<                         (empty)
 11<                       (empty)
 10<                     (empty)
  9<                   (empty)
  8<                 (empty)
  7<               (empty)
  6<             (empty)
  5<           (empty)
  4<         (empty)
  3<       (empty)
  2<     (empty)
  1<   (empty)
error: Macro %__requires_exclude failed to expand
  0< (empty)
error: Macro % has illegal name (%define)
error: Recursion depth(17) greater than max(16)
 15<                               (empty)
 14<                             (empty)
 13<                           (empty)
 12<                         (empty)
 11<                       (empty)
 10<                     (empty)
  9<                   (empty)
  8<                 (empty)
  7<               (empty)
  6<             (empty)
  5<           (empty)
  4<         (empty)
  3<       (empty)
  2<     (empty)
  1<   (empty)
error: Macro %__requires_exclude failed to expand
  0< (empty)
error: Recursion depth(17) greater than max(16)
 15<                               (empty)
 14<                             (empty)
 13<                           (empty)
 12<                         (empty)
 11<                       (empty)
 10<                     (empty)
  9<                   (empty)
  8<                 (empty)
  7<               (empty)
  6<             (empty)
  5<           (empty)
  4<         (empty)
  3<       (empty)
  2<     (empty)
  1<   (empty)
error: Macro %__requires_exclude failed to expand
  0< (empty)
error: Macro % has illegal name (%define)
error: Recursion depth(17) greater than max(16)
 15<                               (empty)
 14<                             (empty)
 13<                           (empty)
 12<                         (empty)
 11<                       (empty)
 10<                     (empty)
  9<                   (empty)
  8<                 (empty)
  7<               (empty)
  6<             (empty)
  5<           (empty)
  4<         (empty)
  3<       (empty)
  2<     (empty)
  1<   (empty)
error: Macro %__requires_exclude failed to expand
  0< (empty)
error: Recursion depth(17) greater than max(16)
 15<                               (empty)
 14<                             (empty)
 13<                           (empty)
 12<                         (empty)
 11<                       (empty)
 10<                     (empty)
  9<                   (empty)
  8<                 (empty)
  7<               (empty)
  6<             (empty)
  5<           (empty)
  4<         (empty)
  3<       (empty)
  2<     (empty)
  1<   (empty)
error: Macro %__requires_exclude failed to expand
  0< (empty)
error: Macro % has illegal name (%define)
error: Recursion depth(17) greater than max(16)
 15<                               (empty)
 14<                             (empty)
 13<                           (empty)
 12<                         (empty)
 11<                       (empty)
 10<                     (empty)
  9<                   (empty)
  8<                 (empty)
  7<               (empty)
  6<             (empty)
  5<           (empty)
  4<         (empty)
  3<       (empty)
  2<     (empty)
  1<   (empty)
error: Macro %__requires_exclude failed to expand
  0< (empty)
error: Recursion depth(17) greater than max(16)
 15<                               (empty)
 14<                             (empty)
 13<                           (empty)
 12<                         (empty)
 11<                       (empty)
 10<                     (empty)
  9<                   (empty)
  8<                 (empty)
  7<               (empty)
  6<             (empty)
  5<           (empty)
  4<         (empty)
  3<       (empty)
  2<     (empty)
  1<   (empty)
error: Macro %__requires_exclude failed to expand
  0< (empty)
error: Macro % has illegal name (%define)
error: Recursion depth(17) greater than max(16)
 15<                               (empty)
 14<                             (empty)
 13<                           (empty)
 12<                         (empty)
 11<                       (empty)
 10<                     (empty)
  9<                   (empty)
  8<                 (empty)
  7<               (empty)
  6<             (empty)
  5<           (empty)
  4<         (empty)
  3<       (empty)
  2<     (empty)
  1<   (empty)
error: Macro %__requires_exclude failed to expand
  0< (empty)
error: Recursion depth(17) greater than max(16)
 15<                               (empty)
 14<                             (empty)
 13<                           (empty)
 12<                         (empty)
 11<                       (empty)
 10<                     (empty)
  9<                   (empty)
  8<                 (empty)
  7<               (empty)
  6<             (empty)
  5<           (empty)
  4<         (empty)
  3<       (empty)
  2<     (empty)
  1<   (empty)
error: Macro %__requires_exclude failed to expand
  0< (empty)

comment:9 Changed 3 years ago by alexlan

  • Cc alexlan added

comment:10 in reply to: ↑ 8 ; follow-up: ↓ 11 Changed 2 years ago by ppisar

  • Cc ppisar added

Replying to corsepiu:

The new macros don't work with "fedpkg srpm" when using fedora-14 as development environment:

Because you need rpm 4.9.

comment:11 in reply to: ↑ 10 Changed 2 years ago by corsepiu

Replying to ppisar:

Replying to corsepiu:

The new macros don't work with "fedpkg srpm" when using fedora-14 as development environment:

Because you need rpm 4.9.

Well, though this is technically true, my view is a bit different: You cannot presume fedora packages to use rpm-4.9 based hostsystems (I currently use f14, others probably are using CentOS5). Therefore these macros can not go anywhere before they don't break building srpms on with rpm < 4.9.

comment:12 Changed 21 months ago by mmaslano

I agree that incompatible change of rpm filters was unfortunate. It's a problem maintain various macros in older and new releases, but the current state of Fedora documentation isn't good. There should be at least mentioned different possibilities with different releases. The old filtering macros are not working in some cases. At least some Perl packages have section starting with comment "old filters" and another section started with comment "RPM4.9 filters".

comment:13 Changed 21 months ago by spot

mmaslano, I agree that this is needed, but no one on the FPC understands (or can find documentation for) the new "RPM 4.9 filters". We'd need someone who does understand them to write up a set of guidelines on how to use them properly.

comment:14 Changed 20 months ago by tibbs

Just a note that I did find http://rpm.org/wiki/PackagerDocs/DependencyGenerator which has actually existed for a while now.

Basic use of the macros is quite simple, and if the default perl filtering stuff uses the new macros then some of my concerns go away. The semester's ramping up for me so free time is going to be tight, but I'll try to revise the draft and see what questions remain.

comment:15 Changed 14 months ago by toshio

I started working on this in reply to a different ticket: https://fedorahosted.org/fpc/ticket/189 before seeing that ticket is a duplicate of this one. I'm working on merging the things that seemed to still be pertinent: https://fedorahosted.org/fpc/ticket/189#comment:10

comment:16 Changed 13 months ago by toshio

  • Resolution set to fixed
  • Status changed from new to closed

ticket #189 has been approved. Marking this as fixed

Note: See TracTickets for help on using tickets.