talked on the bug report -- the %python_default_filter is ready for voting, we're going to discuss the other macros more on the python sig mailing list.

%python_default_filter approved - (+1:5, 0:0, -1:0)

We'll keep this open for the other macros, just let us know when they're ready.

So, the python sig didn't express any concerns, the only reaction was from Nick Coghlan (Python upstream committer) who expressed an opinion that these macros might be beneficial [1]. Is there anything else preventing FPC to vote on these macros?

Thanks.

[1] http://lists.fedoraproject.org/pipermail/python-devel/2013-May/000468.html

Sorry -- I have lots of concerns but I've been busy. I'll try to reply today. Are you going to be able to make it to flock by chance? It would be vey very nice if you, Nick, and I could take a day or two at flock and experiment with the new packaging stuff and come up with updated guidelines for Python.

Hmm, doesn't seem very likely that I would make it to flock. Maybe an IRC session would do?

Wow, this got dropped by the wayside. I guess there's a writeup here which could be done, except that I don't see these macros in any package anywhere (in F20, at least). What am I missing? Is there anything I can write into the guidelines now, or anything the committee can address now?

Huh, I almost forgot that this ever existed :)

So, one of the guys from python sig is currently working on guidelines for using wheels in packaging. When he's finished, I'd like to add some macros supposed to work with wheels into this proposal (e.g. %py_build_wheel, %py_install_wheel). I'm not sure what is the best approach - should we first wait for the wheel guidelines to be submitted to FPC, then update this proposal and prepare it for voting? Or shall I create a package with these macros right away and then just update it when wheel guidelines are approved?

Unfortunately we kind of forgot, too. I'm trying to clean up our tickets.

I've no idea what 'wheels' are, but as far as the guidelines go, I would do the wheels stuff all at once. In other words, just work the part of this proposal that would deal with wheels into the wheels proposal, and work on the rest of this proposal independent of that. Assuming that's even possible.

Honestly anything that makes python packaging simpler instead of the macro spaghetti it has become is fine in my book.

Is there anything the committee can look at here? Just making sure this ticket doesn't stall out again.

I still want to push this through, but due to the lack of time I currently have no time to spare. I'll get back to this when I can. Thanks for understanding.

No problem; if there's anything we can do, please let us know.

I'm going to keep this ticket open. Just pinging because it popped up on my tickets to ping list.

So, incorporating https://fedorahosted.org/fpc/ticket/513, can we do something like:

{{{
%py_shbang_opts -s
%py2_shbang_opts %py_shbang_opts
%py3_shbang_opts %py_shbang_opts

%py2_build %py_rt_build %__python2 %py2_shbang_opts
%py2_install %py_rt_install %__python2 %py2_shbang_opts

%py3_build %py_rt_build %__python3 %py3_shbang_opts
%py3_install %py_rt_install %__python3 %py3_shbang_opts

%py_rt_build() %{expand:\
CFLAGS="$RPM_OPT_FLAGS" %{!?1:%{__python}}%{?1:%{1}} %{py_setup} %{?py_setup_args} build --executable='%{!?1:%{__python}}%{?1:%{1}} %{!?2:%{py_shbang_opts}}%{?2:%{2}}' %{?py_build_args}\
}
}}}

Then users could override %py{,2,3}_shbang_opts as needed.

We discussed this a bit at today's meeting (http://meetbot.fedoraproject.org/fedora-meeting-1/2015-04-02/fpc.2015-04-02-16.00.txt):

• 281 New Python Macros for Easier Packaging (tibbs|w, 16:34:40)

• LINK: https://fedorahosted.org/fpc/ticket/281 (tibbs|w, 16:34:43)
• ACTION: Test this a bit and revisit next week. (tibbs|w, 16:41:54)

One comment was that %py_shbang_opts probably shouldn't be there, because the two pythons probably won't end up with identical options and we wouldn't want people assume they're the same by giving them one macro to override both.

The other side effect of these in getting /usr/bin/python2 or /usr/bin/python3 in the shbang. But I believe that is desired.

Indeed, that's something we want.

Suggest using these so nobody gets the idea that the default py2 and py3 options have to be the same. Still need to try these out, though.

{{{
%py2_shbang_opts -s
%py3_shbang_opts -s

%py2_build %py_rt_build %__python2 %py2_shbang_opts
%py2_install %py_rt_install %__python2 %py2_shbang_opts

%py3_build %py_rt_build %__python3 %py3_shbang_opts
%py3_install %py_rt_install %__python3 %py3_shbang_opts

%py_rt_build() %{expand:\
CFLAGS="$RPM_OPT_FLAGS" %{!?1:%{__python}}%{?1:%{1}} %{py_setup} %{?py_setup_args} build --executable='%{!?1:%{__python}}%{?1:%{1}} %{!?2:%{py_shbang_opts}}%{?2:%{2}}' %{?py_build_args}\
}
}}}

It would be nice to call it later with {{{%py2build setupegg.py}}} as there are projecty that don't use the file {{{setup.py}}} but it could also be something else.

My current nonworking macros are:

{{{
%py_setup setup.py
%py2_shbang_opts -s
%py3_shbang_opts -s

%py2_build() %{expand:\
%py_rt_build %__python2 %py2_shbang_opts %{!?1:%{?py_setup}}%{?1:%{1}} \
}
%py2_install %py_rt_install %__python2 %py2_shbang_opts

%py3_build %py_rt_build %__python3 %py3_shbang_opts
%py3_install %py_rt_install %__python3 %py3_shbang_opts

%py_rt_build() %{expand:\
CFLAGS="$RPM_OPT_FLAGS" %{!?1:%{__python}}%{?1:%{1}} %{!?3:%{?py_setup}}%{?3:%{3}} %{?py_setup_args} build --executable='%{!?1:%{__python}}%{?1:%{1}} %{!?2:%{py_shbang_opts}}%{?2:%{2}}' %{?py_build_args}\
}
}}}

Somehow the {{{%{3}}}} of {{{%py_rt_build}}} is also pointing to {{{/usr/bin/python2}}}. Does anyone see why?

These macros work like a charm:

{{{
%py_setup setup.py
%py_shbang_opts -s
%py2_shbang_opts -s
%py3_shbang_opts -s

%py_build() %{expand:\
CFLAGS="%{optflags}" %{__python} %{py_setup} %{?py_setup_args} build --executable="%{__python2} %{py_shbang_opts}" %{?1}\
}

%py_install() %{expand:\
CFLAGS="%{optflags}" %{__python} %{py_setup} %{?py_setup_args} install -O1 --skip-build --root %{buildroot} %{?1}\
}

%py2_build() %{expand:\
CFLAGS="%{optflags}" %{__python2} %{py_setup} %{?py_setup_args} build --executable="%{__python2} %{py_shbang_opts}" %{?1}\
}

%py2_install() %{expand:\
CFLAGS="%{optflags}" %{__python2} %{py_setup} %{?py_setup_args} install -O1 --skip-build --root %{buildroot} %{?1}\
}

%py3_build() %{expand:\
CFLAGS="%{optflags}" %{__python3} %{py_setup} %{?py_setup_args} build --executable="%{__python3} %{py_shbang_opts}" %{?1}\
}

%py3_install() %{expand:\
CFLAGS="%{optflags}" %{__python3} %{py_setup} %{?py_setup_args} install -O1 --skip-build --root %{buildroot} %{?1}\
}

}}}

• I don't think it makes sense to add a {{{%py_test}}} macro as there are so many ways to test the packages in {{{%check}}}.

• Overwriting {{{%py_setup}}} also works in cases where one needs to call another {{{setupegg.py}}} or so.

• Technically one doesn't need CFLAGS when installing, but I saw cases, where the --skip-build wasn't enough and the module is build again (without the appropriate CFLAGS), so it is better to just leave it there.

We discussed this at this weeks meeting (http://meetbot.fedoraproject.org/fedora-meeting-1/2015-06-11/fpc.2015-06-11-16.00.txt):

• 281 New Python Macros for Easier Packaging (geppetto, 16:47:55)

• LINK: https://fedorahosted.org/fpc/ticket/281 (geppetto, 16:47:55)

...there were no actions, but IRC logs suggest we can test/tweak the macros and vote on adding them next week.

The diff of the guidelines on top of #534 are [https://fedoraproject.org/w/index.php?title=User%3ATomspur%2FPackaging%3APython&diff=cur&oldid=414855 here].

Maybe a note about how to overwrite the setup.py is missing, but we could discuss this in the next meeting...

We discussed this at this weeks meeting (http://meetbot.fedoraproject.org/fedora-meeting-1/2015-06-18/fpc.2015-06-18-16.00.txt):

• 281 New Python Macros for Easier Packaging (geppetto, 16:40:32)

• LINK: https://fedorahosted.org/fpc/ticket/281 (geppetto, 16:40:32)
• ACTION: New Python Macros for Easier Packaging,
  py2_build/py2_install/etc. (+1:4, 0:1, -1:0) (geppetto, 16:49:05)
• ACTION: Need more votes (geppetto, 16:49:12)

We discussed this at this weeks meeting (http://meetbot.fedoraproject.org/fedora-meeting-1/2015-06-25/fpc.2015-06-25-16.01.txt):

• 281 New Python Macros for Easier Packaging (geppetto, 16:06:59)

• LINK: https://fedorahosted.org/fpc/ticket/281 (geppetto, 16:07:00)
• LINK:
  https://fedoraproject.org/w/index.php?title=User%3ATomspur%2FPackaging%3APython&diff=cur&oldid=414855
  (geppetto, 16:07:55)
• ACTION: New Python Macros for Easier Packaging,
  py2_build/py2_install/etc. (+1:7, 0:1, -1:0) (geppetto, 16:32:21)

Finally got the wiki to actually take my edits.

Announcement text:

The guidelines were overhauled to reference the new macros available in Fedora 22 (and EL7).

• https://fedoraproject.org/w/index.php?title=Packaging%3APython&diff=418542&oldid=418541
• https://fedoraproject.org/wiki/Packaging:Python
• https://fedorahosted.org/fpc/ticket/281

Hey @tibbs, I think that the diff is a bit wrong in some places. Specifically:
- In the specfile example, it uses {{{python_provides}}}, but the macro is {{{python_provide}}} (it doesn't end with "s").
- There is a typo in "%{_bindir}/sample-execl" on one place (it shouldn't end with "l").
- I think that the line "The unversioned executable must be the python 3 version." should specifically say that this is since Fedora 22 and doesn't apply to EPEL <= 7.
- I find it kind of hard to understand "'''Must''': If you build for a single python runtime you must add {{{%python_provide python-$module}}} so that the current default python is provided from the unversioned python package." I think that even if only single python runtime is used, the {{{python_provide}}} macro must be invoked with {{{pythonX-packagename}}} as argument (i.e. it must explicitly state the python major version). The macro itself then decides the proper Provides to include. Using the unversioned provide would imply that this is the python2 version, which is IMO wrong. Maybe Tom Spur could comment on this as well, since he created the macro.
- I'm not sure where the decision that unversioned binaries should point to python 3 versions came from. As was discussed in [1], the unversioned binaries should stay consistent with what /usr/bin/python points to, which is Python 2 ATM. I think that this point is still sound, is it not?

[1] https://fedorahosted.org/fpc/ticket/475#comment:11

Yeah, sorry, I'm not completely done but I've been having trouble getting the wiki to take my edits so I've given up and am making my fixes offline. Will make sure to fix up the typos.

On the third point, the new guidelines are F22+/EL7 only. I'm making this explicit in the latest revision, and will link to a copy of the old guidelines for other cases.

If I've mismerged some things, I apologize. I'll try to get them fixed up but I still have one more thing to write up which is a huge change that I have to merge with the other big changes, so I would urge you to hold off a complete review until that's done, because I'm sure I will have more merge problems.

The fifth point is almost certainly a mistake on my part; will get it fixed up.

As for the fourth point, I don't think I changed any of that from the approved draft and will wait for tomspur to comment.

Actually, thinking about it, the fifth point is currently correct as I understand it. People are talking about doing mass bug filings for applications which support python3 but still by default use python2. What's the point of the guidelines saying that every application supporting both python versions must still depend on python2?

/usr/bin/python isn't going to change for a long time, but that doesn't matter because nothing in Fedora should care.

I'll try to review the proposal that you sent to python-devel ML, that's the final proposal, right?

As for the fifth point, I think we misunderstand each other. I'm talking about binaries of applications that do something different on py2 and on py3. For example pytest, which tests with py2 if you invoke the py2 version and with py3 if you invoke the py3 version. For applications like these, the unversioned binary should be the python2 version (which is what was discussed in the linked ticket). There are applications where it shouldn't matter which python you use to invoke them (like, you don't care which python is used to invoke utilities in nfs-utils, it should always do the same), and for these py3 should be used now - this is what the mass bug filing proposal is about.

Hope this makes the situation clearer.

I don't know that anything is ever the final proposal. Every time through it I find something that needs fixing. But any changes here will be minor except for documenting the new macros.

Your explanation does make things clearer, thanks, but I'm still unsure whether that was contained in any of the votes. But since that is unclear, I'll try to get it back to the way it was.

OK, we talked about this at length. Basically, both the current guidelines and my draft were wrong regarding the default versions of executables with differing functionality between py2 and py3. I've reverted everything to the language in use previously.

My fixed draft will become the current version of the guidelines in a few minutes.