Can we re-open the discussion about tilde in version (#178, [1, 2])? The main argument against tilde - "it is not supported by rpm in currently supported Fedora releases." - Does not hold anymore, if I am not mistaken.
[1] https://lists.fedoraproject.org/pipermail/packaging/2013-December/009899.html [[BR]] [2] https://bugzilla.redhat.com/show_bug.cgi?id=734802
Replying to [ticket:398 vondruch]:
The main argument against tilde - "it is not supported by rpm in currently supported Fedora releases." - Does not hold anymore, if I am not mistaken.
That hasn't been the case since F17 went EOL. Besides all current Fedora releases, its also supported by rpm in RHEL >= 6 (it was backported to RHEL-6 just to enable tilde usage in Fedora, talk about a wasted effort... oh well)
Started a vote at today's meeting but didn't have quorum to finish
Proposal: Change guidelines to allow tilde in version
As noted in the FPC meeting, the lack of support in rpm was only one of several reasons to not use tilde (and not even the one that was most concerning to FPC last time this came up.)
May be I should explain why I reopened the discussion. The thing is that tilde is indeed not needed when you are specifying version in .spec file manually, but it does not work that well for automatic generator, which I implemented for Ruby [1].
For example, there is release candidate for upcoming version of Ruby on Rails with version 4.1.0.rc1. The stable release will have version 4.1.0. But what RPM thinks about that?
{{{ $ rpmdev-vercmp 4.1.0.rc1 4.1.0 4.1.0.rc1 > 4.1.0 }}}
So what does the generator do? If it discovers that version contains something different then just number, it cares the version as prerelease, e.g. it modifies the 4.1.0.rc1 to 4.1.0~rc1. And what RPM says about this now?
{{{ $ rpmdev-vercmp 4.1.0~rc1 4.1.0 4.1.0~rc1 < 4.1.0 }}}
Voila, the version comparison works now as expected.
I believe that this is the most elegant approach. Although you might say that tilde is non-intuitive, I can hardly imagine something better for this case.
[1] http://pkgs.fedoraproject.org/cgit/ruby.git/tree/rubygems.prov?h=private-ruby-2.1#n12
The usual way would be to have the autogen. do a version an release prefix, so you get:
{{{ Version: 4.1.0 Release: 0.rc1.<real-release> }}}
...from:
https://fedoraproject.org/wiki/Packaging:NamingGuidelines#Pre-Release_packages
...but fwiw my 0 was very close to a +1 anyway, so if you can convince other members and you come up with the policy wording changes, you've got a chance.
I am speaking about RPM dependency generator. So I assume that the Version/Release you are speaking about is probably misunderstanding or I'm sorry, I have no clue ;)
[1] http://www.rpm.org/wiki/PackagerDocs/DependencyGenerator
I think allowing ~ in version, will make things clearer
Let me comment on yesterdays FPC meeting.
17:47:49 <abadger1999> Rathann: I can see cases where an upstream author might decide to require a prerelease. 17:47:59 <abadger1999> but I'm not sure it is "correct" 17:50:01 <abadger1999> it would be that hte upstream developed against foo-1.0a1 and then foo-1.0a3 changed the api.
This is exactly case for Ruby on Rails. The core is released as 8 components. The components are more or less just code organization and they tightly specify they version. Lets check the current release candidate [1].
And let me explain once more on Ruby on Rails why the tilde.
Stable: 4.0.3[[BR]] Pre-release: 4.0.4.rc1[[BR]] Next stable: 4.0.4
The increasing order for versions is 4.0.3 < 4.0.4.rc1 < 4.0.4, but this does not work for RPM:
{{{ $ rpmdev-vercmp 4.0.3 4.0.4.rc1 4.0.3 < 4.0.4.rc1 $ rpmdev-vercmp 4.0.4 4.0.4.rc1 4.0.4 < 4.0.4.rc1 }}}
Here is the 4.0.4.rc1 always the highest. We cannot just drop the rc1, since that would allow RPM to mix the stable releases with pre-releases. While that looks OK from RPM's point of view, later comes RubyGems dep-solver and it still has the information about rc1 and tries to load appropriate version, which might fail if there is mix of 4.0.4 and 4.0.4.rc1.
In current state, there would be following solution:
{{{ $ rpmdev-vercmp 4.0.3-1 4.0.4-0.1.rc1 4.0.3-1 < 4.0.4-0.1.rc1 $ rpmdev-vercmp 4.0.4-1 4.0.4-0.1.rc1 4.0.4-1 > 4.0.4-0.1.rc1 }}}
Please note, that in that case, we have to come with fake release numbers, which has nothing to do with releases specified in .spec file. That is confusing. Moreover, even the stable version has to be extended by the release magic.
And here is the solution with tilde:
{{{ $ rpmdev-vercmp 4.0.3 4.0.4~rc1 4.0.3 < 4.0.4~rc1 $ rpmdev-vercmp 4.0.4 4.0.4~rc1 4.0.4 > 4.0.4~rc1 }}}
The last dot in the pre-release version is converted into tilde, that is the only difference from upstream version. This is lot shorter and comprehensive solution then some fake releases IMO.
[1] http://rubygems.org/gems/rails/versions/4.0.4.rc1
It's actually worse: When specifying versioned dependencies/Obsoletes/Conflicts, you have to know what values of Release were actually used in the specfiles for the previous RCs! The comparison is really against the Release, not the alphatag, so only a full history of the RC<->Release tag mappings can tell you what to use. (Needless to say, this is also distribution-specific, and in extreme cases, even specific to the particular release of Fedora!) I have personally had to deal with this more than once, it is not pretty (even if you only target Fedora and the Release values used where the same on all branches).
In addition, I have seen MANY cases where the versioning guidelines for prereleases were not followed correctly, they are clearly hard to understand. Even some long-term Red Hat employees manage to get them wrong. That only leaves "it introduces another special character to achieve something which can be handled in other ways" as the last remaining justification, and that's a very weak excuse for banning the use of an RPM feature.
IMHO, the scheme we are currently using is clearly an ugly hack and a workaround for RPM historically not having supported this feature. Shouldn't the goal of the Fedora packaging guidelines be to have clean, legible spec files and to avoid hacks wherever possible?
So I really see no good reason why we should not use this new RPM feature. IMHO, the guidelines should clearly mention the existing scheme as deprecated and recommend using the tilde instead (with the only caveat that packages using the legacy scheme need to keep it until Version gets bumped, or the comparisons including the upgrade path will fail).
Replying to [comment:11 kkofler]:
In addition, I have seen MANY cases where the versioning guidelines for prereleases were not followed correctly,
If people don't follow guidelines then I don't see how changing the guidelines will help anything, esp. when changing them to something nobody currently does that works in a magic way.
No. The guidelines currently work, and it's obvious to a normal person that when they have:
foo-1.2.3-0.rc1.1.fc20 foo-1.2.3-1.fc20
...that the former comes before the later (and any sane sort will order the packages this way). Now the desire is to change this to:
foo-1.2.3~rc1-1.fc20 foo-1.2.3-1.fc20
...which is much more magic, so both average specwriters and users will be confused. And most sorts will order the wrong way (rpm and ls --sort=version being the exceptions).
What we really need to see is a good reason why the current method is problematic (vondruch mentioned auto deps.), if a real problem is brought forward you might be able to convince enough of the group that the problems with the new syntax are worth it.
Replying to [comment:12 james]:
The guidelines currently work, and it's obvious to a normal person that when they have: foo-1.2.3-0.rc1.1.fc20 foo-1.2.3-1.fc20
The guidelines currently work, and it's obvious to a normal person that when they have:
James, I'm missing <sarcasm> tags here. Normal person, obvious? Come on. Anybody who thinks this goo is obvious has simply been involved this goo far too long :) The tilde avoids abusing (yes, abusing) the release for what actually belongs to the upstream version. And sure its newish, different magic, but it might even be that little bit less black magic to an actual normal person than the release tag abuse.
Speaking of magic, tilde is no magic silver bullet for handling non-numeric versions in automatically generated dependencies. It certainly makes things easier, or even remotely possible, but just keep in mind non-numeric characters in version are not always pre-release versions. "1.0alpha" and the like are usually considered pre-release versions by us humans but "1.0a" style is commonly used as post-release minor update notation as well. And variations to the theme are endless...
Replying to [comment:13 pmatilai]:
Replying to [comment:12 james]: The guidelines currently work, and it's obvious to a normal person that when they have: {{{ foo-1.2.3-0.rc1.1.fc20 foo-1.2.3-1.fc20 }}} James, I'm missing <sarcasm> tags here. Normal person, obvious? Come on. Anybody who thinks this goo is obvious has simply been involved this goo far too long :)
The guidelines currently work, and it's obvious to a normal person that when they have: {{{ foo-1.2.3-0.rc1.1.fc20 foo-1.2.3-1.fc20 }}}
James, I'm missing <sarcasm> tags here. Normal person, obvious? Come on. Anybody who thinks this goo is obvious has simply been involved this goo far too long :)
I disagree, I would bet that a lot of people can understand or guess the order correctly for 0.rc1.1.fc20 vs. 1.fc20 ... and that number will go down when you show them the version with ~ in it (esp. given that ls gives them the correct answer for one, and the incorrect one for the other).
The tilde avoids abusing (yes, abusing) the release for what actually belongs to the upstream version.
Sure, but nobody has shown why this abuse is bad.
And sure its newish, different magic, but it might even be that little bit less black magic to an actual normal person than the release tag abuse.
I would disagree very strongly, when ls or the html listing on download.blah.com (or general lexical sorting) gets it wrong it is absolutely more magic.
So currently we're pretty heavily against this. The two 0 votes expressed a desire to know of an actual example of what you want to do where using release is not sufficient if you'd potentially want to convince them.
tibbs pointed out that tilde solves less cases than using the release tag as what occurs in the version string as the alpha characters might also not be intended to parse in the order in which rpm will order them.
I'm going to close this as without some startling new information this is unlikely to pass. If you have such an example we can see where that takes us but you'll need to change four votes so it may not be worthwhile.
Replying to [comment:16 james]:
Replying to [comment:13 pmatilai]: Replying to [comment:12 james]: The guidelines currently work, and it's obvious to a normal person that when they have: {{{ foo-1.2.3-0.rc1.1.fc20 foo-1.2.3-1.fc20 }}} James, I'm missing <sarcasm> tags here. Normal person, obvious? Come on. Anybody who thinks this goo is obvious has simply been involved this goo far too long :) I disagree, I would bet that a lot of people can understand or guess the order correctly for 0.rc1.1.fc20 vs. 1.fc20 ... and that number will go down when you show them the version with ~ in it (esp. given that ls gives them the correct answer for one, and the incorrect one for the other).
Actually I might agree with you on this. But you are looking at the issue only from the consumer side, e.g. I already have library with some version. But you completely ignore the opposite, when I am package maintainer and I'd like to package some prerelease version. How often we see that the pre-release is not versioned correctly? Not surprisingly, neither your example from comment 12 follows the guidelines ...
The versioning issue for pre-/post- release code has come up again, this time in regards to [https://lists.fedoraproject.org/pipermail/devel/2015-November/216559.html causing issues with installing packages].
Frankly, it's very silly at this point.
With the notable exception of EL5, all currently supported distro/releases support the "~" character for adjusting ordering in version comparison. I don't know if we have the corresponding "+" character that does the opposite of "~" (moving it up the version ordering rather than down as "~" does), but it's confusing and obnoxious to figure out how to do proper pre-release and post-release versioning.
Can we please revise the guidelines to actually take advantage of these things to make it so these version are formatted to match upstream intent, as they are part of the version, not the release?
I'm still not for the use of tildes in the version, but I would be happy to reassess my position if this request actually came with a draft revision of the NamingGuidelines page showing how things would look if the tilde method were used instead of the current method (which I find perfectly understandable, honestly).
I would also like to see an assessment of how all of our usual tools would support this (including libraries which have RPM version comparison functions) and a statement from inra or releng stating that they don't think this would break the buildsys.
In other words, if you want this, please do a bit more work than reopening an old ticket and calling things silly.
I can only reiterate what I wrote in comment:11. The proof that the legacy hack is hard to get right is that even the rpm package (!) got it wrong (see the thread ngompa linked to)! I see no valid reason to continue banning the real solution that puts the upstream version into the Version field where it belongs.
Can someone please link to some decent documentation about what exactly ~ and + achieve? I can't easily find any.
On the face of it, I'd like to add my voice to those who are sceptical: this seems too coarse, to me. It looks like all you can do is indicate 'before stable' or 'after stable', but this mechanism seems not to provide for any way of indicating relative order of pre- and post-release builds. It's 'lucky' that commonly-used milestones - Alpha, Beta, RC - happen to sort correctly alphabetically, but it's hardly a sufficient solution: there are projects that use other milestones which don't sort alphabetically, and what happens if you, say, want to build Alpha 1, then a git snapshot, then Alpha 2?
For whatever it's worth I'm opposed to any proposal which doesn't maintain the current system's ability to reliably impose order on builds with arbitrary identifiers.
Replying to [comment:23 adamwill]:
I'm opposed to any proposal which doesn't maintain the current system's ability to reliably impose order on builds with arbitrary identifiers.
That's also, why I am opposed to this proposal. To me, "+/~" are far from being self-explanatory (I do not understand them), the consequences of allowing them unknown.
Actually, I believe all of this to be a Debian imitation cult, which originates from Debian users, who do not understand rpm's versioning and want "RPM" to be like "Deb".
Just FWIW, openSUSE encourages tilde for pre-release versioning: https://en.opensuse.org/openSUSE:Package_naming_guidelines#Pre-Release_packages
So its actively used by at least one major distro, whom which we share our depsolver base with (libsolv underneath dnf) so the tooling side should be fine.
there are projects that use other milestones which don't sort alphabetically, and what happens if you, say, want to build Alpha 1, then a git snapshot, then Alpha 2?
You can switch to the currently mandated versioning scheme whenever needed in those (quite rare I dare say) cases. In the above example you would do it between the git snapshot and Alpha 2.
The dpkg + is not implemented in RPM, + is a separator that is skipped in version comparisons, just like the . separator.
The safest versioning scheme with the ~ operator would be 1.0.0~0.alpha1, 1.0.0~0.20151120gitdeadbeef12345, 1.0.0~1.alpha2 (sort of micro-Epoch, sticking to the lowest integer possible), or 1.0.0~0.alpha1, 1.0.0~1.20151120gitdeadbeef12345, 1.0.0~2.alpha2 (just always bumping the integer as in the current -0.1.foo scheme, though of course you wouldn't bump this new integer when bumping only Release).
I'll note that it would also be safe to allow a suffix directly after the ~ as long as it is alphabetic, and only require the "micro-Epoch" if a date or some other numeric item is introduced, i.e., e.g.: 1.0.0~alpha1, 1.0.0~0.20151120gitdeadbeef12345, 1.0.0~1.alpha2
This is probably the cleanest scheme (and thus my favorite), but also probably the hardest to understand for new packagers, unfortunately.
Replying to [comment:28 kkofler]:
I'll note that it would also be safe to allow a suffix directly after the ~ as long as it is alphabetic, and only require the "micro-Epoch" if a date or some other numeric item is introduced, i.e., e.g.: 1.0.0~alpha1, 1.0.0~0.20151120gitdeadbeef12345, 1.0.0~1.alpha2 This is probably the cleanest scheme (and thus my favorite), but also probably the hardest to understand for new packagers, unfortunately.
Frankly, I don't understand this or the convention mentioned in comment:27. I can't tell which part is the version and which part is the release. Having both textual and numbered substrings in the same position is confusing. Adam's points (which are really a reiteration of what was said before) still stand.
Replying to [comment:26 scop]:
Replying to [comment:23 adamwill]: there are projects that use other milestones which don't sort alphabetically, and what happens if you, say, want to build Alpha 1, then a git snapshot, then Alpha 2? You can switch to the currently mandated versioning scheme whenever needed in those (quite rare I dare say) cases. In the above example you would do it between the git snapshot and Alpha 2.
If you are proposing using both the current scheme and some new scheme, then please let's just keep the current scheme. I am not going to vote for doubling the size of the guidelines here.
Replying to [comment:31 mbooth]:
I think a lot of the examples/suggestions/whatever want blessing to just use ~ in specific situations, that make those situations simpler ... but then make the whole less simple, because you are just adding an exception to the whole. I'm unlikely to +1 that.
However I think this can be done by changing everything so, when upstream releases the rc1/rc2/rc3 and final of 1.0.0, instead of:
{{{ foo-1.0.0-0.1.rc1.fc23 foo-1.0.0-0.2.rc1.fc23 foo-1.0.0-0.3.rc2.fc23 foo-1.0.0-0.4.rc2.fc23 foo-1.0.0-0.5.rc3.fc23 foo-1.0.0-1.fc23 }}}
...which I think is what packagers should do now, you get a policy that specified:
{{{ foo-1.0.0~0.rc1-1.fc23 foo-1.0.0~0.rc1-2.fc23 foo-1.0.0~1.rc2-1.fc23 foo-1.0.0~1.rc2-2.fc23 foo-1.0.0~2.rc3-1.fc23 foo-1.0.0-1.fc23 }}}
...and if that helps vondruch/etc. out, I'm certainly open minded about changing it.
Rough pro/con, from my POV (and in no particular order):
PROS
CONS
To clarify I'm saying that the changed policy specifically wouldn't allow:
{{{ foo-1.0.0~rc1-1.fc23 foo-1.0.0~rc1-2.fc23 foo-1.0.0~rc2-1.fc23 foo-1.0.0~rc2-2.fc23 foo-1.0.0~rc3-1.fc23 foo-1.0.0-1.fc23 }}}
...even though that would work in this specific case, of upstream just releasing those RCs.
I must confess, that re-exploring what the tilde actually does, I'm not as enthusiastic about it as I used to be. The point is that james is probably right that the only convention which covers the issue completely is foo-1.0.0~0.rc1-1.fc23 and this is unfortunately not any better then the current guidelines.
Nevertheless, I have sill one point which I was aiming to solve. Let me give you and example. I might have upstream releasing packages such as:
{{{ foo-1.0.prebeta foo-1.0.beta foo-1.0 }}}
I as an inexperienced packager will convert these into NVR such as
{{{ foo-1.0.prebeta-1.fc23 foo-1.0.beta-1.fc23 foo-1.0-1.fc23 }}}
This means that I will never get the stable version installed. But if the rule is something like "split any release tag in version by tilde instead of dot", there are chances that the NVRs will look like:
{{{ foo-1.0~prebeta-1.fc23 foo-1.0~beta-1.fc23 foo-1.0-1.fc23 }}}
This means that the upgrade patch is still not correct, but if nothing else, the stable version once released can be installed.
Also note that I wanted this especially for RPM's automatic dependency generator, which can hardly guess if prebeta was released before beta or what it actually means.
Replying to [comment:29 rathann]:
Replying to [comment:28 kkofler]: I'll note that it would also be safe to allow a suffix directly after the ~ as long as it is alphabetic, and only require the "micro-Epoch" if a date or some other numeric item is introduced, i.e., e.g.: 1.0.0~alpha1, 1.0.0~0.20151120gitdeadbeef12345, 1.0.0~1.alpha2 This is probably the cleanest scheme (and thus my favorite), but also probably the hardest to understand for new packagers, unfortunately. Frankly, I don't understand this or the convention mentioned in comment:27. I can't tell which part is the version and which part is the release. Having both textual and numbered substrings in the same position is confusing. Adam's points (which are really a reiteration of what was said before) still stand.
All my examples are '''only''' the Version tag! Release would be separate (of the form 1%{?dist}, 2%{?dist}, …) and only contain what Release is supposed to contain (the version of the packaging), no component of the upstream version would end up hacked into it anymore (as they are in the current legacy scheme).
So the full EVR would be, e.g., 0:1.0.0~1.alpha2-1.fc24, where Epoch is 0, Version is 1.0.0~1.alpha2, and Release is 1.fc24.
0:1.0.0~1.alpha2-1.fc24
0
1.0.0~1.alpha2
1.fc24
And my proposed scheme addresses Adam's objections, that's exactly what I designed it for. It's just that you apparently did not understand it.
I would also like to point out that james's scheme from comment:32 is exactly my scheme from comment:27. My more complex scheme from comment:28 would also allow the example in comment:33 (because you can actually safely switch from that to comment:27 versioning due to the numbers > letters rule in rpmvercmp), my simpler scheme from comment:27 would not.
Replying to [comment:34 vondruch]:
I think we should revive this.
Draft: https://fedoraproject.org/wiki/PackagingDrafts/TildeVersioning
Will you be at the meeting in 35 mins?
A quick look at the draft doesn't seem to answer any of the questions/concerns that had been brought up before. A lot of the examples only work by accident, and don't really explain that or require only usage that would codify the accidents as the only option.
The last example has to bump epoch because the accident fails, where I would be shocked if all of the FPC didn't prefer the current non-tilda usge which doesn't require epoch bumps. Related to that there's no comparison to the current method, or what we gain by using this new feature.
Note that a proper diff is: https://fedoraproject.org/w/index.php?title=PackagingDrafts%2FTildeVersioning&diff=470507&oldid=470466
Now, I'm still inclined to vote against this unless somehow you have overcome the previous objections. This still needs an integer after the tilde in the general case, so how has this simplified anything, besides swapping a tilde for a dash? What does this actually simplify?
Also, the proposed use of "+", while interesting, seems to be a new (and possibly separate) issue. Is '+' magic to RPM in some way, or are you just depending on the fact that it sorts newer than '-'? Have you thought about how this interacts with '+' in package names? (I don't think there's any issue, but I haven't actually run any examples.)
Finally, I do have to say that with the addition of the '+' there's a certain symmetry I find appealing. Our current rules:
Proposed:
There is something to be said for the semi-obviousness of this.
Other points:
(James beat me by 79 seconds.)
@tibbs:
The foundation of this draft revision was written by me, and I took inspiration from the openSUSE guidelines regarding versions, which in turn are somewhat derived from Debian Policy, though obviously tweaked for the RPM-centric environment.
Addressing a few points here:
There's no special magic with '+', it's mainly useful for us to intuitively understand pre-release, regular release, and post-release packages (as you noted).
It is not forbidden to do the minor Release bumps: https://fedoraproject.org/wiki/PackagingDrafts/TildeVersioning#Minor_release_bumps_for_old_branches
That said, with our move to distro-sync based upgrades, it's not really necessary, but I certainly didn't forbid it. We just shouldn't do Release field hackery for representing upstream version information.
I believe that the clear separation of software version and package version is more clearly defined with this proposal than the current way we do it. While it's still fudgy a bit, I think it's less so than the status quo.
I believe the separation of the upstream version and the downstream package version is important enough to warrant this change, especially since there's a level of clear intuition in the scheme (~,-,+, etc.) as designed.
The "Release Tag" section which was added definitely forbids minor release bumps, and needs to be amended.
I do not believe Epoch is an acceptable option.
Can't vote for this until it includes a reasonable way to handle non-sorted alphatags, sorry. The status quo handles all cases and we shouldn't regress.
We discussed this at this weeks meeting (http://meetbot.fedoraproject.org/fedora-meeting-1/2016-09-08/fpc.2016-09-08-16.00.txt):
...lots of things were said in the meeting though, and comments above. A significant amount of which was constructive.
This proposal is definitely step forward and I like the "balance" mentioned in comment:43.
But what I would like to see improved is the "Basic versioning rules" section and mainly the "separators" bullet. The list should contain all separators, not just some incomplete example (it does not mention ~ which this is all about).
Also, it does mention upper case letters, but does not mention lower case letters. Apparently a > A in the comparison.
And as others pointed out, it probably does not solve this scenarios:
{{{ $ rpmdev-vercmp 1.0~preview1 1.0~alpha1 1.0~preview1 > 1.0~alpha1 }}}
i.e. the preview1 is released during the development cycle, while alpha is released later in the development cycle.
Note that the draft presented actually does solve that unordered alphatag issue, but the solution there is to just bump Epoch. This was soundly rejected in the IRC meeting.
One solution is to always prepend a number.
Another solution, which I hesitate to suggest, is to just modify the tag until the ordering works:
1.0~alpha1 => 1.0~Zalpha1 (or z depending on how capitals sort). I need to check how numbers sort versus letters, because it might also be possible to just start using a number if the sorting works.
This would be needed pretty rarely. It's certainly better than using Epoch and allows the thing to be simpler in the common case (where the tags sort properly) but it's aesthetically displeasing on more than one level.
Also, I would add that the fact that rpm-vercmp seems to not work the same way as the proposal indicates that rpm itself works is a real problem for testing these things. You should be able to use rpm --eval and some lua, though.
I wrote bullet how ordering for letter works (z > Z).
I would prefer to prepend number only when it's really needed (/me hopes there will be not much cases).
Before you put this comment I already fixed draft to prepend number: https://fedoraproject.org/w/index.php?title=PackagingDrafts%2FTildeVersioning&diff=470737&oldid=470589
So here's my question: is it possible to have a reasonable scheme for ordering the alphatags which you only need to bring into use at the point where you have discovered that upstream has started to use something which doesn't sort?
I agree that it would be nice to only prepend a number when it's necessary, except:
I'm not sure what the best option is.
You don't necessarily know that upstream is going to do that until they actually do it. this.
Usually upstream follows some guidelines for their language (Python - PEP440, ...) or some generic conceptions: alpha -> beta -> rc1 -> rc2 -> rcX -> release.
In this case everything works fine. Once some weird upstream does: prebeta -> beta, there is a problem which can be solved by prepending num and separator (to be safe, I will correct page).
It will be less confusing and a better than current: Now vs Proposed
Replying to [comment:43 tibbs]:
Your draft forbids (in the final section) using Release = N.%dist.M for minor bumps necessary to maintain ordering between releases, which is explicitly allowed elsewhere. I think it's same as what's in current versioning guidelines (at least it seems like that).
16:21:47 <Rathann> also, what do you do if kismet upstream releases 1.0.2 after you packaged kismet-1.0+20050515cvs-1%{?dist} ? rpmdev-vercmp is correct, unfortunately. Currently '+' and '.' is separators just for splitting sections.
But I raised discussion for adding "meaning" for '+' and proposed patches to related components (see below).
Once we will have patch merged to RPM - I will update wiki to match its behavior.
Answering to question before it could come: It is '''safe''' to put such change for Fedora (any version). We have ''2'' packages which making use of '+' and I'm not sure that maintainers know what they do:
tibbs, geppetto, others, anything else I forgot to address?
Replying to [comment:52 ignatenkobrain]:
You don't necessarily know that upstream is going to do that until they actually do it. this. Usually upstream follows some guidelines for their language (Python - PEP440, ...) or some generic conceptions: alpha -> beta -> rc1 -> rc2 -> rcX -> release. In this case everything works fine. Once some weird upstream does: prebeta -> beta, there is a problem which can be solved by prepending num and separator (to be safe, I will correct page).
One obvious split here is to say "if your upstream uses these 3 tags (with numbered suffix), then you can include them directly otherwise do XYZ" where XYZ will sort after rc666 and include numbers. Maybe add one or two more "well known" tags, or map them to alphaN/betaN/rcN.
Replying to [comment:55 ignatenkobrain]:
16:21:47 <Rathann> also, what do you do if kismet upstream releases 1.0.2 after you packaged kismet-1.0+20050515cvs-1%{?dist} ? rpmdev-vercmp is correct, unfortunately. Currently '+' and '.' is separators just for splitting sections. But I raised discussion for adding "meaning" for '+' and proposed patches to related components (see below).
At the very least we couldn't accept anything that relied on those patches until one year after a changed rpm was in the GA for a release. And even then I don't think I'd love it.
Answering to question before it could come: It is '''safe''' to put such change for Fedora (any version). We have ''2'' packages which making use of '+' and I'm not sure that maintainers know what they do.
If we change the policy to include that usage as recommended then lots of people would start using it, or am I missing something?
Replying to [comment:57 james]:
Replying to [comment:55 ignatenkobrain]: 16:21:47 <Rathann> also, what do you do if kismet upstream releases 1.0.2 after you packaged kismet-1.0+20050515cvs-1%{?dist} ? rpmdev-vercmp is correct, unfortunately. Currently '+' and '.' is separators just for splitting sections. But I raised discussion for adding "meaning" for '+' and proposed patches to related components (see below). At the very least we couldn't accept anything that relied on those patches until one year after a changed rpm was in the GA for a release. It depends how you look on that. From my POV it's bugfix. And even then I don't think I'd love it. Why? Wasn't point that "1.0.1 < 1.0+2012"? I tried to address exactly that point. Answering to question before it could come: It is '''safe''' to put such change for Fedora (any version). We have ''2'' packages which making use of '+' and I'm not sure that maintainers know what they do. If we change the policy to include that usage as recommended then lots of people would start using it, or am I missing something? Here was analysis if we apply that patch - what it will change for already existing packages.
At the very least we couldn't accept anything that relied on those patches until one year after a changed rpm was in the GA for a release. It depends how you look on that. From my POV it's bugfix. And even then I don't think I'd love it. Why? Wasn't point that "1.0.1 < 1.0+2012"? I tried to address exactly that point.
If we change the policy to include that usage as recommended then lots of people would start using it, or am I missing something? Here was analysis if we apply that patch - what it will change for already existing packages.
It's definitely possible instead of "+" to fallback for previous scheme which is in guidelines now, but I would like to have proper separation as noted tibbs:
Replying to [comment:56 james]:
Replying to [comment:52 ignatenkobrain]: You don't necessarily know that upstream is going to do that until they actually do it. this. Usually upstream follows some guidelines for their language (Python - PEP440, ...) or some generic conceptions: alpha -> beta -> rc1 -> rc2 -> rcX -> release. In this case everything works fine. Once some weird upstream does: prebeta -> beta, there is a problem which can be solved by prepending num and separator (to be safe, I will correct page). One obvious split here is to say "if your upstream uses these 3 tags (with numbered suffix), then you can include them directly otherwise do XYZ" where XYZ will sort after rc666 and include numbers. Maybe add one or two more "well known" tags, or map them to alphaN/betaN/rcN. I will add example. But. I think there should be rule "unless it's non-sortable, don't add anything". You don't know in advance what would happen in upstream. Even they use alpha/beta/rc currently, they might release 'prebeta', so in the future you will add 'N.', but they might never release such thing again, but you already will stick into such scheme.
One obvious split here is to say "if your upstream uses these 3 tags (with numbered suffix), then you can include them directly otherwise do XYZ" where XYZ will sort after rc666 and include numbers. Maybe add one or two more "well known" tags, or map them to alphaN/betaN/rcN. I will add example. But. I think there should be rule "unless it's non-sortable, don't add anything". You don't know in advance what would happen in upstream. Even they use alpha/beta/rc currently, they might release 'prebeta', so in the future you will add 'N.', but they might never release such thing again, but you already will stick into such scheme.
So the fix for the "+" without requiring special patches is to simply move the vcs/snap label from the end to the beginning.
That is:
1.0+20120101git -> 1.0+git20120101
By doing so, it sorts correctly. openSUSE does this now (though they insert a period between "git" and the snapshot date). This approach fixes the version comparison issues, such that post-releases are now lower than the next formal release.
I've already updated the draft guidelines accordingly.
We discussed this at this weeks meeting (http://meetbot.fedoraproject.org/fedora-meeting-1/2016-09-15/fpc.2016-09-15-16.01.txt):
...lots of concerns, but try to tidy up the draft for next week and give a summary of the benefits.
Version/Release strings allowed to have ASCII letters (a-zA-Z), digits (0-9) and separators (._+%{}~)
Please take %{} out of the allowed separators. Technically speaking, yes rpm does accept them as of current versions but due to a defiency, not because anybody wants to allow them. Certainly absolutely nobody should be putting them into version/release on purpose.
See http://rpm.org/gitweb?p=rpm.git;a=commit;h=507f21f6bb4bf7029a0bca255cfe4aae3361f358 and https://bugzilla.redhat.com/show_bug.cgi?id=547997 for the background.
@pmatilai: Removed mention of the offending separators.
I've made some edits to reorganize the document a bit more logically. It starts with general explanations of RPM versioning, then goes into non-numeric versions, from there goes into pre-release, post-release, and then snapshot versions (which involve the two previous guidelines).
I've tried to clear up some of the confusing points mentioned before as well.
Here's a summary of some advantages of this proposal:
It provides a clean separation of software and packaging versions. By this, I mean that the Version field contains all the information representing the version of the software being packaged, and the Release field contains purely the packaging versioning.
It provides a fairly logical scheme where it is obvious that something is pre-release or post-release snapshot by using ~ or +, respectively. At a glance, snapshot releases stand out and are obvious.
It provides a standardized scheme for representing software versions for bundled() Provides. Since Fedora now allows bundled libraries as long as they are documented, the question comes up with how to properly document snapshot versions. Currently, packages are inconsistent on this. Some even don't indicate they have a snapshot version at all (cough PackageKit in F25 cough). This scheme can be used as the guideline for that, enabling for saner introspection of bundled software.
It enables new kinds of automation. By using this scheme, it is now much easier to have automation for doing things like minor spec bumps, mass rebuilds, etc. Because the Release field only contains the packaging version (which most of the time will be an integer followed by the dist tag), tools like the mass rebuild script don't have to do as much crazy parsing to properly handle bumping the spec. It also makes it possible for supporting workflows where tools like Koschei could potentially be used to automatically rebuild packages as dependency drift occurs and submit it as an update, rather than waiting for a packager to do it. This is how SUSE handles the problem with OBS, and moving to this scheme may allow us to experiment with doing the same.
Snapshot versions would now show up in version checks, which can be useful for audit tools. If the audit tool has some awareness of when vulnerabilities were fixed or such, the snapshot information in the version field can prove to be useful in this manner.
It brings us in sync with other distributions on how packages are versioned (notably openSUSE and increasingly Mageia). With the exception of one small thing (the lack of period separator between "git" and the snapshot date), this scheme is pretty much in line with openSUSE's, which makes it easier to directly compare packages. This isn't a huge deal, but it is useful when trying to figure out who has what when you have to deal with snapshot packages. (If we want to, we could put the separator, since we are changing a lot anyway...)
We discussed this at this weeks meeting (http://meetbot.fedoraproject.org/fedora-meeting-1/2016-09-29/fpc.2016-09-29-16.00.txt):
Metadata Update from @tibbs: - Issue assigned to ignatenkobrain
Now that my reorganization/rewrite of the existing versioning guidelines is done and in place, I did what I promised and made a version that mentions tilde-based prerelease versioning instead.
https://fedoraproject.org/wiki/User:Tibbs/TildeVersioning https://fedoraproject.org/w/index.php?title=User%3ATibbs%2FTildeVersioning&diff=latest&oldid=487415
I had done the rewrite with tilde versioning in mind, so it's a minimal change which sadly doesn't really result in a simpler explanation for anything. Though maybe use of tilde could somehow save the "unsortable versions" case and perhaps allow us to remove <extraver> from that document. I don't know.
In any case, the straw man draft is there. Edit at your leisure.
Metadata Update from @tibbs: - Issue close_status updated to: None
@tibbs I wonder if we can move this forward. Can we just go with your draft?
There was no support for it when we last talked about it. It's not really any simpler, and so just seems to be needless churn. (And remember that if we did adopt this we'd have things using both versioning schemes, which I fear means that both sets of guidelines would persist because RHEL6 only has RPM 4.8 and thus does not support the tilde syntax.) So I don't see it moving forward without some clear articulation of some really significdant benefits.
But the draft is still there and I am happy to see folks edit it if simplifications are possible.
I fear means that both sets of guidelines would persist because RHEL6 only has RPM 4.8 and thus does not support the tilde syntax.
This is not true. The RPM team (specifically @ffesti) backported this for us years ago to EL6 partly so that we could use it in Fedora.
* Sun Oct 14 2012 Florian Festi <ffesti@redhat.com> - 4.8.0-28 - Handle identical binaries for debug-info (#727872) - Fix typos in Japanese rpm man page (#845065) - Document -D and -E options in man page (#845063) - Add --setperms and --setuids to the man page (#839126) - Update man page that SHA256 is also used for file digest (#804049) - Remove --fileid from man page to get rid of md5 - Remove -s from patch calls (#773503) - Force _host_vendor to redhat to better match toolchain (#743229) - Backport reloadConfig for Python API (#825147) - Support for dpkg-style sorting of tilde in version/release (#825087) - Fix explicit directory %attr() when %defattr() is active (#730473) - Don't load keyring if signature checking is disabled (#664696) - Retry read() to fix rpm2cpio with pipe as stdin (#802839)
This went out with RHEL/CentOS 6.4.
And I think it would really be worthwhile to get rid of the "alphatag in Release" hack. The alphatag is part of the upstream version and has absolutely no business being in the Release tag.
Cleaning up Release will also decrease the amount of bad Release bumps from automated tools.
If only they had backported %license to EL6 instead.... I can never comprehend which parts of RHEL can change and which can't. But in any case the rest of my comment still stands.
And my personal opinion is unchanged from the meetings where this was discussed: I'm generally supportive of it but I'm not going to advocate for it.
Well that was terrible. Looks like pagure got updated and is behaving oddly.
As a packager and macro writer, I don't find tilde any good, or intuitive:
A good working human-friendly system would:
Those do not need to be supported by old rpm versions, it can just ignore them.
Anything else is just mopping around and moving problems from one point of the spec to another.
Coming from this [1] discussion, it would be nice if the last @tibbs [2] was accepted.
@nim I don't disagree with your point, nevertheless you don't provide any tangible fix to the issues such as [1] (scroll to the bottom), [2] or [3].
And I would not mind so much the holes in historical rpm metadata definitions, if rpm upstream could just purge all the misplaced assertions in rpmbuild that check for Tag existence before reaching %prep.
%prep
Right now, even if you identify problems in historical tags, and set up something more appropriate, and more intuitive, using rpm variables, you can not generate the corresponding rpm tags in spec files because rpmbuild will lobby at you assertions like "Version is not defined" "Name not defined" "Source not defined" before evaluating the macro that sets the corresponding line to its correct value.
Tilde does not fix those either
@tibbs [2] states pkgrel stays simple and Version: gets the complexity
I will go further Version: gets the breakage
Because I know macros in Fedora that read version and that this change will break.
Tilde will allow you to remove workaround code in rubygems processing, only to get new workaround code in other places of Fedora. At least the old workaround code exists and is QAed today.
You can move non-semver decoration around as much as you want, as long as it is in Release or Version it will break something. It's just an alien thing that should have had its own place from day one.
And you can not manage the breakage via rpm macro code because of the way rpmbuild is dead set in having manual immutable Version and Release tags in the spec.
@nim, that's putting cart before the horse. Pre/post version info is a fundamental, unseparable part of the upstream version. You can certainly further dissect it into major, minor etc parts but the version itself is a whole, that much would be pretty damn obvious if people were not so acclimated to the insane mapping of some parts into release.
And yes stuff built around kludges is likely to break when the kludges are removed, your macros just need to construct the actual version from the subcomponents and not the other way around.
As for human-friendliness, the versioning around tilde is infinitely more obvious to normal human beings (long-term packagers do not count) than the release-mapping. It might not be the perfect solution but you don't always get to architect perfect solutions when retrofitting stuff to 20+ year old systems.
Oh and FWIW, my +1 for @tibbs latest.
@pmatilai
Thank you for chiming in, you just wrote what I didn't have the time to write an hour ago. This is all about the fundamental, unseparable part of the upstream version
So presented this way, +1 to let people using upstream versions, tilde and all, when the upstream syntax is understood by rpm. No objection from me. That's the right way to do things.
Where the change is oversold, and @tibbs proposal won't work, is when it proposes to use tilde to move in bulk the things we do in Release to Version. Because there is a ton of things we do there which are definitely not fundamental, unseparable parts of the upstream version.
Typically "Fedora hit a serious bug, upstream accepted the fix in its VCS, but Fedora is not critical enough for upstream to release or tag an official version". That's something that happens all the time. That's what forces Fedora to use unofficial upstream version states (which are then tagged as such in the release decorations we use). And you have cases where the change has not even been accepted upstream yet, you need to tag a PR request, a bugzilla number, or something like that.
Moving those to version will break things for the very same reasons that moving upstream version parts to version will fix things. They are inherently not part of the upstream version. Yet they exist, Fedora needs to use them, and Fedora needs them tracked because it needs them and worse, is the sole responsible for them as upstream didn't deem it necessary to put them in an official release.
And (worse yet) to decide if foo VCS tag matches the upstream version (and does not need to be explicitly tracked) or if it is something else, you need to read the upstream version first. So the processing flow is
upstream version name + VCS info → identify if matches upstream official version or not → set Fedora track info and (right now) → set Release
What @tibbs proposes would lead to
upstream version name + VCS info → identify if matches upstream official version or not → set Fedora track info and (@tibbs change) → set version with Fedora additions
So, circular processing, that is ugly theoretically, and practically can't be done today since rpmbuild will object via assertions to setting Version to a new value if it's already been set earlier in the spec.
.
Big +1 to allow ~ in version.
Version = upstream information Release = downstream information
P.S. I already use ~ for months (mostly for local build), works perfectly, as expected.
@remi, thank you for stating what should be obvious but gotten terribly obfuscated over the years :thumbsup:
So for @nim's case of non-released upstream fixes: normally you'd cherry-pick the fix as a patch or a few == downstream decision -> bump release. But an upstream VCS snapshot is still all upstream, so the info belongs into version, arguably. Technically the packager could do a diff between the upstream release and VCS, add it as a patch and just bump release. But if you take foo-1.2 tarball and apply the delta between 1.2 and 1.3 as a patch, calling the result 1.2 would be fairly bizarre...
I agree that we finally need to get this proposal approved. The current "solution" is a hack. We should not keep cargo-culting it when a better solution has been available for FOUR YEARS (!) now.
@nim:
As a packager and macro writer, I don't find tilde any good, or intuitive: it's just another messy way to shove two distinct informations in a single rpm tag. As all such systems it will at best be hugely inconvenient for humans
it's just another messy way to shove two distinct informations in a single rpm tag. As all such systems it will at best be hugely inconvenient for humans
I don't see how the prerelease information is distinct from the version. 1.2.3, 1.2.3~alpha2, and 1.2.3~20181023gitdeadbeef are all different upstream releases or snapshots, with completely different upstream tarballs. I don't see how calling them all 1.2.3 helps in any way.
1.2.3
1.2.3~alpha2
1.2.3~20181023gitdeadbeef
worse, it moves the mess from release to version, so it's not just a kludge format change, it invalidates the existing kludging in release, and forces you to invent new kludging for version
Some hackery may still be needed to keep things sortable, but that has always been the case, even with pure Version (e.g., if upstream treats version numbers like regular floats and releases 3.1, 3.11, 3.2 instead of 3.1, 3.1.1, 3.2 or 3.10, 3.11, 3.20).
3.1
3.11
3.2
3.1.1
3.10
3.20
it replaces simple release separators like dot and 0 with exotic tilde and +
Because this is necessary to help the sorting algorithm.
it makes version unusable as version for anything else than rpm itself (so for example you can no longer use version to identify the upstream release id, as is done by the forge macros in redhat-rpm-config since last year)
Define "release ID", especially in the case of a VCS snapshot, which upstream has not released to begin with.
In addition, if both ~ and + were used, you could just truncate at the first character in [~+] if you really want only the current Version, so this is actually an argument for using + instead of . for post-releases.
~
+
[~+]
Version
it's not any more automatable than the current system. It's only automatable if you assume everything not a semver is a pre-version, which clearly is not the case. With the generalised use of git you have commits that can occur both pre and post release, they are absolutely the same from an automation POW, the information "it's a pre-release" or "it's a post release" can not be automated it's a human judgement call
This judgement call is already necessary to actually comply with the current guidelines. It is just being ignored by those automated scripts, but that does not mean that it is not there.
In the current guidelines (and omitting %{?dist} for simplicity):
%{?dist}
Release: 0.n.alphatag
Release: n.alphatag
n>0
So the proposal does NOT introduce a new requirement that is not already there.
Pretending that 1.2.3~20181023gitdeadbeef is an upstream version string, when upstream never released such a thing, does not help either.
as @pmatilai wrote: a fundamental, unseparable part of the upstream version as @remi wrote: upstream information
Upstream has not released → it's not an upstream version or an upstream information, it's a Fedora Release part
As I explained the @tibbs proposal leads to putting back the info Fedora wants to add, in the original version field. So even if you start by truncating you end up rewriting the rpm Tag you started with
That does not work technically in today's rpm.
it's not any more automatable than the current system. This judgement call is already necessary to actually comply with the current guidelines.
it's not any more automatable than the current system.
This judgement call is already necessary to actually comply with the current guidelines.
So you violently agree with me here.
When you do things manually, no it does not. Humans can cope with fuzzy logic and read between lines. When you start automating parts of it, the @tibbs proposal just does not work.
So again, +1 for accepting tilde in version, assuming it is actually used to express upstream release versions, -1 for using it as a pretext for moving Fedora downstream information from Release to the upstream Version tag.
I think the proposal I submitted is broken in pretty much the same ways that the current non-tilde-using guidelines are broken. Though I will say that if we're really going to revisit this, the proposal needs to be gone over again to look at a couple of corner cases that I was unable to adequately address.
Fundamentally versioning simply hard as soon as you get too far from having a single monotonically increasing positive integer. Both the current scheme and the proposal are simply attempts to deal with the fact that we want a computer to be able to discern "newer" out of information which isn't uniformly sortable by a simple algorithm. (For example "alpha, beta, gamma, delta", or a sequence of git tags.) So all of the corner cases are about sticking an integer somewhere.
We could certainly use a hybrid scheme where tilde is allowed only for versions actually named by upstream, but, well, I simply am not going to write that up. I simply don't have the time or the will. I'm not opposed to it on its face but I just can't do the work (again).
What I really don't want is any situation where don't have uniform rules which produce a single acceptable set of Version: and Release: tags for any particular situation. We really must give clear guidance and we can't leave it up to the packager to just insert a tilde or not because they felt like it. Versioning is one of the most difficult things to get right. Not because our guidelines are terrible, but because it's a difficult problem. If we have any duty at all, it is to provide packagers with a clear way to do this which is simple in the simple cases.
I will note that there is a third possibility which involves using Epoch. Now, we all hate Epoch but given the way that distro updates work now, it is technically possible for a package to go "backwards" in versioning, including dropping Epoch, between Fedora releases. The only caveat is that rawhide users would "periodically" need to use distro-sync. (Basically that means "always" if they want to ensure that they're up to date.) Obviously this is a policy issue which would need to be decided by FESCo and can be addressed separately. If it was acked there then it would give us another option.
And also, we do need releng buyin here if we do want to adopt tildes in any way, because we know that some things do break if someone tries to push a package with a tilde in the version. We know this because it seems remi decided to push an update for php in F29. (Disagree with a rule? Ignore it!) I got it untagged, but not before it exposed some issues in the release infrastructure which will need to be addressed. For example, some links within koji break: https://koji.fedoraproject.org/koji/search?terms=php-7.2.12~RC1-1.fc29&type=build&match=glob
Here is an exercise for those who want to propose a different scheme than the current one or the tilde-using writeup I made. What Version: and Release: tags (and Epoch: when needed) would your scheme mandate for the following release sequences:
There are more things there than are strictly necessary for me to get a feel for how a different scheme would work but I lack the time to come up with a minimal set. I could probably write a set of guidelines from answers to the above assuming that the scheme doesn't differ on the more normal cases.
Edit: I did mean gamma instead of beta in the first case. Sorry about that.
Also, I should add that it looks like remi has been doing whatever he feels like doing with the PHP package for some time now and has been using tildes in his versions for most of this year and perhaps longer. I guess he just doesn't care for any rules at all if he disagrees with them.
@tibbs, tilde versioning does not have to replace what we have been doing for years. It has been designed to keep sortable stuff easier. Like 1~alpha1 -> 1~beta1 → 1.
1~alpha1
1~beta1
1
IMO we should draft how to use these and say that if it doesn't fit, use old-school Release mangling. Do you see any problem with that?
Well certainly it would replace some of what we have been doing for years. And I know what it does.
It does not suffice to draft "how to use it" as some kind of addendum to the existing guidelines. It needs to be properly and clearly integrated into the process so a packager can come to the document with an upstream versioning mess come out with a clear idea of what they should put in their Version: and Release: tags.
So go ahead, look at the stuff I typed in above and show me your scheme. It certainly won't match either the current guidelines or the proposal I provided, so those answers will provide the basis for a new document which tells someone how to implement your scheme. I can't do that because frankly you haven't presented enough of a concrete proposal to guess at what versioning you would come up with in the above-listed situations.
I am listing only Version, not Release, unless specified otherwise.
Release
First case:
1.4~gamma1
1.4~gamma2
1.4~beta2
1.4~1.beta2
1.4~delta1
1.4~1.delta1
1.4~zeta1
1.4~1.zeta1
1.4~1.RC1
1.4~2.RC1
1.4
Second case:
1.4+0.20180722gitf6bd018
1.4.0.20180722gitf6bd018
1.4+0.20180816gitf6bd018
1.5~0.20180925gitcdb4135
1.5~0.20181013git244f240
1.5~1.beta1
1.5~20181014beta1
1.5~1.20181015gitc5ed1cf
1.5~2.TC1
1.5~3.RC1
1.5
Third case:
1.5+GA1
1.5+1.CP1
Fourth case:
0~0.20180926gitcb0aa0d
Version: 0
0~alphatag
0.
0~0.20181017git2c41e9f
2018.1
Fifth case:
20090825
2010.1
This is both how I would do things and how I would interpret your proposed guidelines (which say that the integer kludge should be used only when needed).
As for making things maximally consistent: While I normally believe in consistency, I think that in this case we need to allow for the variability of making the integer (e.g. 1.5~1.beta1) optional, so that we do not have to use this kludge in the common case where it is not needed (1.5~alpha1, 1.5~alpha2, 1.5~beta1, 1.5~beta2, 1.5~rc1, 1.5~rc2, 1.5), but can still use it (and if we do it right, switch to it at any time) in the corner cases you pointed out (e.g., if you insist on upper-caps RC, you have to switch to 1.5~1.RC1, 1.5~1.RC2).
1.5~alpha1
1.5~alpha2
1.5~beta1
1.5~beta2
1.5~rc1
1.5~rc2
RC
1.5~1.RC1
1.5~1.RC2
I guess he just doesn't care for any rules at all if he disagrees with them.
Yes I'm a very bad guy. Throw me some stones.
Or perhaps you can ask yourself how is it possible than a 4 years old issue is not yet fixed. Or perhaps you should take benefit of months of experience using tilde in versions.
@kkofler My proposal (that @tibbs has helpfully cleaned up quite nicely) more or less works the same way, though I flip git/snap/etc. to the beginning of the date stamp rather than the end so that we reduce the number of cases a digit or letter has to be prepended.
git
snap
@tibbs For what it's worth, I like your cleaned up version of my proposal. :smile:
@ngompa Where do you put the revision/commit ID then? Or do you omit it? What I like about the existing yyyymmddgitcommitid scheme is that no dedicated separator is needed between the date and the ID.
I don't see this is going somewhere. I filed separate two ticket for my small use cases which hopefully won't be that controversial:
#818 - Allow exception for tilde in bootstrap macros #819 - Allow use of tilde for Ruby generators
added #820 - Allow use of tilde for PHP (and some extensions)
gityyyymmdd.shorthash. Yes, there's a separator, but IMO, that makes it clearer for people to understand what is the commit has vs the snapshot date.
gityyyymmdd.shorthash
This needs saying: It is not proper to say that this isn't going anywhere. It went somewhere and the answer was no. Why is that so hard to understand? Do not use a tilde in a version. End of story.
Now, that doesn't mean we can't still discuss it. But saying that you need to work around the process (which has already worked) just because you don't like the result isn't how things should be done.
I'll respond to other things in a separate comment.
I fully agree with @remi, I don't see why we still need to keep working around a deficiency in RPM that was fixed more than 4 years ago.
Why not remove the single line prohibiting tilde from the guidelines, and let packagers experiment and develop best practices as @remi has been doing? Don't need to have complex rules for everything.
@jorton I've already provided an answer to that question. We need clear, consistent rules. If everyone is going to just make things up, we might as well not bother having guidelines. If you want to experiment, all you need to do is run rpmdev-vercmp a bunch of times to see how things work; nothing prohibits you from doing that. Just don't push packages using that into the distro.
@nim: You appear to argue stridently that we shouldn't put snapshot information into Version:. It sounds like you're arguing against my proposal, but that can't be right as what I wrote keeps snapshot information in Release: as it is currently.
And note that we must have some standard for versioning, because automated tools must still be able to bump a release. And simply because we really do owe it to anyone who might ever want to know what version of something we actually package.
Some commenters (here and on IRC) appear to be confused about my personal stance, so I'll state it here: I would like to adopt something which simplifies the common case. I personally support the use of tilde, even though if you read the beginning of this ticket, I voted against the change. Mainly because it wasn't accompanied by a set of clear guidelines as to how the tilde would be used.
Regardless of my opinion, a majority of FPC members have not voted to change the existing prohibition and adopt new rules for package versioning. I'm certainly going to campaign against ignoring rules as fundamental as these, and I'm going to resist any effort to introduce random exceptions which provide no benefit except to placate the people who for whatever reason don't feel like following the existing rules.
And finally, I need to respond to @kkofler's detailed answers. But if I don't submit this comment then I may get busy and lose it.
@tibbs
Yes, right, sorry about this, I reacted to things other people wrote, and I misunderstood this part of your proposal. I worry still that others will misunderstand your proposal the same way as I did.
The main suggestions I’d make to avoid this would be:
Thank you for providing those answers I will assume that in all cases, Release (ignoring %dist) is simply a plain integer that starts at 1 as normal and is incremented for any Fedora rebuild that doesn't also change Version or Epoch. Sorry about the confusion between beta/gamma in that one place; I edited my message to fix that. I had intended to respond yesterday but got busier than expected.
It appears that your proposal moves more into Version than mine does. That's fine, but seems in opposition to @nim and a comment by @ignatenkobrain. Personally I like the idea of that, rather more than I first realized, though I don't like 100% of the proposal. Let me try to summarize it, using the language of the current guidelines as well as the tilde proposal I wrote up. Note that this is just a quick thing and the terms are not well chosen.
In all cases, use Release: <positive integer>{%?dist}.
Release: <positive integer>{%?dist}
No change for "simple versioning".
Prerelease versions:
Version: <nextversion>~<sort><prerelease>
<nextversion>
<sort>
<prerelease>
<snapinfo>
Postrelease:
Version: <pkgrelease>+<sort><postrelease>
<pkgrelease>
<postrel>
Note that @kkoffler's answers show <sort> starting at zero so I've stated "nonnegative integer" above. I'd probably start it at 1 ("positive integer") because in this proposal there is no special meaning ascribed to things between 0 and 1.
One important thing everyone should keep in mind is the how integers and letters sort:
ἐπιθυμία:~❯ rpmdev-vercmp 1.4~zeta1-1 1.4~rc1-1 1.4~zeta1-1 > 1.4~rc1-1 ἐπιθυμία:~❯ rpmdev-vercmp 1.4~zeta1-1 1.4~1.rc1-1 1.4~zeta1-1 < 1.4~1.rc1-1
This is what permits the introduction of the integer after the fact to ensure ordering without Epoch:. It's something we can't really rely on currently because of how we need that integer to be there for rebuilds and such.
There has been some discussion about whether a date or git hash is part of the upstream version or is some Fedora-specific thing. I would argue that it's really both, but mostly it comes from upstream. After all, we don't choose the git hash; that comes from the upstream repo. But we do choose the date, though of course it is bounded by the date when upstream made the commit, so it's not entirely up to Fedora. Personally I don't have any problems including the snapshot info in Version:, because it completely simplifies Release: down to an integer plus dist.
Version:
Release:
Now, what I don't like about this: The post-release case where you have to take 1.5patch1 and turn it into 1.5+patch1. Yes, the plus shows that it's a post-release thing, and that's fine, but it's a change that I think most will get wrong because we're just not used to doing things that way. Obviously we have no choice for prereleases because the tilde must be there. I just think that things work fine without it, except on the odd chance that upstream introduces an unsortable post-release sequence. And for that, then you can insert + and an integer, conveniently avoiding epoch:
1.5patch1
1.5+patch1
ἐπιθυμία:~❯ rpmdev-vercmp 1.4patch1-1 1.4+1.fix2-1 1.4patch1-1 < 1.4+1.fix2-1
It even works if they do terrible things like moving from 1.4.1 to 1.4fix2:
ἐπιθυμία:~❯ rpmdev-vercmp 1.4.1-1 1.4+1.fix2-1 1.4.1-1 < 1.4+1.fix2-1
Finally, the use of +. I don't have a problem with it, and it does indicate post-release nicely. It's not a problem throughout infrastructure (the cdparanoia version has included it forever), and as @kkoffler indicated, it makes no difference to RPM. I don't even think we need to worry about the case where upstream uses plusses in their versions already, except that it could potentially lead to a doubled plus if someone follows the above rules exactly.
I'm tired of typing now.
@nim: The proposal exactly follows the structure of the current guidelines, and so is presented as a minimal change from them. I had already rewritten the guidelines into the current state and wasn't about to rewrite them again. If you want to have a go, you are more than welcome to do so.
Now, to @kkoffler's messages It appears that your proposal moves more into Version than mine does. That's fine, but seems in opposition to @nim and a comment by @ignatenkobrain.
Now, to @kkoffler's messages
It appears that your proposal moves more into Version than mine does. That's fine, but seems in opposition to @nim and a comment by @ignatenkobrain.
Yes, I confirm:
from a practical POW that would require me to rewrite version unless I add a real-upstream-version variable to my specs, and I won't do that, so it’ll go directly to the rpm issue tracker as "allow rpm macros to rewrite Version" with the guideline being ignored till rpmbuild is fixed in all the releases my specs target.
… or we will fix it ourselves :)
@tibbs;
First of all, there is only one 'f' at "Kofler" (and the 'k' is only doubled because my first name is "Kevin"). That gives @kkofler.
Thank you for providing those answers I will assume that in all cases, Release (ignoring %dist) is simply a plain integer that starts at 1 as normal and is incremented for any Fedora rebuild that doesn't also change Version or Epoch.
Normally yes, though I wouldn't remove the optional .<minorbump>, e.g., Release: 1%{?dist}.1 (e.g. 1.fc27.1), for branch-only bumps (https://fedoraproject.org/wiki/Packaging:Versioning#You_need_to_change_an_old_branch_without_rebuilding_the_others).
.<minorbump>
Release: 1%{?dist}.1
1.fc27.1
It appears that your proposal moves more into Version than mine does.
Yes.
As explained above, IMHO, this should be "<positive integer>{%?dist} or <positive integer>{%?dist}.<other positive integer>". Otherwise, we get badly stuck if we need to do some bump on a branch only.
<positive integer>{%?dist}
<positive integer>{%?dist}.<other positive integer>
What I wrote was actually more subtle: If I introduced <sort> from the beginning, I started it at 0, but if I bumped it, I treated empty <sort> as 0 and hence bumped directly to 1. But this may actually be too complicated to explain and follow, so we can simplify it.
I also treated <sort> kinda like Epoch and only bumped it when actually needed. Your writeup is more like the current guidelines which say to always bump it when you bump the snapshot/prerelease (but then again, in the current guidelines, the integer is also the Release counter and so will be bumped even if there were no upstream changes at all, which would not happen anymore with the new guidelines that clearly separate Version from Release).
Epoch
from a design cleanliness POW injecting Fedora downstream info (choice of a particular git commit, or patch, or whatever) in upstream versioning is plain wrong
I don't see how the upstream git commit is downstream info. This is neither a downstream patch nor a packaging change, but a different git commit normally results in a completely different Source tarball (unless you use git am with a patch sequence from the last preceding release, but in that case, you probably should just treat your patch sequence as a downstream backport and omit the snapshot information from both Version and Release entirely).
git am
I would really like to see Release working again as intended, i.e., counting the number of Fedora packaging versions for the same given upstream version. A different snapshot is a different upstream version, even if it has not been officially released.
This comment has been heavily edited.
Comparison of the current guidelines, my original proposal ("tilde proposal 1") and a third one ("tilde proposal 2") based somewhat on @kkofler's proposal above. I will try to write up the second one soon if I can find some time.
Oh, and I just noticed that https://docs.fedoraproject.org/en-US/packaging-guidelines/Versioning/ seems to have fared poorly in the conversion to asciidoc. I guess I'll need to look into that.
I should also point out that @kkofler's answers for the first version sequence go backwards from 1.4~gamma2 to 1.4~delta1. Which is kind of why I asked that question. Obviously the real intent is to introduce an integer at that point. The proposal I submitted sort of talks about it but isn't prescriptive so I've adopted the "insert integer" convention here. Of course the current guidelines have no such messiness.
A difficult prerelease version sequence:
So both of the "new" proposals agree here after fixing issues in the new one. I think it is relatively clear that using tilde without forcing an integer for sorting when not necessary is a bit simpler. If you don't already know the "release <1 implies prerelease" rule of the current guidelines and don't understand tilde, nothing except internalized understanding of what the Greek letters or "RC" mean actually suggests that these are prereleases. Once you understand either tilde or the "release <1" rule then all are pretty obvious, but the simplicity of the Release: field for the two tilde-using proposals is, to me, quite compelling.
A sequence moving from post-release to prerelease with tagged releases interspersed:
I know this is a contrived example, but I really don't think it outlandish to think that someone might tag a beta in with some prerelease snapshots. It shows a few interesting things:
Our current guidelines are quite consistent and elegant once you get past the overloading of Release:.
Proposal 1 handles this situation quite poorly. I'm tired and I've now done this work twice, but I'm still not seeing how to fix it completely. Either all of the version information is in Version: or only the "primary" version bit is there, or we designate prerelease versions using the previous release number, or something else. Nothing I can think of is really palatable.
Proposal 2 is about as elegant as the current guidelines at this point.
Upsteam never provided a version:
No big surprises here.
My intent was not to remove the minorbump field, only to write up how that proposal differed from the others. I didn't write up other places where it did not differ.
I will edit my previous misspellings of your user ID.
(snip for readability)
Just curious, why don't we just do Release: 1%{?dist} → Release: 1.1%{?dist} here? Adding the .1 bumps after the %{?dist} always felt wrong to me for some reason.
Release: 1%{?dist}
Release: 1.1%{?dist}
.1
I don't see how the upstream git commit is downstream info. This is neither a downstream patch nor a packaging change, but a different git commit normally results in a completely different Source tarball (unless you use git am with a patch sequence from the last preceding release, but in that case, you probably should just treat your patch sequence as a downstream backport and omit the snapshot information from both Version and Release entirely). I would really like to see Release working again as intended, i.e., counting the number of Fedora packaging versions for the same given upstream version. A different snapshot is a different upstream version, even if it has not been officially released.
↑ yes, this. :thumbsup:
Well that was terrible; I spent 45 minutes editing my comment. When this new comment came in my edit window reverted to the original content.
I'm sorry, but I've spent so much time on this today.
@decathorpe: Because the whole point of doing those bumps is to preserve ordering between Fedora versions without bumping all Fedora versions.
ἐπιθυμία:~❯ rpmdev-vercmp 1.5-1.1.fc27 1.5-1.fc28 1.5-1.1.fc27 > 1.5-1.fc28 ἐπιθυμία:~❯ rpmdev-vercmp 1.5-1.fc27.1 1.5-1.fc28 1.5-1.fc27.1 < 1.5-1.fc28
I should also point out that @kkofler's answers for the first version sequence go backwards from 1.4~gamma2 to 1.4~delta1. Which is kind of why I asked that question.
Oh yeah, nasty greek letters not transliterating alphabetically. I fixed the example now (and kept the old one stricken through for reference).
As for your tables: I'd propose actually not going: 1.4~1.delta1-1→ 1.4~2.zeta1-1→ 1.4~3.RC1-1 but: 1.4~1.delta1-1→ 1.4~1.zeta1-1→ 1.4~2.RC1-1 i.e., bump the integer only when actually needed. This is more intuitive especially if we have ~1.beta1, ~1.beta2 etc., where only the number of the beta really changes.
1.4~1.delta1-1
1.4~2.zeta1-1
1.4~3.RC1-1
1.4~1.zeta1-1
1.4~2.RC1-1
~1.beta1
~1.beta2
As for mixing prereleases and snapshots, you make a good point there. This pretty much shows that snapshot information belongs into Version, not Release. The only way to get it right with the Release variant would be to use 1.5~beta1-0.1.20180925.cdb4135 for the pre-beta snapshots, but that is only possible with upstreams that are organized enough that you can know what the next prerelease will actually be. Some upstreams are way too ad-hoc to be able to predict it reliably.
1.5~beta1-0.1.20180925.cdb4135
@decathorpe: Because the whole point of doing those bumps is to preserve ordering between Fedora versions without bumping all Fedora versions. ἐπιθυμία:~❯ rpmdev-vercmp 1.5-1.1.fc27 1.5-1.fc28 1.5-1.1.fc27 > 1.5-1.fc28 ἐπιθυμία:~❯ rpmdev-vercmp 1.5-1.fc27.1 1.5-1.fc28 1.5-1.fc27.1 < 1.5-1.fc28
@decathorpe: Because the whole point of doing those bumps is to preserve ordering between Fedora versions without bumping all Fedora versions. ἐπιθυμία:~❯ rpmdev-vercmp 1.5-1.1.fc27 1.5-1.fc28 1.5-1.1.fc27 > 1.5-1.fc28
ἐπιθυμία:~❯ rpmdev-vercmp 1.5-1.fc27.1 1.5-1.fc28 1.5-1.fc27.1 < 1.5-1.fc28
Oh. That should have been obvious :see_no_evil: Sorry for posting a stupid question and making you lose your edits. Guess I really should sleep instead of posting here at 2AM ...
Several points:
I really see snapshot and pre-release as two distinct things. Pre-release is presumably official upstream provided tarball, where the snapshot is something we decide to do. Therefore I'd prefer if pre-releases goes into version and snapshots are kept in the release.
It is nice that you think about all possible corner cases and pre-release naming scenarios, but at the end, what is the most important thing is that no matter how bad the upstream pre-release naming is, the tilde should allow me to get the official stable release, although there is first released "zeta" followed by "rc1". That said, I am quite sure the <sort> will be the most forgotten feature of these guidelines.
Do we really have to care about "post-release"? Just update the tarball and bump the release works just fine if I am not missing something.
All in all, we should really discourage usage of pre-releases and snapshots IMO (unless we discourage it already). These should go just into Rawhide and just in exceptional cases (although I understand that usage of the latest code from some forge is very modern these days).
Unless I am mistaken, the correct version of line "1.4gamma2 1.4-0.3.gamma3 1.4~gamma2-2 1.4~gamma2-2 Fedora rebuild" should be s/1.4-0.3.gamma3/1.4-0.3.gamma2/.
Also I'd like to provide my own table and compare unexperienced packager versioning:
Upstream version | First time packager | Packager heard about pre-release | Packager knows tilde | Comments ---- | ---- | ---- | ---- 1.4gamma1 | 1.4gamma1-1 | 1.4-1.gamma1 | 1.4~gamma1-1 | 1.4delta1 | 1.4delat1-1 | 1.4-1.delta1 | 1.4~delta1-1 | Unsortable 1.4zeta1 | 1.4zeta1-1 | 1.4-1.zeta1 | 1.4~zeta1-1 | Continue bumping the integer 1.4RC1 | 1.4RC1-1 | 1.4-1.RC1 | 1.4~RC1-1 | Also unsortable 1.4 | 1.4-1 | 1.4-1 | 1.4-1 | The release version
These are IMO examples how the current guidelines works in reality. If the packager knows about tilde, then there are better chances that I'll get installed the stable version at the end.
Also it works better for generators (#819), since in generator, I am afraid I won't be able to apply <sort> somewhere.
@vondruch
Do we really have to care about "post-release"?
One reason we get post releases nowadays is the following common scenario
%checks
upstream commits and does not release
Well, we are applying patches for various reasons. I don't consider this post-release.
Post-release, IMHO, would be if upstream actually updated the tarball, but without changing the name.
But none of these requires some special treatment IMO.
none 0-0.1.20180926.cb0aa0d 0-0.1.20180926.cb0aa0d 0~1.20180926.cb0aa0d-1
In situations where upstream has never provided a version, switching to using a tilde after the zero will present upgrade path issues for packages following the existing guidelines.
$ rpmdev-vercmp 0-0.1.20180926.cb0aa0d 0~1.20180926.cb0aa0d-1 0-0.1.20180926.cb0aa0d > 0~1.20180926.cb0aa0d-1
Using a plus after the zero would work better.
$ rpmdev-vercmp 0-0.1.20180926.cb0aa0d 0+1.20180926.cb0aa0d-1 0-0.1.20180926.cb0aa0d < 0+1.20180926.cb0aa0d-1
@carlwgeorge Using a plus is exactly the wrong thing because your version is now greater than zero. Upstream could release version zero and you'd be stuck.
Packages using the existing guidelines will have to be careful if they want to switch to some theoretical new guidelines. That's not really any different than any change we make.
Upstream could release version zero and you'd be stuck.
Seriously? Has anybody here ever seen an upstream release a version zero? FWIW, in my 24 years of open source, I don't recall seeing version zero anywhere but when introduced by packagers.
My point being, abandoning a practical solution just because a theoretical exception exists on paper ... perfect being the enemy of good and so on.
I've seen v0 APIs, but never v0 packages. So I think this is pretty much a theoretical concern.
v0
Well, the point is that if you use ~ then it's less than anything. So you don't have to think about what upstream might release.
So you've never seen 0 or 0.0. I have, but we can easily ignore that. After all, I'm pretty sure you've seen 0.1. How do you go from the above proposed 0+1.20180926.cb0aa0d-1 (or 0.1.20180926.cb0aa0d-1 or whatever) to the released 0.1? Sure, you can say that we should have started with more zeroes and used 0+0.1.20180926.cb0aa0d-1. Yet I've seen version 0.0.1 as well. How many zeroes is enough? I know that people are just going to accuse me of making things up when I ask questions like this but everything I'm talking about is a practical concern that I've seen in actual packaging. We talk about "version 0" because we simply can't know where they'll start.
Now, if we could use Epoch: more freely, we wouldn't really need to care so much about many of these corner cases.
Anyway, I've also just come to realize that the tilde magic gives us enough to mostly manage prerelease stuff but that postrelease stuff is still impossible without either cramming things into Release or having a tilde-like rpm version comparison "operator" that works like this (with '+' standing in for this mythical thing):
foo-1.4 < foo-1.4+snapshotstuff < foo-1.4.1
Because otherwise there is no sequence of post-release snapshots that you can use which get recorded in Version that don't interfere with whatever upstream might put out next. And so allowing tilde in version and having to keep some snapshot info in release gives us either:
I cannot think of a scheme besides the one we currently use which avoids doing one of those. At least what we use now is consistent. But people really, really want tilde because it sort of simplifies the one case want and are willing to hope they never see any of the other cases.
For the zero version problem, actually there's no problem whatsoever when one stops to think about it: if upstream hasn't released anything yet, it's simply a pre-release snapshot, and so you use tilde: 0~20180926. Which will work even if upstream releases version 0.
"post prerelease snapshots" are not a problem: [pmatilai@sopuli ~]$ rpmdev-vercmp 1.4~beta1 1.4~beta1+20180926 1.4~beta1 < 1.4~beta1+20180926 [pmatilai@sopuli ~]$ rpmdev-vercmp 1.4~beta1+20180926 1.4~beta2 1.4~beta1+20180926 < 1.4~beta2
It's "post release snapshots" that are (ie the foo-1.4 < foo-1.4+snapshotstuff < foo-1.4.1 case), although AFAICT you can sort that out by adding a zero in there: 1.4 < 1.4+0.20180926 < 1.4.1
Sure it's ugly, but the same artificial zeros get added to release in several scenarious so it's not exactly anything new.
Yes, exactly, and that's exactly, precisely, to the letter what I proposed originally. So, great!
As for "sure it's ugly":
Just an update: I have now generated three drafts:
I have not yet gone through and tried to torture the caret draft yet, and all of these still need some linting which I'll be working on. In a couple of cases I also have open questions that I'll need to work out.
Since this Thursday is the US thanksgiving holiday, I doubt we'll be meeting but I still wanted to present these to everyone.
The "TildeVersioning2" is probably the best variant, which could be used right now, but this [1] change is just muddying water IMO.
Nevertheless, netiher of the proposals covers the bootstrap macros [2], because the guidelines focus too much on some specific scenarios instead of applying general principles.
BTW "TildeCaretVersioning" should be treated separately, becase:
Please note that I have opened ticket #832 to present the current situation to present the current situation and track a vote on the TildeVersioning2 draft.
That ticket treats TildeCaretVersioning separately, as was always intended. People asked me how the guidelines would look with caret as part of an argument for getting it implemented, so I produced them. It's so much better that I believe the right thing to do now is to minimally patch things to allow tilde in some situations and then do a full overhaul once we have caret.
Bootstrapping needs to be considered separately as well. There are only so many different things I can concentrate on at once. If it looks like I'm done tweaking these drafts, then I can work on that one.
Bootstrapping needs to be considered separately as well.
So my #818 won't be useless at the end. I am fine with that. Thx
Finally closing this as the current guidelines permit some use of tilde. Upstream RPM now supports both tilde and caret, which means that a more complete system will be available at some point in the not too distant future. At that time, I'll propose much-simplified guidelines which make use of both.
Metadata Update from @tibbs: - Issue close_status updated to: nothingtodo - Issue status updated to: Closed (was: Open)
Login to comment on this ticket.