Ticket #31 (closed change: fixed)

Opened 7 years ago

Last modified 6 years ago

New Wiki

Reported by: mmcgrath Owned by: ricky
Priority: major Milestone: Fedora 10
Component: Web Application Version: Production
Severity: Normal Keywords:
Cc: poelstra@…, petersen@…, ianweller, ivazquez, jstanley Blocked By:
Blocking: Sensitive:

Description

Its time to investigate a new wiki. Moin just isn't cutting it. We need to

1) Determine what needs we have out of a wiki

2) Determine what wiki's are out there

3) Figure out if migration is possible

4) Proof of concept

5) Choose

6) Implement

Change History

comment:1 Changed 7 years ago by poelstra

  • Cc poelstra@… added

comment:2 Changed 7 years ago by jcollie

I've been investigating ikiwiki for some personal use. It might be interesting to see if it'll work for Fedora. The main benefit is that it compiles the wiki markup into static pages, thus reading pages ought to be extremely fast. Josh Triplett has been working on converting a Moin to ikiwiki. The main downside is that it's implemented in Perl...

comment:3 Changed 7 years ago by jcollie

here is a start at defining the requirements for a new Wiki.

comment:4 Changed 7 years ago by petersen

  • Cc petersen@… added

comment:5 Changed 7 years ago by stahnma

The discssion on requirements for a new wiki seems rather short-sited. Ruling out PHP because of a security track-record doesn't hold a lot of water IMO. Yes, PHP has a bad track-record, mostly due to poorly written applications that happen to have 'PHP' in the name, not the lanugage itself. Also, the top wikis in the world use Mediawiki (PHP) and we don't read headlines of them constantly being destroyed, hacked, or whatnot. The wiki should apply appropriate security controls and evaluations before being chosen. It shouldn't be "not this one because it uses foo."

I am just asking to be fair in the evaluation of product and not state bias in a requiments document.

comment:6 Changed 7 years ago by ricky

  • Keywords Meeting removed

comment:7 Changed 6 years ago by mmcgrath

Just an update to this ticket. We ended up choosing mediawiki for a number of reasons, all of which I'll link to (it was discussed on mailing lists (fedora-infrastructure-list) and in meetings.

comment:8 Changed 6 years ago by ianweller

  • Cc ianweller@… added

i'm willing to help with whatever needs done. i have a significant amount of MediaWiki? experience, especially with Wikitext syntax.

comment:9 Changed 6 years ago by ianweller

Here are a list of changes necessary to the communication script so far, in a sort of outline format:

  1. Note/message/warning boxes needed to be fixed
  2. (non-communication script) Clicking the Fedora logo on the wiki goes to /w/Index, which 404s. Should be changed to /wiki/Main_Page.
  3. Internal links for some reason are being translated into sorta-kinda external links (for example, Main Page, shows 'Overview' as an external link to the wiki page Overview, instead of an internal link...)
    • Old format:
      [http://fedoraproject.org/wiki/Overview Overview] for a quick overview on what makes Fedora, Fedora.
      
    • New format:
      [[Overview]] for a quick overview on what makes Fedora, Fedora. 
      
  4. <tablestyle> tags fail. Check the MediaWiki syntax on tables to fix this...
    • Also <tableclass>
  5. User pages (such as IanWeller) should be moved to proper User namespaces (i.e.: User:Ianweller, if following FAS scheme)
    • Redirect recommended
    • First letter of each page is always capitalized, so 'spot' turns into User:Spot, mmcgrath turns into User:Mmcgrath... not sure if we wanna do it this way or User:Ian Weller, or User:IanWeller... however it should be the User namespace.
  6. [[MailTo()]] needs to be fixed -- I recommend just changing it to 'ianweller AT gmail DOT com' format. Maybe there's an extension to do what Moin's [[MailTo()]] does...
  7. Every single attachment needs to be checked for duplicity, then 'uploaded' using a sane file name, and included using the following syntax:
    [[Image:Filename.png]]
    
    • Or linked to using the following syntax:
      [[:Image:Filename.png]]
      
    • We should have SVG rendering support now, btw
  8. Double backticks () need to be removed, there is no autolinking
  9. Should emoticons and other symbols be fixed?
    [[Image:Emoticon smile.png]]
    
  10. Take a look at ConsensusProcess (old wiki)...
    • fails at list nesting...
  11. Setup different namespaces for different languages
    • See if we can implement the 'in other languages' sidebar that Wikipedia has

Also will we be dealing with spaces in page names, as in moving from no spaces to spacing? It's possible that changing the page name to what's in the level 1 header (= Level 1 header =) might work for most pages, however we'd probably lose the directory-like structure. *shrug*

This would also be a wonderful time to clean out pages that could be deemed useless. One way to look for these is see what pages are very short, or are not linked to at all...

A team probably needs to be created so that we can crawl the entire wiki manually and fix any mistakes we see in one surge, however I don't see this as being very plausible.

comment:10 Changed 6 years ago by ivazquez

  • Cc ivazquez added

comment:11 Changed 6 years ago by mmcgrath

WOW, thanks for these markups.

comment:12 Changed 6 years ago by ianweller

no problem, let me know the next time you run the wiki through the communication script so i can keep looking for more problems.

comment:13 Changed 6 years ago by ianweller

a few design notes I want to make on the wiki --

  1. quite a bit of stuff in the header seems out of place.
    • I wouldn't put the user links in the blue bar at the top, I'd have them just under
    • I'd move the article tabs over to the right so that the start of the first tab is as much to the right of the left side of the title of the page as it is now to the left.
    • the article tabs also should be moved down so that they line up with the end of the header gradient
    • what is that third column doing there?
  2. footer is all weird
    • padding on the right after each item shouldn't be there

comment:14 Changed 6 years ago by ianweller

also, i think the communication script can kill all the MoinEditorBackup? pages. those pop up a lot when i hit 'random page'

comment:15 Changed 6 years ago by ianweller

  • Cc ianweller added; ianweller@… removed

redirects are broken... from what i can tell Moin uses a feature that denotes that on the page to be redirected to, while MediaWiki? puts a redirect directive in the page that is to redirect to something else. I tried this on PackagingGuidelines?, which redirects to Packaging/Guidelines?...

also Anchor() functions should probably be removed. MW automatically makes anchors.

comment:16 Changed 6 years ago by jstanley

  • Cc jstanley added

comment:17 Changed 6 years ago by mmcgrath

  • Milestone changed from Fedora 9 to Fedora 10

This actually won't get done in time for F9 but will happen shortly after.

comment:18 follow-ups: ↓ 19 ↓ 31 Changed 6 years ago by ivazquez

  • [ [BR]] doesn't translated into line breaks
  • Initial spaces in <pre> sections get changed to have a : in front
  • Syntax highlighters (e.g. #!cpluscplus) aren't working

comment:19 in reply to: ↑ 18 Changed 6 years ago by ianweller

Replying to ivazquez:

  • Syntax highlighters (e.g. #!cpluscplus) aren't working

There's no built-in syntax highlighting, but if we want it, this extension will do it --

http://www.mediawiki.org/wiki/Extension:SyntaxHighlight_GeSHi

If it needs packaged, lemee know.

comment:20 Changed 6 years ago by kwade

For any HTML headers (<h2>) in the Docs.* namespace, those are a workaround for the DocBook? converter in Moin. Let's convert those in the script that are in the Docs.* namespace into the equivalent <h2>foo</h2> => == foo ==.

For HTML of IRC logs, maybe convert them to attachments and create a contents page?

comment:21 Changed 6 years ago by kwade

Looking at the DocsProject? pages conversion I see on wiki/DocsProject:

http://docs.fedoraproject.org became a 1? icon in MediaWiki?. Not sure how much of this usage there is, but we need to convert that to a link that shows the URL, e.g. just a plain URL without brackets should suffice.

This is a legitimate Moin construct:

Open Publication License

It converted into MediaWiki? syntax like this:

Open Publication License

Formatting on the page like this:

[Publication License]

... with a URL of http://opencontent.org/openpub|Open

comment:22 Changed 6 years ago by kwade

Rather than convert the TableOfContents? macro from Moin, just vaporize it. MediaWiki? supplies a ToC automagically, halleluja!

comment:23 Changed 6 years ago by kwade

Some general formatting items all bunched together:

I forgot, is there a manual anchoring construct in MW? If so, can we just have all Anchor() macros converted so things don't break or require a bunch of manual intervention? Then tell people to stop making new ones, etc.

Indented bulleted lists (ref. https://publictest1.fedoraproject.org/wiki/DocsProject/WorkFlow) are converting flat without nesting. If nesting is going to be lost ... we have to see how much is potentially lost. It might be easier to wrap all lists in &lt;code /&rt; blocks and manually removing when fixing syntax. I think that'd be a better way than having to compare each page with each old page, back and forth. Better to have the proper thing inline and ready for manual fixin'.

People were taught to use the admonition graphics (/!\ etc.) and generally use these as admonitions in a consistent way. What if we defined a div class for them, then whenever the parser finds one of these image constructs, that line and any lines below until the first blank line are wrapped in a div specific to the image type. This would be consistent with how people do first and following lines with one of these images in an admonition. This would catch more true positives than leaving them alone.

Pages like these are getting wrapped in a pre tag:

https://publictest1.fedoraproject.org/wiki/Docs/Drafts/GettingStartedGuide

https://publictest1.fedoraproject.org/wiki/Docs/Drafts/SoftwareManagementGuide

https://publictest1.fedoraproject.org/wiki/Docs/Drafts/AdministrationGuide

I reckon it's from the HTML call? Those aren't really needed, maybe the conversion of those HTML segments to standard = Wiki Header = format will help? Should we add those to a list of pre-conversion tasks to complete in Moin Moin?

comment:24 follow-up: ↓ 25 Changed 6 years ago by kwade

WikiEditing? is really the canonical source for how things look, so looking at the success and details of converting that page should cover most other pages, at least where they follow the WikiEditing? conventions.

https://publictest1.fedoraproject.org/wiki/WikiEditing

comment:25 in reply to: ↑ 24 Changed 6 years ago by kwade

Replying to kwade:

WikiEditing? is really the canonical source for how things look, so looking at the success and details of converting that page should cover most other pages, at least where they follow the WikiEditing? conventions.

https://publictest1.fedoraproject.org/wiki/WikiEditing

... in particular, note how the formatting doesn't come through in the formatted examples here:

https://publictest1.fedoraproject.org/wiki/WikiEditing#Marking_Technical_Terms

The pre block goes awry here:

https://publictest1.fedoraproject.org/wiki/WikiEditing#Writing_Example_Commands

All the admonition styles are here:

https://publictest1.fedoraproject.org/wiki/WikiEditing#Notes.2C_Tips.2C_and_Other_Boxed_Text

Note that there have been some updates to the source WikiEditing? page since this test conversion was done.

comment:26 follow-up: ↓ 27 Changed 6 years ago by poelstra

Here are a few things I've found so far that are not quite there. Thanks for all your hard work on this!!!

  1. Table cell color formatting is missing
  1. links to artwork is broken
  1. Categories
  1. Message boxes and Icons
  1. Make sure pages with 'includes' get properly 'included'
  1. Table conversion needs help: https://publictest1.fedoraproject.org/wiki/DocsProject/Schedule
  1. Section headings seem overly large in comparison to the associated text
  1. handling of embedded html doesn't work
  1. Last table doesn't quite convert correctly in that the vertical bar separating "Add to Cc" and "Other Notes" is missing
  1. Are you doing a *complete* conversion from the existing wiki to the new one?
  1. Under the sections "import your package" and "Checkout the module" the code blocks/examples are blank:
  1. Several tables do not have appeared to have converted correctly:
  1. Category links don't appear to be working correctly
  1. https://publictest1.fedoraproject.org/wiki/Releases/9/FeatureList
    1. the table listing all of the features is missing
    2. at the bottom of the page it suggests a category of "Category: AcceptedFedora9 (alpha order)" which is not correct. See item 3 above as it relates to the AcceptedFedora9 category.

comment:27 in reply to: ↑ 26 ; follow-up: ↓ 29 Changed 6 years ago by mmcgrath

Replying to poelstra:

  1. Table cell color formatting is missing
  1. links to artwork is broken

I think ianweller is working on getting these up and going. Once I can see how its supposed to look like in mediawiki I can convert them.

  1. Categories

This should be fixed now (including the weird MoinEditorBackup? thing). I haven't done a full upload yet.

  1. Message boxes and Icons

I think these are related to 1) and 2)

  1. Make sure pages with 'includes' get properly 'included'

Now working... but I'm not that happy with the formatting. I'll keep working on that.

  1. Table conversion needs help: https://publictest1.fedoraproject.org/wiki/DocsProject/Schedule

Fixed but with poor formatting (I'll work with ianweller on this)

  1. Section headings seem overly large in comparison to the associated text

Thats css stuff which I think will be worked on.

  1. handling of embedded html doesn't work

This is related to - https://fedorahosted.org/fedora-infrastructure/ticket/270 and won't be enabled in the future after the migration (there are security concerns)

  1. Are you doing a *complete* conversion from the existing wiki to the new one?

I think this was related to a failed import attempt. It should be in better shape now (please verify)

  1. Category links don't appear to be working correctly

Interesting. After clicking on the category and actually creating the page, it no longer does this. I'm not sure how to make mediawiki do this automatically but will look into it.

  1. https://publictest1.fedoraproject.org/wiki/Releases/9/FeatureList
    1. the table listing all of the features is missing
    2. at the bottom of the page it suggests a category of "Category: AcceptedFedora9 (alpha order)" which is not correct. See item 3 above as it relates to the AcceptedFedora9 category.

Hmm, yeah thats not right. I'll take a look there.

comment:28 Changed 6 years ago by mmcgrath

Ok, number 14) was stale data and should be fixed. i've been having a problem I didn't notice before but it seems that when I sync the production copy with whats in test for manipulation the rsync is timing out. I'm looking at that now.

comment:29 in reply to: ↑ 27 Changed 6 years ago by ianweller

  1. Section headings seem overly large in comparison to the associated text

Thats css stuff which I think will be worked on.

I'm working on that.

comment:30 Changed 6 years ago by kwade

13:19 < quaid> https://publictest1.fedoraproject.org/wiki/Main_Page 13:19 < quaid> Image:Important.png 13:20 < mmcgrath> ah, thats something ianweller is working on. He hasn't uploaded that

image yet.

13:20 < quaid> in that case, it comes from the CSS 13:20 < quaid> <tableclass="message notice"> 13:20 < mmcgrath> https://publictest1.fedoraproject.org/wiki/Artwork/DesignService <-- is a

good example of images coming up.

13:20 < quaid> I think this is a CSS then 13:21 < mmcgrath> nope, this is just that the images haven't been uploaded. 13:21 < mmcgrath> <tableclass="message notice" has been converted to a template {{

Template:message/notice }}

13:21 < mmcgrath> {{ Template:message/notice | The Fedora common bugs?

page is useful for finding fixes to already known issues.

13:21 < mmcgrath> }} 13:21 < mmcgrath> for example

comment:31 in reply to: ↑ 18 ; follow-up: ↓ 32 Changed 6 years ago by ivazquez

Replying to ivazquez:

  • Initial spaces in <pre> sections get changed to have a : in front

This seems to still be an issue, in e.g. https://publictest1.fedoraproject.org/wiki/Docs/Drafts/BuildingPackagesGuide .

comment:32 in reply to: ↑ 31 ; follow-up: ↓ 33 Changed 6 years ago by mmcgrath

Replying to ivazquez:

Replying to ivazquez:

  • Initial spaces in <pre> sections get changed to have a : in front

This seems to still be an issue, in e.g. https://publictest1.fedoraproject.org/wiki/Docs/Drafts/BuildingPackagesGuide .

k, should be fixed. Please confirm

comment:33 in reply to: ↑ 32 Changed 6 years ago by ivazquez

Replying to mmcgrath:

k, should be fixed. Please confirm

The : is no longer there, but now the content is flush against the left.

comment:34 follow-up: ↓ 35 Changed 6 years ago by nim

More pages not converted properly: all the fonts SIG pages.

For example: https://publictest1.fedoraproject.org/wiki/SIGs/Fonts/Triaging/Pipeline

  1. content table not converted
  2. moinmoin tables not 100% converted
  3. tables templates not converted
  4. banners not properly converted
  5. head shortcuts not properly converted

I may have missed a few convertion errors on that page

comment:35 in reply to: ↑ 34 ; follow-up: ↓ 36 Changed 6 years ago by mmcgrath

Replying to nim:

More pages not converted properly: all the fonts SIG pages.

  1. banners not properly converted

Its likely this won't happen and will require manual intervention.

comment:36 in reply to: ↑ 35 Changed 6 years ago by ianweller

Replying to mmcgrath:

Replying to nim:

  1. banners not properly converted

Its likely this won't happen and will require manual intervention.

Yup. I'll come up with a banner template, and then we can do something like {{banner/docs}} for the docs template, or what have you. Maybe I can get this done today or tomorrow.

comment:37 Changed 6 years ago by toshio

I found that some links aren't being converted correctly... probably because Moin has several valid syntax's for specifying links. Here's the formats I've found that aren't being converted:

 [wiki:Self:Development/SteeringCommittee Fedora Steering Committee (FESCo)]
 ["Packaging/Minutes"]

Examples on: https://publictest1.fedoraproject.org/wiki/Packaging/Committee#Step_Two:_Packaging_Committee_Review

comment:38 Changed 6 years ago by ianweller

MailTo?() macros need fixed.

comment:39 Changed 6 years ago by kwade

I noticed that the URL doesn't work without the final slash. Can this be fixed before production switchover?

http://fedoraproject.org/wikinew == doesn't work http://fedoraproject.org/wikinew/ == does work

comment:40 Changed 6 years ago by mmcgrath

  • Status changed from new to closed
  • Resolution set to fixed

Closing this ticket. The /wikiold stuff does work properly now. As future issues come up please put them in a new ticket.

Note: See TracTickets for help on using tickets.