Suggested that i create a ticket here by threebean and puiterwijk on IRC
Liquid Telecom is mirroring content for other key Open Source projects including sourceforge in its East Africa Datacenter in Nairobi Kenya
The Datacenter has multiple redundant 10gbps links and the mirror service (at this time) has multiple gigabit links for internet as well as being running a fully native IPv4/IPv6 network
Liquid Telecom would like to become a Tier 1 mirror for africa and mirror the whole fedora-buffet0 from master servers and then mirror this out to other mirrors
its not obvious what the process is for us to achieve this but i am already starting to sync /pub from UK mirror service to at least reduce the leadtime once we are ready to proceed further
Please do let us know how we can best proceed and what extra information you would require for us to get this rolling?
our mirror server IP's are:
197.155.77.1 for IPv4
2c0f:fe40:8001:10::1 for IPv6
Thanks for providing a mirror. I could add you to the rsync master ACL, although looking at the output of traceroute I am not sure that would make a lot of sense. Maybe you should sync from a closer mirror. You can try to sync from rsync://dl.fedoraproject.org/fedora-buffet/ without any ACL changes to see if that performs good enough for you. Or if syncing from a closer mirror would make more sense?
Are you still in the process of syncing as the content on your mirror still looks incomplete?
I see that you have added rsync URLs to the categories you are carrying which is very good but the URLs you provided are still wrong you should provide following URLs for the Fedora Linux category:
http://mirror.liquidtelecom.com/fedora/fedora/linux/ (you could leave away one fedora) rsync://mirror.liquidtelecom.com/fedora-enchilada/linux/
The EPEL category looks correct.
we are currently syncing from rsync://dl.fedoraproject.org/fedora-buffet and yes incomplete - its a bit slow but its working
RE: http://mirror.liquidtelecom.com/fedora/fedora/linux/ (you could leave away one fedora) - this is just because we are syncing the buffet (/pub) into http://mirror.liquidtelecom.com/fedora/ hence the double fedora but happy to change this
we could sync from closer but not aware of any other mirror closer offering buffet as an option - in testing the logically closest tier1 mirrors all of them seem to offer the same slow speed "issues"
heanet seem to be my best bet - will just need to check with them on pre-bitflip sync though
Try to sync from my mirror. I added you to my fedora-buffet0 module:
rsync ftp-stud.hs-esslingen.de::fedora-buffet0/
Let me know if the performance is adequate.
Also let me know if you still want access to fedora-buffet0 on dl.fedoraproject.org.
working much better on the DE side now - we've also identified a problem on the gigabit lag which will be solved monday anyway once the 10gb optics arrive and get installed
initial syncs of EPEL and Fedora are done, its just processing through archive and secondary at the moment, once thats done, i'll configure the rsync modules for public syncing of post-bitflip and private syncing of pre-bitflip - ideally using ACL's from mirror manager but i'm also open to publically allowing pre-bitflip for epel and fedora modules
Ok we are fully synced up now
i have configured following rsync modules
{{{ fedora-epel fedora-epel0 fedora-enchilada fedora-enchilada0 fedora-buffet fedora-buffet0 }}}
the '0' repo's are pre-bitflip permissions but not yet locked down at this stage - i'd like to ultimately drive these acl from mirror manager somehow but the information i have found appears to be out of date - will be left open for now unless we start seeing major issues with it
everything else is as per the mirror manager site
do you still need me to correct the paths for the fedora linux category? or rather do they look correct now?
Secondly relating to mirror.liquidtelecom.com - it would be much appreciated if we can also have the following Peer ASN's added to the site:
{{{ AS36997 AS33770 AS37006 AS37374 AS56696 AS37146 AS30969 }}}
these specifically are Subsidiary company's of Liquid Telecom and we'd like to explicitly direct those ASN's to our mirror - i'd be happy for any other ASN's that peer with Liquid Telecom at any point to be added to the list but we didn't want to blanket add them all
Regarding ACLs. This is definitely not working and needs manual setup. I am adding mirrors on request to my modules if required. Bitflip has become, however, rather unimportant as the bits are flipped now much earlier as it used to be to allow better content propagation. I would recommend to lock the -0 modules and add mirrors to it if someone really needs access.
If you like you could open an issue at https://github.com/fedora-infra/mirrormanager2 to get working ACL support from MirrorManager.
I added you Peer ASN's to your host.
Unfortunately I also saw that you are hitting one of MirrorManager's problems in our setup. The maximum time in which a mirror has to be crawled is 3 hours. If it takes longer than three hours the scan counts as failed. After 4 failed scans the mirror is auto-disabled. Mirrors like yours (and mine) which carry everything have too much content to be crawled in three hours. I am using two different MirrorManager hosts (basically DNS aliases) which point to the same server with different URLs. That way the crawler does not have to crawl so much and finishes earlier and thus the mirror is not auto-deactivated. So this is currently not optimal but unfortunately the status quo. Please list rsync modules for all Categories carried to give the crawler a chance to finish in the three hours.
You can use whatever URLs you want, if they work. You can always compare the URLs to other working mirrors. You do not (maybe even should not) list the -0 modules under URLs. That is not required for the clients and also may make troubles with the crawler.
ok thats cleaned up, i split into 3 hosts under Liquid Telecom
epel.mirror.liquidtelecom.com - for epel
fedora.mirror.liquidtelecom.com - for fedora linux and archive
mirror.liquidtelecom.com - for alt and fedora-secondary
the peer ASN's will need copying to the 2 new hosts
remaining RSYNC url's setup for all and the 0 url's now removed from mirror manager
I've spent some time monitoring crawl logs and run times and now down to 2 hosts - mirror.liquid - alt, secondary and epel and fedora.mirror.liquid for fedora and archive - looks like both complete routinely within 1-2 hours especially now that RSYNC fixed on all modules (had a missing group issue for a while there)
just needs the Peer ASN copied from mirror.liquid to fedora.mirror.liquid and everything is good - as well as anything fedora team needs to do to confirm Tier 1 criteria matched and confirm tier 1 status
I added the Peer ASNs to your other host. I added your mirror under
https://fedoraproject.org/wiki/Infrastructure/Mirroring/Tiering
as it is a wiki you can change it to your needs if necessary. As everything seems to be working now I am closing this ticket. Thanks for providing a tier 1 mirror in Africa.
{{{ $ curl -s "http://mirrors.fedoraproject.org/mirrorlist?repo=fedora-23&arch=x86_64&country=ke"
http://fedora.mirror.liquidtelecom.com/fedora/linux/releases/23/Everything/x86_64/os/ }}}
Login to comment on this ticket.