yum-plugin-timemirrorbandwidth

-
Something I came up in during the small FAD we had in .MY last week.

Yum's fastestmirror plugin is kindof broken by design, because the plugin's definition of 'fastest' means shortest handshake time, rather than shortest download time (highest bandwidth).

If a particular country had nearby mirrors which have a really low bandwidth, fastestmirror actually made the download speed slower for users who are using it. (in my case, the nearby mirrors are extremely slow, its faster to simply force yum to grab from Taiwan mirrors, and I believe lots of local users here are also affected by this)

This plugin is a hack which complements fastestmirror by timing the metadata download time of repositories, sort them and set fastestmirror.conf to only include the top 3 mirrors with shortest metadata download time for each repository.

Source: timemirrorbandwidth.py

To install, simply copy the file into /usr/lib/yum-plugins/ and create a /etc/yum/pluginconf.d/timemirrorbandwidth.conf with this contents:

[main]
enabled=1

(I should put this in RPM i guess)

To use, simply execute:

yum time-mirrors --disableplugin=fastestmirror

the plugin will start scoring the available mirrors and add the fastest 3 mirrors for each repository into fastestmirror.conf's include_only list. I wont make this plugin to run the timing each time yum is executed, because that will mean extreme slowdown.

Feel free to improve it .. (probably i should move this into github or something).. Have fun!

Comments

Anonymous said…
you, sir, rock most mightily. there's a small typo in the post, though - you have the config file the extension .py instead of .conf . i've wanted this for ages though!
2:40 AM
Ben C said…
This is a great plugin. If you do make it into an RPM, I'd recommend putting something in /etc/cron.weekly so that the list stays fresh.

Also, what adamwill said above.
2:55 AM
Unknown said…
@adamwill

Thanks!. fixed :D

@ben

nice idea!. thanks :D
10:43 AM
luke said…
Nice work.

I have been wanting to add support for more sophisticated algorithms to fastestmirror for a long time, but have never gotten around to it.

fastestmirror's current method, which involves timing how long it takes to open a socket to the mirror, is very fast, and generally yields reasonable results most of the time. However, it's extremely dumb, and isn't optimal for situations like yours with low-latency low-bandwidth mirrors.

If you wanted to add bandwidth tests to fastestmirror as a configurable option, I would be glad to merge it upstream.
1:44 PM
skvidal said…
Nice plugin. Suggestion - maybe think about taking your plugin and making it into a patch to yum itself. So users get this data automatically.


pop by #yum if you get a chance
11:59 PM
Anonymous said…
luke: fastestmirror always seems to wind up giving me such a mirror, btw. I think it's most useless when you have a pretty fast, say 5mbit+, link: latency is good for weeding out *really* slow mirrors, but it does almost nothing when it comes to a set of mirrors that can provide between, say, 0.5mbit and 10mbit.
2:53 AM
Unknown said…
Hi, it's a great plugin. In my case, mirrorlist return only 2 http mirrors and plugins wait forever. Also, the plugin only test http mirrors, ftp mirror are excluded.
11:18 PM
Post a Comment

Popular posts from this blog

Tee'ing Python subprocess.Popen output

-
A little hack for python coders out there who wanted to have a functionality similar to the unix's tee command for redirecting output to multiple places.


import sys
from subprocess import Popen,PIPE
p = Popen(['put','command','and','arguments','here'],stdout=PIPE)

while True:
o = p.stdout.readline()
if o == '' and p.poll() != None: break
# the 'o' variable stores a line from the command's stdout
# do anything u wish with the 'o' variable here
# this loop will break once theres a blank output
# from stdout and the subprocess have ended
Read more

Announcing PlatoCDP, a Plone distribution for enterprises.

-
Image
Announcing the first release of PlatoCDP Plone distribution. A Plone distribution with the goal of attempting to build a more supportable, enterprise-ready, document management and content management  solution for government and enterprises. The ideal aim is to develop improvements on the distribution so that it will be a viable alternative to Sharepoint.
So what is it exactly?
An enterprise Plone distribution - that is, an integrated product which includes Plone, selected Plone add-ons, 3rd party softwares and integration/automation improvements to deliver an enterprise ready product for customers.Is it a rebrand of Plone? - It is an enterprise distribution product. PlatoCDP to Plone is like RHEL to Fedora. It is not a mere rebrand. PlatoCDP includes core Plone, additional components from the collective and its own improvements. We engage on packaging, certification and development activities to make the whole stack supportable for enterprises. These components of the dist…
Read more

Consolidated community site infrastructure on Plone

-
In Inigo, we believe in helping out local FOSS communities and help them grow. We help out in community events where we can, present FOSS talks, and provide some platforms for local communities to grow. One of such platform is our consolidated community site infrastructure on Plone.

The system/infra and its components was originally developed for the Fedora Malaysia website, while keeping in mind to keep it generic enough so that other communities could use the same components for their own community sites. The infra is already at a usable state, and we can add new sites easily with just a few clicks.

Features in this consolidated infra are:
Document/Content management (Plone built-in)Calendar system (powered by solgema.fullcalendar addon)Conference/BarCamp system (powered by collective.conference addon, which was developed for FUDCon Kuala Lumpur 2012)Blog (powered by Products.Scrawl)Simple yet powerful theming engine (powered by plone.app.theming/ Diazo) - Check out Diazo, you'l…
Read more