Skip to main content

Hacks: Replace yum's downloader with Axel

/!\ This is an ugly hack, use it at your own risk, 
I am not responsible for any breakage

P/S, your sysadmin might not gonna like this


Disclaimer aside, I really love Python.

So, I'm behind such a bad network which keep timing out, and downloading stuff using yum in this network is a real PITA. And so, I had enough of it and decided to hack around.

Axel is a CLI download accelerator thats lightweight and pretty fast. I've been using it quite frequently to get a bit more speed for my downloads. I think, why not just replace this urllib2 grabber with axel, so, I did. (I am such an evil guy)

After a few hours looking through yum code to find what to modify, I ended up with this



# file: /usr/lib/python2.5/site-packages/urlgrabber/customgrabber.py

import grabber
import os

class AxelGrabber(grabber.URLGrabber):
def urlgrab(self, url, filename=None, **kwargs):
"""grab the file at and make a local copy at
If filename is none, the basename of the url is used.
urlgrab returns the filename of the local file, which may be
different from the passed-in filename if copy_local == 0.
"""

opts = self.opts.derive(**kwargs)
(url,parts) = opts.urlparser.parse(url, opts)
(scheme, host, path, parm, query, frag) = parts


def retryfunc(opts, url, filename):
if os.environ.has_key('http_proxy'):
os.environ['HTTP_PROXY'] = os.environ['http_proxy']
os.system('/usr/bin/axel -a -o %s %s' % (filename,url))
return filename

return self._retry(opts, retryfunc, url, filename)


Make sure you put the file above at its path unless you are sure what you are doing. To make it usable with yum, some modification is needed in one of yum's source file.

Edit
/usr/lib/python2.5/site-packages/yum/yumRepo.py

change
from urlgrabber.grabber import URLGrabber

to
from urlgrabber.customgrabber import AxelGrabber as URLGrabber


and you are good to go. Do it at YOUR own risk remember.
4 comments

Popular posts from this blog

Consolidated community site infrastructure on Plone

In Inigo, we believe in helping out local FOSS communities and help them grow. We help out in community events where we can, present FOSS talks, and provide some platforms for local communities to grow. One of such platform is our consolidated community site infrastructure on Plone.

The system/infra and its components was originally developed for the Fedora Malaysia website, while keeping in mind to keep it generic enough so that other communities could use the same components for their own community sites. The infra is already at a usable state, and we can add new sites easily with just a few clicks.

Features in this consolidated infra are:
Document/Content management (Plone built-in)Calendar system (powered by solgema.fullcalendar addon)Conference/BarCamp system (powered by collective.conference addon, which was developed for FUDCon Kuala Lumpur 2012)Blog (powered by Products.Scrawl)Simple yet powerful theming engine (powered by plone.app.theming/ Diazo) - Check out Diazo, you'l…

Adding simple popup to Plone frontpage

Here is a little guide for those who want to add a simple popup to the Plone frontpage for some purpose (eg: announcements, advertisements, etc).

Create a basic html file containing the content you want to appear in the popup. Upload it into $PLONE_SITE/portal_skins/custom (as Page Template) and for the sake of this example, name it popup.html

Afterward, create a Javascript file with your Pop-Up loader script. For example , this script:


function popup(mylink, windowname)
{
if (! window.focus)return true;
var href;
if (typeof(mylink) == 'string')
href=mylink;
else
href=mylink.href;
window.open(href, windowname, 'width=220,height=400,scrollbars=no');
return false;
};

popup('popup.html', 'My Popup');


Also upload this file into $PLONE_SITE/portal_skins/custom (as Page Template too). For this example, name it as popup.js

Afterward, in $PLONE_SITE/portal_javascripts , add popup.js as a new script into portal_javascripts…

HOWTO: Mirroring Yum repositories using Yum-Utils

As promised before in one of my previous post, a Howto on how to mirror and manage yum repositories using some of the utilities in yum-utils.

The first step is, well, of course, is to get yum-utils from fedora repository
yum install yum-utils

Reposync
Reposync is a utility for mirroring and synchronizing local copy of a yum/rpmmetadata repository.

This utility is very useful if you wanted to make a yum repository mirror. Before this, I used "wget -R -np -N" but this method is a little bit tedious and it doesnt work with repos that didn't use directory listing. Plus, it also download together additional site stuff that I don't need/want and it doesn't verify checksum of the downloaded packages.

Mirroring a repo using this utility is easy, just execute this command
reposync -r <repoid> -a <arch> -n
and the repo will be mirrored in a folder with the same name of the repoid in the directory you executed the command. Eg: you executed the command in /mnt/storage/mi…