Using terminal based download managers

While porting form Windows to Linux, you are sure going to miss those fancy GUI download managers; Free Download Manager, Flashget, DAP to name some. There are GUI download managers available for Linux as well-Kget, Gget, Multiget, but LInux is better known for console based download managers like wget and cURL. While a simple download can be performed by issuing commands followed by the URI of the resource to download, complex function need some look at the their man pages :) Here, two popular download managers- wget and cURL are discussed:

wget

wget was written under the GNU project. So, you will find this tool on almost very Linux distribution. It is very simple to use, can do lot of featured retrieval work like website download, cookie and referer based download, et cetra

Straight forward download can be performed by the following command:

wget "http://chuangtzu.acc.umu.se/debian-cd/4.0_r3/i386/iso-cd/debian-40r3-i386-CD-1.iso"

This will get you the file debian-40r3-i386-CD-1.iso

If a file fails to download, then wget will fetch the remaining bytes of the file without having to start downloading the file all over again.

This is done by adding a -c option, like

wget -c "http://chuangtzu.acc.umu.se/debian-cd/4.0_r3/i386/iso-cd/debian-40r3-i386-CD-1.iso"

Similarly, wget can forge referer, Useragent, cookie et cetra

The following command will allow to impersonate as Googlebot while fetching the file debian-40r3-i386-CD-1.iso with referer set as indicated.

wget "http://chuangtzu.acc.umu.se/debian-cd/4.0_r3/i386/iso-cd/debian-40r3-i386-CD-1.iso" --user-agent="Googlebot/2.1 (+http://www.googlebot.com/bot.html)" --referer="www.google.com/search?q=download%20debian"

Similarly recursive website download can be done as

wget -r -l 0 http://codinghorror.com/

There is more to wget then that described here. Refer to the Wget article on Wikipedia for more.

cURL

cURL is another feature rich downloading program. cURL can work with a lot of protocol including HTTP, HTTPS, FTP, FTPS, SCP, TFTP, TELNET etc. Examples and comparison of cURL with other download programs can be found at its website: http://curl.haxx.se/

These download programs have no support for multiple threaded download using the Range HTTP header (but, there might be some way of doing this). Most of the preloaded download managers like KGet also do not have this feature. So you might as well want to try another GUI based download manager called Multiget for this feature.

For a text-based multi-threaded download manager have a look at ProZilla

ProZilla

ProZilla is a program that makes multiple connections and downloads a file in multiple parts simultaneously thus enhancing the download speed and downloading the file faster than a download with a single connection. For more information and download visit http://prozilla.genesys.ro/

GUI Download Managers

Multiget

Multiget is a nice feature rich GUI download manager like the DAP/IDM for windows. It has a download basket to drag and drop a link, supports multipart downloading, resume broken download and HTTP/FTP/HTTPS/SOCKS proxies.

Multiget isn't available in the ubuntu repository but you can get it from www.getdeb.net http://www.getdeb.net/app/MultiGet

Orbit Download Manager

If you are unsatisfied by Linux GUI Download Manager, you can use windows download manager in Linux through Wine. Not all Windows Download Manager will run though, but Orbit Download Manager runs smoothly in Wine.

The nice feature about Orbit Download Manager in linux is that it integrates well with FlashGot for a smooth download operation (means you dont have to copy and paste download URL from firefox to ODM).

DownThemAll

This is a download manager built into Firefox so that you do not have to install another program just to download a file. The new version has very good download resume support (which earlier versions restarted a download after session expired). DownThemAll website at http://www.'''downthemall'''.net and the firefox extension can be downloaded at https://addons.mozilla.org/en-US/firefox/addon/201 .

DownloadManagers (last edited 2008-07-19 10:42:27 by SujinJoshi)