wget is a free and very useful command-line tool that can be used for downloading files from the web using HTTP, HTTPS, FTP and FTPS protocols.
It is a non-interactive command-line utility that means it can work in the background and can be called from scripts and cron jobs.
It can be runs on most Unix-based operating systems as well as macOS and Windows.
It can handle complex download situations including, resuming downloads, downloading multiple files, limit the bandwidth, recursive downloads, download in the background and much more.
In this tutorial, we will show you how to use wget command with several hands-on examples.
Install Wget
By default, wget comes pre-installed in most Unix-based operating systems. If not installed you can install it using the following method.
For Ubuntu/Debian operating systems, run the following command:
apt-get install wget -y
For RHEL/CentOS/Fedora operating systems, run the following command:
yum install wget -y
Once installed, you can see the installed version of wget with the following command:
wget -V
You should see the following screen:
Once you are finished, you can proceed to the next step.
Basic Syntax
The basic syntax of the wget command is shown below:
wget [OPTION] [URL]
A brief explanation of each option is shown below:
- -h : Used to print all options available with wget command.
- -o : Used to redirect all the messages generated by the system to the logfile.
- -b : Used to send the downloading process to the background.
- -a : Used to append the messages to the current log file.
- -i : Used to read the URL from the file.
- -c : Used to resume a partially downloaded file.
Download a Single File with Wget
You can use wget command without any option to download a single file.
For example, run the following command to download latest.tar.gz file:
wget https://wordpress.org/latest.tar.gz
You should get the following output:
--2020-04-04 17:47:26-- https://wordpress.org/latest.tar.gz
Resolving wordpress.org (wordpress.org)... 198.143.164.252
Connecting to wordpress.org (wordpress.org)|198.143.164.252|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 12230487 (12M) [application/octet-stream]
Saving to: ‘latest.tar.gz’
100%[=====================================================================================================>] 1,22,30,487 219KB/s in 50s
2020-04-04 17:48:18 (238 KB/s) – ‘latest.tar.gz’ saved [12230487/12230487]
The above output contains the following information:
- Name of the file.
- Progress bar with the percentage.
- The number of the file that has been downloaded.
- The download speed.
- The estimated time to complete the download.
You can use -q option with wget command to turn off the output.
Download a File in Background with Wget
If want to download a large file then you can use -b option to download the file in the background.
wget -b https://wordpress.org/latest.tar.gz
You should see the following output:
Continuing in background, pid 7614.
Output will be written to ‘wget-log’.
The downloading progress output will be written in the wget-log in your current working directory.
You can see it with the following command:
tail -f wget-log
You should see the following screen:
Resume Uncompleted Download with Wget
If you are downloading a large file and it stops download due to internet problem then you can resume the download from where it got disconnected using the -c option.
wget -c https://wordpress.org/latest.tar.gz
You should see the following screen:
Limit Download Speed with Wget
By default, wget command will download the specified file using the full bandwidth this will slow down the internet of other users. In this case, you can limit the download speed using –limit-rate option:
For example, download the file latest.tar.gz at the speed of 50k using the following command:
wget --limit-rate=50k https://wordpress.org/latest.tar.gz
You should see the following output:
wget --limit-rate=50k https://wordpress.org/latest.tar.gz
--2020-04-04 18:13:11-- https://wordpress.org/latest.tar.gz
Resolving wordpress.org (wordpress.org)... 198.143.164.252
Connecting to wordpress.org (wordpress.org)|198.143.164.252|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 12230487 (12M) [application/octet-stream]
Saving to: ‘latest.tar.gz’
26% [==========================> ] 32,92,770 50.0KB/s eta 2m 56s
Download Multiple Files with Wget
If you want to download multiple files at the same time then you can achieve this with wget command. You can need to create a text file and place all URLs that you want to download.
Let’s create a text file with the following command:
nano file1.txt
Add all URLs of the file that you want to download:
https://cache.ruby-lang.org/pub/ruby/2.7/ruby-2.7.0-preview1.tar.bz2
https://wordpress.org/latest.tar.gz
http://assets.withknown.com/releases/known-0.9.2.zip
Save and close the file when you are finished. Then, download all files stored in the file1.txt using the -i option as shown below:
wget -i file1.txt
You should see the following screen:
Download a File and Save it with Different Name Using Wget
You can use -O option with wget command to save the downloaded file under a different name.
For example, download the file latest.tar.gz and save it under name wordpress.tar.gz using the following command:
wget https://wordpress.org/latest.tar.gz -O wordpress.tar.gz
You should see the following output:
--2020-04-05 09:28:38-- https://wordpress.org/latest.tar.gz
Resolving wordpress.org (wordpress.org)... 198.143.164.252
Connecting to wordpress.org (wordpress.org)|198.143.164.252|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 12230487 (12M) [application/octet-stream]
Saving to: ‘wordpress.tar.gz’
100%[=====================================================================================================>] 1,22,30,487 59.7KB/s in 40s
2020-04-05 09:29:20 (299 KB/s) – ‘wordpress.tar.gz’ saved [12230487/12230487]
Download Full Website with Wget
You can download or mirror a complete local copy of any website using the wget command easily.
For example, to download a website www.website.com for local browsing, run the following command:
wget --mirror -p --convert-links http://www.website.com
Where:
- –p :-mirror : This option will enable mirroring options.
- -p : This option will download all necessary files required to properly display a given HTML page.
- –convert-links : This option will convert the links in document for local viewing.
The above command will download or mirror a complete website into your current working directory.
Download File from Password Protected Website
If you want to download a file from the password-protected FTP website then you can achieve this by specifying username and password with wget command:
wget --ftp-user= --ftp-password= FTP-URL
Redirect Wget Log to a Specific File
If you want to save wget log to a specific file then you can redirect it using -o option.
wget -o wget.log https://wordpress.org/latest.tar.gz
The above command will create a wget.log file in the user’s current directory.
Download Files in Specified Directory with Wget
You can also save the downloaded file to the specified directory using -P option:
wget -P /mnt/ https://wordpress.org/latest.tar.gz
You should see the following output:
--2020-04-05 09:42:20-- https://wordpress.org/latest.tar.gz
Resolving wordpress.org (wordpress.org)... 198.143.164.252
Connecting to wordpress.org (wordpress.org)|198.143.164.252|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 12230487 (12M) [application/octet-stream]
Saving to: ‘/mnt/latest.tar.gz’
100%[=====================================================================================================>] 1,22,30,487 405KB/s in 21s
2020-04-05 09:42:42 (578 KB/s) – ‘/mnt/latest.tar.gz’ saved [12230487/12230487]
Conclusion
In the above tutorial, you learned how to use wget command with different options for various download scenarios. I hope this will help you to perform your day-to-day tasks.