Download files on Linux using curl

Introduction

curl is a command-line utility for transferring data from or to a server designed to work without user interaction. With curl, you can download or upload data using one of the supported protocols including HTTP, HTTPS, SCP, SFTP, and FTP. curl provides a number of options allowing you to resume transfers, limit the bandwidth, proxy support, user authentication, and much more.

Installing Curl

The wget package is available on most Linux distributions. To check if the wget package is present on your system by running the following command:

curl --version

If the wget package is not available on your system, you can easily install it as follows:

Ubuntu/Debian:

apt-get install curl

Centos 7:

yum install wget -y

yum install curl -y

Centos 8:

dnf install curl -y

Curl command example

• The curl command will get the content of the URL and display the following output:

curl http://www.centos.org

• Save the Output to a File

curl -o test.html https://green.cloud/ve-blogd/index.md

Through the results of the above command, we see that the index.md page will be saved in a file named test.html.

Uppercase -O saves the file with its original filename:

curl -O https://cdn.jsdelivr.net/npm/vue/dist/vue.js

• Download Multiple files

You can resume a download by using the -C – option. This is useful if your connection drops during the download of a large file, and instead of starting the download from scratch, you can continue the previous one.

curl -O http://releases.ubuntu.com/18.04/ubuntu-18.04-live-server-amd64.iso

and suddenly your connection drops you can resume the download with:

curl -C - -O http://releases.ubuntu.com/18.04/ubuntu-18.04-live-server-amd64.iso

• Get the HTTP Headers of a URL

HTTP headers are colon-separated key-value pairs containing information such as user agent, content type, and encoding. Headers are passed between the client and the server with the request or the response.

curl -I --http2 https://www.ubuntu.com/

• Download files from FTP servers with or without authentication

If downloading the file from an FTP server at ftp://ftpserver, the following command will download the file yourfile.zip to your current working directory.

curl -u username:password -O ftp://ftpserver/yourfile.zip

The username:password option can be omitted if the FTP server allows anonymous login.

• Upload files to an FTP server

curl -u username:password -T ocalfile.tar.gz ftp://ftpserver/

• Using Proxies

curl supports different types of proxies, including HTTP, HTTPS and SOCKS. To transfer data through a proxy server, use the -x (–proxy) option, followed by the proxy URL.

curl -x 192.168.44.1:8888 http://linux.com/

If the proxy server requires authentication, use the -U (–proxy-user) option followed by the user name and password separated by a colon (user:password):

curl -U username:password -x 192.168.44.1:8888 http://linux.com/

Conclusion

curl lets you quickly download files from a remote system. curl supports many different protocols and can also make more complex web requests, including interacting with remote APIs to send and receive data.