Curl — A Command Line Browser

5 min readSep 16, 2020

Curl is a simple command line utility that is used for transferring data to or from the server and for making different types of requests.

It supports cookies, HTTP, HTTPS, LDAP, SMB and various other protocols which are commonly used. Due to various functionalities provided by this tool, it is also referred as a command line browser.

Curl is a multi-platform and it works on linux, windows as well as macOS, but here in the following section, we will be using it for a linux platform.

Since we already have curl installed on our machine, let us see which version of curl we are running.

Curl provides a vast majority of options to work with and sometimes can be daunting to some users. To work around with that, we can use curl –help command.

The options that we have are enormously huge but we will cover some important functionalities that it has to offer.

With the basic steps in mind, let us see how to perform a query of a url.

**Command: curl** **https://stackoverflow.com**

We can see the content of the entire web page returned to us or in technical terms, the source code of the website can be seen as above.

Return only the HTTP headers

We might not always want to have a look at the whole page being loaded but only the http headers associated with it. A simple curl –I command helps us to view only the http headers associated with the website.

**Command: curl –I** **https://stackoverflow.com**

Another thing to keep in mind is that if we make any HTTP request to the website, the default method which curl uses is GET.

Curl Downloading Files

Another important functionality that we can take advantage of using curl is to download items from the web. To those people who are familiar with wget in linux, it works similar to that for retrieving the data hosted on the web.

Command: curl –O http://sersc.org/journals/index.php/IJAST/artical/download/25847/13907

The –O command here is used to write the output to a file named as the remote file. Don’t get confused with –o option which writes to a file instead of stdout.

Following Redirection

In many cases, a website will redirect us to another url so we have to know how to specify this to curl. Let us see a basic example of how this works.

We can see that gmail.com gave us a 301 Moved redirection which is a common thing visible on modern websites. Now if any website contains these type of redirection then we can use –L option to fetch the data following the redirection of the web page.

We can see the contents of the website provided by curl after following the redirection.

A key thing to note here is that we are only retrieving the contents of the page and not the response headers associated with it. A response header contains a pool of resources that will help us gather information of the website that we are looking for.

Curl OPTIONS Request

So far till now, we have looked at how to send HTTP request using the GET method but what if we want to send the request using some other methods? Before that, it is important to find out which request method does the server supports so that we can query for the information accordingly.

**Command: curl –I –X OPTIONS** **https://www.codeappstore.com**

We can see that the allowed methods here are GET, POST, OPTIONS and HEAD.

Let us see how to use HEAD method to request the data which gives only the header information in its response.

We can see that the server here is Apache which basically gives us an idea of the application server which generated the request.

**Command: curl –I –X HEAD** **https://www.codeappstore.com**

These obtained information can really help us during our web assessments.

Most of the modern day websites use HTTPS which means that it is sending data securely using the SSL/TLS. Now in a case where the remote server has self signed certificate, we can skip the SSL check using the –k option.

Command: curl –k https://www.linkedin.com

Using Proxies

Curl also supports various types of proxies which includes HTTP and HTTPS. In order to send data through the proxy, we can use the –x option.

Command: curl -x 127.0.0.1:8080 http://linux.com

I have used the burp proxy to intercept the request. We can simply use the repeater tab to send the request and also see the response sent by the server.

We get the same required output using the curl by sending the traffic through the proxy.

Conclusion

So far, we saw the basic and widely used options provided by the curl utility. Apart from these, there are many more commands which we can leverage for information gathering. For the detailed information of this utility, we can use man curl command.