Curl is one of the most ubiquitous tools on Linux. It comes pre-installed with every flavor, and we often use it in demos or for downloading packages from the Internet. By design, though, the tool doesn’t automatically follow redirects. There are reasons for this behavior, having to do with security. Phishing attacks these days rely heavily on tricking the user into clicking on legit-sounding URLs, and it’s even possible that a person within the organization could exploit a real page to redirect someone to a malicious one. But if you’re sure you want curl to follow redirects, then it’s a simple matter of including the appropriate flag.
How curl Handles Redirects
Let’s say we have a website that redirects to another one. I’ll use the following URL:
http://httpbin.org/redirect-to?url=https://httpbin.org/get
This URL forces a redirect from the “http” version of the website to the “https” one. This is a perfect candidate to test curl redirects. Now let’s see how curl handles this URL:
curl -v http://httpbin.org/redirect-to?url=https://httpbin.org/get 2>&1 | grep Location
I’m using the “-v” flag with curl to inform it that I want a verbose output. I want it to tell me as much as possible about what’s happening. In this mode, curl sends the content to the regular stdout channel, but sends the other verbose output, like the header information, to stderr. I’d written an earlier article on how to redirect stderr to stdout, so check out that article for more details.
Since I don’t want to wade through a lot of details, I use grep to narrow down what I’m looking for. Specifically, I want to know what curl has to say about the <Location> tag. When we run this command, here’s what I see:
As you can see, curl correctly identifies that the content is located on another URL – this time starting with “https” instead of “http”. The problem is that curl doesn’t automatically follow this redirect, and so it informs us that the resource is located elsewhere and stops. It doesn’t fetch the resource from the new location.
Getting curl to Follow Redirects
To correct the above behaviour, we include the “-L” flag along with the verbose tag to make curl follow the redirect and show us the output. The command to do that is:
curl -Lv http://httpbin.org/redirect-to?url=https://httpbin.org/get
Now when we run this command, we get:
As you can see, curl has found the 302 redirect as before. But this time, instead of leaving it there, it goes ahead and issues a new request to the redirected URL like this:
Here you see the message “Issue another request to this URL”. Finally, we can see that curl has successfully managed to obtain the URL as evidenced by the “200” response code shown here:
So this is a complete examination of how curl access redirected resources and then follows through with the right flags.
Curl Follows All Redirects
Through the above mechanism, curl doesn’t just follow a redirect once. It recursively follows redirects one after another, until it receives a non-redirect status code. This could also culminate in an error if the final page isn’t found.
By default, curl follows redirects up to 50 times. This prevents the tool from continuing endlessly on a redirect loop. After that, it’ll send an error saying that too many redirects occurred. You can also change the number of times it tries to follow redirects using the “–max-redirs” option as shown here:
curl -L --max-redirs 10 https://example.com/infinite-redirect
The above command will restrict the redirect attempts to 10.
Curl also follows redirects regardless of the kind of redirect code. That means it follows 301 (permanent moves), 302 (temporary redirects), and even 308 redirects – a more recent error code that indicates that the HTTP method must remain intact.
How to Make Curl Follow Redirects Automatically
As of now, there’s no global setting or configuration file that allows you to automatically change the behavior of the curl command so that it redirects automatically. You need to always include the “-L” or “–location” parameter to enforce that behavior.
However, what you can do is to create a bash alias that automatically converts a regular curl command into one that follows redirects without further prompting. You can use this technique only for yourself, because otherwise someone can modify curl’s behavior without you knowing about it, and that’s a huge security risk, as we’ll see below.
To create an alias, simply add the following line to your ~/.bashrc or ~/.zshrc file:
alias curl='curl -L'
After including the command, don’t forget to reload your shell with:
source ~/.bashrc # or ~/.zshrc
To be clear, curl does have a configuration file called ~/.curlrc to change its default behavior, but this doesn’t include automatic following for redirects due to security concerns. So what are the security concerns, anyway?
Useful for Diagnosing Redirect Chains
Sometimes, it can be very frustrating when your browser enters a URL redirect loop, and you don’t know why. You need to check whether it’s a performance or a security issue. In addition, you want to make sure that it ends up in the desired destination.
For this kind of troubleshooting, using curl with the “-L” parameter is very useful. The command:
curl -Lv http://example.com
Will show you all the details and all the headers for each request so you can track what’s going on. Since curl, by default, redirects a maximum of 50 times, you can easily catch infinite redirects when the tool informs you that it’s reached the maximum limit. You can also follow it between protocols and status codes to get even more information about what’s happening.
Using the “-c” and “-b” flags, you can even track cookie usage between URLs with a command like this:
curl -Lv -c cookies.txt -b cookies.txt http://example.com
In short, curl is a one-shot tool for redirects, and you can save yourself a lot of headaches once you learn how to troubleshoot.
Security Concerns for Curl Following Redirects
Curl redirects are rife with possibilities for abuse. The main reason why it’s insecure is that you can be unwittingly redirected to another URL without your knowledge. It’s often harmless, but it can also come with huge consequences.
Revealing Sensitive Data
Many requests to URLs aren’t simple HTML requests. They can contain POST data that’s sensitive. You can even include password information in these requests using curl’s inbuilt “-d” command like this:
curl -L -d “password=secret” https://trusted.example.com/login
If the above URL is redirected to another one, then curl will re-issue the request to the new URL as shown above, and send it the same parameters it sent to the original URL. As you can see, this will result in your password data being exposed to a potentially malicious website.
Cross-Protocol Redirects
Curl can redirect not just to another HTTP or HTTPS page, but to another protocol entirely. For example, it could redirect to an FTP protocol, and that means you could suddenly find yourself downloading content automatically that you didn’t want, and which might even be unknown to you if the download is so small it happens almost immediately.
Conclusion
By default, curl doesn’t follow redirects. But you can use the “-L” or the “–location” flag to ensure that the tool follows redirects. This can help you in troubleshooting redirect loops in your application.
I’m a NameHero team member, and an expert on WordPress and web hosting. I’ve been in this industry since 2008. I’ve also developed apps on Android and have written extensive tutorials on managing Linux servers. You can contact me on my website WP-Tweaks.com!
Leave a Reply