shown - curl show response body




How to display request headers with command line curl (6)

I believe the command line switch you are looking for to pass to curl is -I.

Example usage:

$ curl -I http://heatmiser.counterhack.com/zone-5-15614E3A-CEA7-4A28-A85A-D688CC418287  
HTTP/1.1 301 Moved Permanently
Date: Sat, 29 Dec 2012 15:22:05 GMT
Server: Apache
Location: http://heatmiser.counterhack.com/zone-5-15614E3A-CEA7-4A28-A85A-D688CC418287/
Content-Type: text/html; charset=iso-8859-1

Additionally, if you encounter a response HTTP status code of 301, you might like to also pass a -L argument switch to tell curl to follow URL redirects, and, in this case, print the headers of all pages (including the URL redirects), illustrated below:

$ curl -I -L http://heatmiser.counterhack.com/zone-5-15614E3A-CEA7-4A28-A85A-D688CC418287
HTTP/1.1 301 Moved Permanently
Date: Sat, 29 Dec 2012 15:22:13 GMT
Server: Apache
Location: http://heatmiser.counterhack.com/zone-5-15614E3A-CEA7-4A28-A85A-D688CC418287/
Content-Type: text/html; charset=iso-8859-1

HTTP/1.1 302 Found
Date: Sat, 29 Dec 2012 15:22:13 GMT
Server: Apache
Set-Cookie: UID=b8c37e33defde51cf91e1e03e51657da
Location: noaccess.php
Content-Type: text/html

HTTP/1.1 200 OK
Date: Sat, 29 Dec 2012 15:22:13 GMT
Server: Apache
Content-Type: text/html

Command line curl can display response header by using -D option, but I want to see what request header it is sending. How can I do that?


I had to overcome this problem myself, when debugging web applications. -v is great, but a little too verbose for my tastes. This is the (bash-only) solution I came up with:

curl -v http://example.com/ 2> >(sed '/^*/d')

This works because the output from -v is sent to stderr, not stdout. By redirecting this to a subshell, we can sed it to remove lines that start with *. Since the real output does not pass through the subshell, it is not affected. Using a subshell is a little heavy-handed, but it's the easiest way to redirect stderr to another command. (As I noted, I'm only using this for testing, so it works fine for me.)


The verbose option is handy, but if you want to see everything that curl does (including the HTTP body that is transmitted, and not just the headers), I suggest using one of the below options:

  • --trace-ascii - # stdout
  • --trace-ascii output_file.txt # file

You get a nice header output with the following command:

 curl -L -v -s -o /dev/null google.de
  • -L, --location follow redirects
  • -v, --verbose more output, indicates the direction
  • -s, --silent don't show a progress bar
  • -o, --output /dev/null don't show received body

Or the shorter version:

 curl -Lvso /dev/null google.de

Results in:

* Rebuilt URL to: google.de/
*   Trying 2a00:1450:4008:802::2003...
* Connected to google.de (2a00:1450:4008:802::2003) port 80 (#0)
> GET / HTTP/1.1
> Host: google.de
> User-Agent: curl/7.43.0
> Accept: */*
>
< HTTP/1.1 301 Moved Permanently
< Location: http://www.google.de/
< Content-Type: text/html; charset=UTF-8
< Date: Fri, 12 Aug 2016 15:45:36 GMT
< Expires: Sun, 11 Sep 2016 15:45:36 GMT
< Cache-Control: public, max-age=2592000
< Server: gws
< Content-Length: 218
< X-XSS-Protection: 1; mode=block
< X-Frame-Options: SAMEORIGIN
<
* Ignoring the response-body
{ [218 bytes data]
* Connection #0 to host google.de left intact
* Issue another request to this URL: 'http://www.google.de/'
*   Trying 2a00:1450:4008:800::2003...
* Connected to www.google.de (2a00:1450:4008:800::2003) port 80 (#1)
> GET / HTTP/1.1
> Host: www.google.de
> User-Agent: curl/7.43.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Date: Fri, 12 Aug 2016 15:45:36 GMT
< Expires: -1
< Cache-Control: private, max-age=0
< Content-Type: text/html; charset=ISO-8859-1
< P3P: CP="This is not a P3P policy! See https://www.google.com/support/accounts/answer/151657?hl=en for more info."
< Server: gws
< X-XSS-Protection: 1; mode=block
< X-Frame-Options: SAMEORIGIN
< Set-Cookie: NID=84=Z0WT_INFoDbf_0FIe_uHqzL9mf3DMSQs0mHyTEDAQOGY2sOrQaKVgN2domEw8frXvo4I3x3QVLqCH340HME3t1-6gNu8R-ArecuaneSURXNxSXYMhW2kBIE8Duty-_w7; expires=Sat, 11-Feb-2017 15:45:36 GMT; path=/; domain=.google.de; HttpOnly
< Accept-Ranges: none
< Vary: Accept-Encoding
< Transfer-Encoding: chunked
<
{ [11080 bytes data]
* Connection #1 to host www.google.de left intact

As you can see curl outputs both the outgoing and the incoming headers and skips the bodydata althought telling you how big the body is.

Additionally for every line the direction is indicated so that it is easy to read. I found it particular useful to trace down long chains of redirects.


in php set curl options

CURLINFO_HEADER_OUT => true,

CURLOPT_HEADER => 1,

and maybe dump header result into mysql.. for easier logging.

also can use wireshark if using browser for more detailed than what F12/Firebug gives


the -v option for curl is too verbose in the error output which contains the leading *(status line) or >(request head field) or <(response head field). to get only the request head field:

curl -v -sS www..com 2>&1 >/dev/null | grep '>' | cut -c1-2 --complement

to get only the request head field:

curl -v -sS www..com 2>&1 >/dev/null | grep '<' | cut -c1-2 --complement

or to dump it into /tmp/test.txt file with the -D option

curl -D /tmp/test.txt -sS www..com > /dev/null

in order to filter the -v output, you should direct the error output to terminal and the std output to /dev/null, the -s option is to forbid the progress metering





curl