cURL is a versatile command-line tool available by default on macOS and Linux (and easily installed on Windows). It allows administrators to remotely transfer data, and it’s most commonly used to interact with URLs. Think of cURL as a terminal-based browser that retrieves web content but doesn’t render it. It’s especially useful for performance testing, debugging, and understanding website behavior.
Getting Started: Basic Website Interaction
To fetch the HTML source code of a website like noc.org, simply run:
curl noc.org
This command retrieves and displays the site’s source code directly in your terminal. To include the HTTP headers in the output, add the -D flag:
curl -s -D - https://noc.org | head
Example Output:
HTTP/2 200
server: nginx
date: Tue, 04 May 2021 17:09:58 GMT
content-type: text/html; charset=UTF-8
vary: Accept-Encoding
noc-cdn-location: cdn-edge-usa-west-la
noc-cdn-cachestatus: HIT
Measuring Website Performance with cURL
One of the standout features of cURL is its ability to measure performance metrics of HTTP/HTTPS requests. Using the -w (write-out) flag, you can extract detailed timing information such as:
| Parameter | Description |
|---|---|
%{time_namelookup} | Time taken for DNS resolution |
%{time_connect} | Time to establish the connection |
%{time_starttransfer} | Time until the first byte is received (TTFB) |
%{time_total} | Total time for the request |
Here’s how to use these variables to measure performance for noc.org:
curl -o /dev/null -w "DNS Lookup: %{time_namelookup} sec\nConnect Time: %{time_connect} sec\nTime to First Byte: %{time_starttransfer} sec\nTotal Time: %{time_total} sec\n" -s https://noc.org
Example Output:
DNS Lookup: 0.001506 sec
Connect Time: 0.012004 sec
Time to First Byte: 0.067635 sec
Total Time: 0.089123 secComparing Performance Across Sites
You can use cURL to compare your site’s performance with others, like cnn.com:
curl -o /dev/null -w "DNS Lookup: %{time_namelookup} sec\nConnect Time: %{time_connect} sec\nTime to First Byte: %{time_starttransfer} sec\nTotal Time: %{time_total} sec\n" -s https://cnn.comExample Output:
DNS Lookup: 0.001483 sec
Connect Time: 0.010323 sec
Time to First Byte: 0.055365 sec
Total Time: 0.815822 sec
In this example, the total time for CNN is higher, likely due to the amount of content or server-side processing.
Advanced Metrics for Troubleshooting
cURL also provides other metrics for deeper analysis:
%{size_download}: Total size of the downloaded content%{response_code}: HTTP response code%{http_version}: HTTP version used by the server
Here’s an example of combining these metrics for a detailed report:
curl -o /dev/null -w "HTTP Version: %{http_version}\nPage Size: %{size_download} bytes\nResponse Code: %{response_code}\nDNS Lookup: %{time_namelookup} sec\nConnect Time: %{time_connect} sec\nTime to First Byte: %{time_starttransfer} sec\nTotal Time: %{time_total} sec\n" -s https://www.cnn.com
Example Output:
HTTP Version: 2
Page Size: 1109810 bytes
Response Code: 200
DNS Lookup: 0.001340 sec
Connect Time: 0.010408 sec
Time to First Byte: 0.081696 sec
Total Time: 0.744870 sec
Analyzing Performance Results
The metrics you collect can help pinpoint performance issues:
- High DNS lookup times may indicate slow DNS resolution or network issues.
- Long connection times could mean latency in establishing the TCP handshake.
- High TTFB (Time to First Byte) could indicate server-side processing delays.
- Large total times suggest issues with content delivery, such as large files or slow network speeds.
Benchmarking with Multiple Requests
To test consistency or monitor performance over time, you can loop through multiple requests:
for i in {1..5}; do curl -o /dev/null -s -w "Request $i Total Time: %{time_total}\n" https://noc.org; done
Example Output:
Request 1 Total Time: 0.089
Request 2 Total Time: 0.091
Request 3 Total Time: 0.092
Request 4 Total Time: 0.088
Request 5 Total Time: 0.090
cURL is a powerful, lightweight tool for diagnosing performance issues, benchmarking, and monitoring website behavior. By leveraging its various metrics, you can quickly identify bottlenecks and optimize your site’s performance. Whether you’re a developer, network engineer, or site administrator, cURL is an invaluable addition to your toolkit.