cURL Cheatsheet

cUrl useful commands with params

Here’s a cheatsheet for the most useful cURL commands with their parameters:

a person with a curl sign

This image above is generated by Flux - text to image AI model.

Basic cURL Usage

  • Download a file: curl http://example.com/file.zip -o file.zip
  • Download multiple files: curl -O URL1 -O URL2
  • Follow redirects: curl -L http://example.com/file
  • Display only headers: curl -I http://example.com

Authentication

  • Basic authentication: curl -u username:password http://example.com
  • Use .netrc file: curl --netrc-file .netrc http://example.com

HTTP Methods

  • POST request: curl -X POST -d 'name=value' http://example.com
  • PUT request: curl -X PUT -d @file http://example.com
  • Custom method: curl -X METHOD http://example.com

Headers and Data

  • Add custom header: curl -H "X-Header: Value" http://example.com
  • Send JSON data: curl -H "Content-Type: application/json" -d '{"key":"value"}' http://example.com
  • Send form data: curl -F "key=value" -F "file=@localfile" http://example.com

SSL/TLS Options

  • Ignore SSL certificate verification: curl -k https://example.com
  • Use client certificate: curl --cert cert.pem --key key.pem https://example.com

Proxy and Networking

  • Use a proxy: curl -x proxysever.server.com:PORT http://example.com
  • Limit download rate: curl --limit-rate 1000B -O http://example.com/file

Output Control

  • Silent mode: curl -s http://example.com
  • Verbose mode: curl -v http://example.com
  • Write output to file: curl -o file.html http://example.com
  • Save with remote filename: curl -O http://example.com/file.zip

Miscellaneous

  • Resume a failed download: curl -C - -o partial_file.zip http://example.com/file.zip
  • Set a timeout: curl --connect-timeout 10 http://example.com
  • Use a config file: curl -K config_file http://example.com

This cheatsheet covers the most commonly used cURL commands and their parameters, providing a quick reference for various HTTP operations and data transfers.

Advanced cUrl commands

Here are some advanced curl commands for debugging:

  1. Verbose output: curl -v https://example.com This command provides detailed information about the request and response, including headers and connection details.

  2. Full trace dump: curl --trace - https://example.com This outputs a hexdump of all incoming and outgoing data, offering a comprehensive view of the entire communication.

  3. Trace with timestamps: curl --trace - --trace-time https://example.com This adds timestamps to the trace output, helping to analyze timing-related issues.

  4. Print only response headers: curl -s -o /dev/null -D - https://example.com This command shows only the response headers, useful for quick header analysis.

  5. Print only request headers: curl -v -s -o /dev/null --stderr - https://example.com | grep '^>' This complex command filters out only the request headers, helpful for inspecting what’s being sent.

  6. Use Pantheon Debugger: curl -I -H "Pantheon-Debug:1" https://example.com This command uses Pantheon’s Debug header to get additional information about a request, useful for Pantheon-hosted sites.

  7. Force HTTP/2 protocol: curl --http2 https://example.com This ensures the use of HTTP/2 protocol, which can be combined with -I to verify HTTP/2 support.

  8. Debug with netcat: In one terminal: nc -l 1234 In another: curl http://127.0.0.1:1234 This allows you to see exactly what curl is sending, useful for payload debugging.

These advanced commands provide powerful debugging capabilities, allowing developers to inspect various aspects of HTTP communications and troubleshoot complex issues.

Common cUrl usecases

curl is a versatile command-line tool with numerous applications. Here are some of the most common use cases for curl:

Web Interactions

  1. Downloading files: curl can download files from URLs.
  2. Web scraping: curl can retrieve web page content for scraping purposes.
  3. Testing websites: curl is useful for checking if a website is accessible and retrieving its content.

API Interactions

  1. Sending API requests: curl supports various HTTP methods (GET, POST, PUT, DELETE) for interacting with APIs.
  2. Testing API endpoints: curl provides a quick way to test and debug API endpoints without needing a dedicated API client.

Network Troubleshooting

  1. Checking connectivity: curl can be used to verify network connections and troubleshoot issues.
  2. Testing DNS: curl allows forcing specific DNS servers for troubleshooting DNS-related problems.
  3. Inspecting headers: curl can display HTTP headers, which is useful for debugging.

Data Transfer

  1. FTP operations: curl supports file transfers using the File Transfer Protocol (FTP).
  2. SMTP interactions: curl can be used to send emails via the Simple Mail Transfer Protocol (SMTP).

Security and Authentication

  1. SSL connections: curl supports secure connections using SSL/TLS.
  2. Authentication: curl can handle various authentication methods, including basic auth and client certificates.

Development and Debugging

  1. Simulating different HTTP methods: curl allows developers to test various HTTP methods easily.
  2. Verbose logging: curl’s verbose mode provides detailed information about the request-response cycle, aiding in debugging.

These use cases demonstrate curl’s flexibility as a tool for web interactions, API testing, network troubleshooting, and development tasks.

Website performance testing with cURL

curl is a powerful tool for testing website performance. Here’s how you can use it to measure various aspects of a website’s loading speed:

Basic Performance Test

To get a simple measure of a website’s loading time, use this command:

curl -o /dev/null -s -w "Total Time: %{time_total}s\n" "https://example.com"

This command will output the total time it took to fetch the website[2].

Detailed Performance Metrics

For a more comprehensive analysis, you can use curl to measure multiple performance metrics:

curl -s -w 'Testing Website Response Time for :%{url_effective}\n\nLookup Time:\t\t%{time_namelookup}\nConnect Time:\t\t%{time_connect}\nPre-transfer Time:\t%{time_pretransfer}\nStart-transfer Time:\t%{time_starttransfer}\n\nTotal Time:\t\t%{time_total}\n' -o /dev/null https://example.com

This command provides a breakdown of different stages in the loading process, including DNS lookup time, connection time, and total time[1].

Advanced Metrics

For even more detailed analysis, you can include additional metrics:

curl -o /dev/null -w "HTTP Version: %{http_version}\nPage Size: %{size_download} bytes\nResponse Code: %{response_code}\nDNS Lookup: %{time_namelookup} sec\nConnect Time: %{time_connect} sec\nTime to First Byte: %{time_starttransfer} sec\nTotal Time: %{time_total} sec\n" -s https://example.com

This command includes information about the HTTP version, page size, and response code[3].

Benchmarking Multiple Requests

To test consistency or monitor performance over time, you can use a loop to make multiple requests:

for i in {1..5}; do curl -o /dev/null -s -w "Request $i Total Time: %{time_total}\n" https://example.com; done

This will run the test five times and show the total time for each request[3].

Creating a Performance Test Script

You can create a simple bash script to automate these tests and save the results:

#!/bin/bash
curl -s -w 'Testing Website Response Time for :%{url_effective}\n\nLookup Time:\t\t%{time_namelookup}\nConnect Time:\t\t%{time_connect}\nPre-transfer Time:\t%{time_pretransfer}\nStart-transfer Time:\t%{time_starttransfer}\n\nTotal Time:\t\t%{time_total}\n' -o /dev/null URL > webspeedtest_"$(date)"

Replace “URL” with the website you want to test. This script will save the results in a date-stamped file[5].

By using these curl commands and scripts, you can effectively measure and monitor website performance, helping you identify potential bottlenecks and optimize loading times.