Build a URL with query string parameters with curl

• Posted 5 July 2024
• Tagged with curl

A combination of --get and --data/--data-urlencode allows you to write curl commands which are readable and expressive.

I want to write an HTTP GET in curl which:

• somebody can copy/paste to run themselves
• would allow somebody else to make the same request in their HTTP library of choice
• has easily readable URL query parameters

How can I do that with curl?

Motivation

I’m writing some documentation for the Wikimedia Commons API, and I want to include some runnable examples. The various query parameters get baked into a single GET URL, but that makes it quite difficult to see what’s going on:

curl 'https://commons.wikimedia.org/w/api.php?action=query&list=logevents&letitle=File%3AMugeni%20Elijah%20-%20Behind%20tha%20sin%20ArtWork.png&format=xml&letype=delete'

There are five query parameters being passed to that URL! But this syntax makes it quite difficult to read and translate into another HTTP library.

In Python, I’m used to expressing the query parameters as a structured object (which also handles the URL encoding for you), for example:

import httpx

httpx.get(
    "https://commons.wikimedia.org/w/api.php",
    params={
        "action": "query",
        "list": "logevents",
        "letitle": "File:Mugeni Elijah - Behind tha sin ArtWork.png",
        "format": "xml",
        "letype": "delete"
    }
)

Compared to the curl command, this is easier for somebody to read and translate to another language, but it’s harder to copy/paste for a quick experiment.

Solution

I looked at the curl manpage, and I found the -G, --get flag (emphasis mine):

When used, this option makes all data specified with -d, --data, --data-binary or --data-urlencode to be used in an HTTP GET request instead of the POST request that otherwise would be used. curl appends the provided data to the URL as a query string.

This allows me to write the curl command in a more verbose, expressive way so somebody can see what’s going on:

curl --get 'https://commons.wikimedia.org/w/api.php' \
  --data 'action=query' \
  --data 'list=logevents' \
  --data-urlencode 'letitle=File:Mugeni Elijah - Behind tha sin ArtWork.png' \
  --data 'format=xml' \
  --data 'letype=delete'

(The --data-urlencode flag was a bonus discovery, which saves me from URL encoding parameters by hand.)