Call an HTTP API from the Command Line with curl

A large part of today's engineering work is talking to HTTP APIs: cloud providers, package registries, monitoring systems, internal services of your own company, etc. Client libraries and purpose-built CLI tools hide the API usage details and make the happy flows easier, but when something goes wrong, the cleanest way to see what an API really answers is to call it directly.

The curl command is one of the most common tools for that. In this challenge, you will use curl against an internal HTTP service to practice performing the most typical API access tasks.

Warm-Up: A Public API

Start with a well-known public API. GitHub is a good candidate - it has an extensive API, which in particular exposes information about every public repository. For example:

curl https://api.github.com/repos/moby/moby

The response is a JSON document with the repository's description, star count, default branch, and the like.

Important

If you got a 403 or 429 response from the GitHub API, simply continue with the tasks below.

GitHub allows a limited number of unauthenticated requests per hour from one IP address, and the information about the current limit is sent in response headers. You can find it by using the -I option of the curl command (look for the x-ratelimit-* lines in the output):

curl -I https://api.github.com/repos/moby/moby

There is nothing special about GitHub here. An internal service on your company network speaks the same protocol and is called with the same curl command. The rest of the challenge uses such an internal service.

The Internal API

The company runs an Artifact Inventory service that keeps track of build artifacts. It is reachable from the workstation-01 machine at http://api.corp.internal:8080 and offers the following endpoints:

GET /api/v1/status - get service health and version information
GET /api/v1/artifacts - list all artifacts known to the service
GET /api/v1/artifacts/<name> - request one artifact by its name
GET /api/v1/artifacts/<name>/manifest - read the manifest of an artifact
GET /api/v1/artifacts/latest - an alias for the most recently published artifact

The tasks below are about finding things out by calling the API.

Task 1: Read a JSON Body

Which version of the service is running? The status endpoint reports it in the version field of its JSON response.

Hint 1

Without any options, curl <url> sends a GET request and prints the response body to the terminal. The body of the status endpoint is a small JSON document, so the value is easy to spot by eye.

Task 2: Read a Response Header

Every artifact is stored on one of several storage nodes, and the API reveals the node in a response header called X-Storage-Node. Which storage node serves the artifact named report-builder?

Hint 2

By default, curl prints only the response body. The -i option includes the response headers in the output, and the -I option asks for the headers only (it makes curl to send a HEAD request instead of the default GET).

Note that header names are case-insensitive, so the server may print them in lower case (e.g., x-storage-node instead of X-Storage-Node).

Task 3: Read a Status Code

The artifact named legacy-exporter was retired a while ago. What HTTP status code does the server return when you request it?

Hint 3

The status code is on the first line of the response, which curl -i and curl -I show.

For scripting, curl -w %{http_code} -o /dev/null <url> can also print just the status code and (the -o option pointing to /dev/null discards the response body).

Check man curl or curl --help all for the details.

Task 4: Follow a Redirect

The latest alias does not return an artifact itself. Instead, the server answers with a redirect to the artifact that is currently the latest one. Which artifact does latest point to?

Hint 4

A redirect response has a 3xx status code and a Location header with the new URL. The -L option makes curl follow redirects automatically, and with -i you can see both responses.

Task 5: Request a Specific Representation

A resource on the server can have more than one representation. The manifest endpoint returns a short plain-text summary by default, but it can also return the full manifest as JSON when the client says it accepts the application/vnd.corp.manifest.v2+json media type.

What is the buildId of the report-builder artifact? It is only present in the JSON form of the manifest.

Hint 5

Request headers are set with the -H option, for example -H 'Name: value'. The header that tells a server which media types the client accepts is called Accept.

Task 6: Save a Response Unchanged

The ultimate challenge. Save the JSON manifest of report-builder to ~/manifest.json on workstation-01. The file must contain the response body exactly as the server sent it, byte for byte. The server sends the SHA-256 digest of the manifest in the X-Manifest-Digest response header, so you can check your copy with sha256sum ~/manifest.json.

Hint 6

The -o <file> option writes the response body to a file instead of the terminal.

If you want to redirect the curl <url> output to the file instead, do not pipe the response through a formatter, and do not include the headers in the file - any change in whitespace or extra bytes gives the file a different digest.

Tasks