M y    b r a i n    h u r t s  !                                           w e                 r e a l l y                 t h i n k                   w h a t                y o u             k n o w

16 October 2017

Barcelona city bikes availability: a console REST client with curl and jq

How do you consume a REST api in the terminal? All we can do is just curl a REST end point which will spit some uglified JSON into the console, right?

Let's say we need to get a list of public bike stations in Barcelona:

curl -s https://bicing.barcelona/get-stations

Hmmm, that's not very readable... Can we do better than this?

Yes, we can!

Meet jq — a full-fledged console JSON parser, as powerful as awk but for JSON.

Available in standard linux repos:

sudo apt install jq

as well as in homebrew on Mac:

brew install jq

Now we can display the list of all bike stations in Barcelona with JSON pretty-printing and colorization:

curl -s https://bicing.barcelona/get-stations | jq

Now, this is more readable. But how about filtering for specific bike stations, say the ones closest to our daily destinations?

We look up the station IDs we need at the offical Barcelona bicing
and pass the station IDs as parameters to jq:

curl -s https://bicing.barcelona/get-stations --compressed | jq -r '.stations[] | select(.id==("153", "154", "339", "382","150"))'

Since we don't want all the fields in the JSON, we pass in the JSON keys for the data we want to keep.

Get bikes by station IDs, and extract station address, number of bikes, and number of parking slots:

curl -s https://bicing.barcelona/get-stations --compressed | jq -r '.stations[] | select(.id==("153", "154", "339", "382","150")) | .streetName,.bikes,.slots'

This looks a bit better, but by default jq prints one key per line, and we want to add some formatting, so that data for one station shows in a single line. We can use jq string interpolation to achieve this. Here is the final version which prints our data in a tabular format (available bikes, parking slots, station id, and address):

echo 'AVL\tEMP\tSID\tADR'
curl -s https://www.bicing.barcelona/get-stations --compressed | jq -r '.stations[] | select(.id==("153", "154", "339", "382","150")) | "\(.bikes)\t\(.slots)\t\(.id)\t\(.streetName), \(.streetNumber)"'

And here is the final result:

And, finally: what's the point of adding the --compressed parameter to curl?

Setting --compressed is equivalent to setting --header 'Accept-Encoding: gzip,deflate' — as a result the server will compress the output with gzip or deflate (if capable of compression), and curl will decompress the response on the fly, which will make the result appear faster since less bytes will be sent over the network.