M y    b r a i n    h u r t s  !                                           w e                 r e a l l y                 t h i n k                   w h a t                y o u             k n o w

16 October 2017

Barcelona city bikes availability: a console REST client with curl and jq

How do you consume a REST api in the terminal? All you can do is just curl a REST end point which will spit some uglified JSON into the console, right?

Let's say you want to get a list of public bike stations in Barcelona:

curl -s

Hmmm, that's not very readable... Can you do better than this?

Yes, you can!

Meet jq — a full-fledged console JSON parser, as powerful as awk but for JSON.

Available in standard linux repos:

sudo apt install jq

as well as in homebrew on Mac:

brew install jq

Now we can display the list of all bike stations in Barcelona with JSON pretty-printing and colorization:

curl -s | jq

Now, this is more readable. But how about filtering for specific bike stations, say the ones closest to our daily destinations?

I pulled the station IDs at offical Barcelona bicing
and pass the IDs as parameters to jq:

curl -s --compressed | jq -r '.stations[] | select(.id==("153", "154", "339", "382","150"))'

Since I don't want all the fields in the JSON, I pass in the JSON keys for the data I want to keep.

Get bikes by station IDs, and extract station address, number of bikes, and number of parking slots:

curl -s --compressed | jq -r '.stations[] | select(.id==("153", "154", "339", "382","150")) | .streetName,.bikes,.slots'

This looks a bit better, but by default jq prints one key per line, and I want to add some formatting, so that data for one station shows in a single line. I can use jq string interpolation to achieve this. Here is the final version which prints our data in a tabular format (available bikes, parking slots, station id, and address):

echo 'AVL\tEMP\tSID\tADR'
curl -s --compressed | jq -r '.stations[] | select(.id==("153", "154", "339", "382","150")) | "\(.bikes)\t\(.slots)\t\(.id)\t\(.streetName), \(.streetNumber)"'

And here is the final result:

And, finally, what's the point of adding the --compressed parameter to curl?

Setting --compressed is equivalent to setting --header 'Accept-Encoding: gzip,deflate' — as a result the server will compress the output with gzip or deflate (if capable of compression), and curl will decompress the response on the fly, which will make the result appear faster since less bytes will be sent over the network.

No comments :

Post a Comment