Hi, Racket Discourse.
I have been exposed over the past two years to a much wider variety of ways in which to interact with computers. I have mostly favored building frog-like software preceding this period; low to the ground, so to speak, for my own use.
One of the areas with which I have had to become more familiar, is interacting with Web APIs via HTTP requests. Although by no means exhaustive, I have learned a great deal.
I have frequently, over this time, had to interact with various APIs in somewhat ephemeral ways, because my work requires testing them and then evaluating their suitability. Depending on the utility, we might move on quickly, or continue development.
I have accumulated some ideas into a yet undocumented project called recce. This is an allusion to the South African special forces brigade. Being a boy from RSA, it was part of the mythology of our youth. The connection being, quick-and-precise, in-and-out operations.
Also, in Afrikaans, my native tongue, rekkie, which is an elastic band, is pronounced as recce.
In the interest of sharing recipes, here are my culinary impressions.[0]
I use the project to define API endpoints quickly and some of the machinery required around that. Let's see some examples, then.
We begin by defining our API HTTP methods, such as GET, POST, etc.
In theory, you could use any library to provide this, but I generally use http-easy because it really is that easy.
I'll use AlienVault (the Open Threat Exchange) as an example, because I am working with this at present, and it is free to use, should you be curious.
(require recce net/http-easy)
(define-api-methods alienvault "https://otx.alienvault.com/api/v1"
get post)
This macro call produces definitions for alienvault:get and alienvault:post structs. These structs are derived from a basic recce-endpoint struct, which has a path, a query, and a move mutator (field) for modifying the former fields.
(struct recce-endpoint [path query move] #:transparent)
> alienvault:get
#<procedure:constructor>
> alienvault:post
#<procedure:constructor>
It also produces a definition for alienvault which is simply the API-base string provided after the aforementioned identifier. This has been useful sometimes, but I am ambivalent about it.
> alienvault
"https://otx.alienvault.com/api/v1"
Next, we define some endpoints for these methods:
(define-endpoint alienvault:get ipv4-indicators
#:path
indicators / IPv4 / {ip} / {section})
There are a couple of things to note:
- We use the appropriate method-struct's identifier as the base of the endpoint definition,
alienvault:get. - We define the endpoint's URL segments using a syntax that is mostly similar to the way it would be presented in an API's documentation, from what I have encountered. The segments wrapped in
{}indicate variable segments we would like to substitute. The rest remain static.
The macro produces a definition for the ipv4-indicators struct, which has fields for each of the segments defined.
> ipv4-indicators
#<procedure:constructor>
> ipv4-indicators-ip
#<procedure:ipv4-indicators-ip>
> ipv4-indicators-section
#<procedure:ipv4-indicators-section>
If we had query parameters for this endpoint, they would be fields also.
; just to demonstrate what that would look like
(define-endpoint alienvault:get pretend-ipv4-indicators
#:path
indicators / IPv4 / {ip} / {section}
#:query
[limit 100])
; note the `?` before the query parameter, to distinguish them from path segments
> pretend-ipv4-indicators-?limit
#<procedure:pretend-ipv4-indicators-?limit>
Query parameters are coerced to lists, so providing multiple values will produce query parameters with a comma-separated argument-list (which is escaped properly when constructing the URL, thankfully).
We construct the following instance of the actual endpoint:
(define endpoint
[alienvault:get-ipv4-indicators
#:ip '8.8.8.8
#:section 'reputation])
The procedure being called here, is not the constructor itself, but a wrapper which is produced during the macro call of the endpoint definition. Any query parameters work the same, although they are optional arguments, also prefixed by ? as above, e.g., #:?limit in our pretend example.
Such an instance has a prop:procedure which allows one to call the endpoint to perform some work.
By default, it will produce the struct for the URL, but by providing a non-false #:delegate (procedure), the call will produce a procedure (in fact, a clobbered version of the HTTP method provided in the endpoint definition), which in turn will result in a network call to the resource when applied.
> (endpoint)
(url
"https"
#f
"otx.alienvault.com"
#f
#t
(list
(path/param "api" '())
(path/param "v1" '())
(path/param "indicators" '())
(path/param "IPv4" '())
(path/param "8.8.8.8" '())
(path/param "reputation" '()))
'()
#f)
> (endpoint #:delegate recce-identity)
#<procedure:curried:composed>
As a convenience, there exists an endpoint->url procedure to abbreviate the first case.
Otherwise, the delegate is applied to the response from the resource, when it is received. This, I have found to be useful when checking status codes, or when having to dispatch recurrent calls to a paginated resource.
The delegate, if non-false, is assumed to be a procedure of 2 arguments, namely: The endpoint used to make the call, and the response. I picked up this idiom from using @bogdan's crontab, which does something similar with his schedules and their timestamps.
(define recce-identity (lambda (endpoint response) response))
#:delegate
(lambda (endpoint response)
(match (response-status-code response)
[200 #;ok
... endpoint response ...]
[400 #;bad-request
... endpoint response ...]
[401 #;unauthorized
... endpoint response ...]
[418 #;Im-a-teapot
... endpoint response ...]))
The delegate can also be controlled via the recce-delegate parameter, although the argument provided to the instance of the endpoint will take precedence if non-false.
(define recce-delegate (make-parameter #false))
This is useful for more dynamic control in certain situations.
> (parameterize ([recce-delegate recce-identity])
(endpoint))
#<procedure:curried:composed>
Now, supposing we would like to modify an existing instance of an endpoint, within the bounds of its definition, we can call the recce-endpoint-move field's procedure:
> ((recce-endpoint-move endpoint)
#:ip '1.1.1.1
#:section 'general)
(ipv4-indicators
'(indicators IPv4 1.1.1.1 general) '() #<procedure:endpoint-mtor> '1.1.1.1 'general)
Although, this is somewhat ugly, so the project provides a nicer wrapper, called endpoint-move:
> (endpoint-move endpoint
#:ip '1.1.1.1
#:section 'general)
(ipv4-indicators
'(indicators IPv4 1.1.1.1 general) '() #<procedure:endpoint-mtor> '1.1.1.1 'general)
Again, the same goes for the query parameters, should any exist.
The instances are immutable, so these are not modifying the originals.
Setting up a request, we use the HTTP method's arguments to refine our intent:
> ((endpoint #:delegate recce-identity)
#:headers
(hasheq 'X-OTX-API-KEY "api-key")
#:timeouts
(make-timeout-config #:connect 10
#:request 60))
#<response>
Because these kinds of requests can often be abstracted to workflows we would like to be reproducible, the project provides a macro, called endpoint-request.
Calls to the macro come in two flavors, one producing a procedure which takes as an argument an endpoint, and, one which is executed immediately, as below:
(define prepared-request
(endpoint-request
#:headers
(hasheq 'X-OTX-API-KEY "api-key")
#:timeouts
(make-timeout-config #:connect 10
#:request 60)))
(prepared-request endpoint)
;=> #<response>
(define immediate-request
(endpoint-request #:to endpoint
#:headers
(hasheq 'X-OTX-API-KEY "api-key")
#:timeouts
(make-timeout-config #:connect 10
#:request 60)))
immediate-request
;=> #<response>
Additionally, as with the call to the instance of the endpoint itself, one is free to provide a #:handler (note the different keyword, to make it clear we want a procedure in this case). The position does not matter.
(endpoint-request
#:headers
(hasheq 'X-OTX-API-KEY "api-key")
#:timeouts
(make-timeout-config #:connect 10
#:request 60)
#:handler
recce-identity)
I am not entirely convinced of the interaction between the #:delegate argument to the endpoint instance and the recce-delegate parameter. Nonetheless, the delegate will be captured at the request definition, if non-false.
(define prepared-request-a
(parameterize ([recce-delegate
(lambda (_ response) (displayln 'captured) response)])
(endpoint-request
#:headers
(hasheq 'X-OTX-API-KEY "api-key")
#:timeouts
(make-timeout-config #:connect 10
#:request 60))))
(parameterize ([recce-delegate
(lambda (_ response) (displayln 'ignored) response)])
(prepared-request-a endpoint))
; captured
;=> #<response>
(define prepared-request-b
(endpoint-request
#:headers
(hasheq 'X-OTX-API-KEY "api-key")
#:timeouts
(make-timeout-config #:connect 10
#:request 60)))
(parameterize ([recce-delegate
(lambda (_ response) (displayln 'ignored) response)])
(prepared-request-b endpoint))
; ignored
;=> #<response>
That's mostly it, for now. I hope this gets your creative juices flowing, too!
The code can be found in this gist, but as I say, it is still experimental.
[0] I was inspired early on in my macro journeys by the Syntax Parse Examples in the Racket docs, in particular @sorawee's js-dict. It's really good stuff, and I learned so much from the few readings I have made of it, already.