Mocking system effectively

Questions & Answers
, ,
1

I'm trying to mock a side-effect-riddled process. Part of this is to wrap a call to ifconfig. The code I'm wrapping looks like this:

(define (get-mac-address #:ifconfig-invoker [system system])
  (define mac-address-regex #px"ether.+?\n")
  (define network-config (with-output-to-string (lambda () (system "/sbin/ifconfig"))))
  (displayln network-config)
  (let ([mac-output (regexp-match mac-address-regex network-config)])
    (if mac-output
        (string-replace (string-replace (string-trim (car mac-output)) "ether " "") ":" "")
        (create-pseudo-mac-address))))

My test code looks like this:

(test-case
        "it generates the mac from ifconfig output"
      (define system-mock (mock #:behavior (thunk* (define output-port (open-output-string IFCONFIG-OUTPUT))
                                                   (define default-output-port (current-output-port))
                                                   (current-output-port output-port)
                                                   (write IFCONFIG-OUTPUT output-port)
                                                   (close-output-port output-port)
                                                   (current-output-port default-output-port))))
      
      (define result (get-mac-address #:ifconfig-invoker system-mock))
      (check-equal? result "abc"))

My thinking was that I could define IFCONIFG-OUTPUT as the output to the default output port and swap it in when the mock is called. But I'm not getting the output of (create-pseudo-mac-address) each time instead.

So:

  1. What am I doing wrong with the mock?
  2. Is there a better way to do this call to an OS-level command?
2

I think there's some confusion with the various ports and strings.

Personally, I would probably try to narrow this down -- the real vs. mock aspect -- to the "get network config string" function.

Probably I'd just define a parameter -- one that defaults to a function using system+ifconfig, but which a test could parameterize to a function returning some hardcoded test data string.

Maybe something like:

#lang racket

;; A parameter whose value is a function to get network config, which
;; defaults to using real ifconfig.

(define (system-get-if-config)
  (with-output-to-string (lambda () (system "/sbin/ifconfig"))))

(define get-network-config (make-parameter system-get-if-config))

;; A function to be tested, that uses the parameter.

(define (get-mac-address)
  (define mac-address-regex #px"ether.+?\n")
  (define network-config ((get-network-config)))
  (displayln network-config)
  (define mac-output (regexp-match mac-address-regex network-config))
  (if mac-output
      (string-replace (string-replace (string-trim (car mac-output))
                                      "ether " "")
                      ":" "")
      (create-pseudo-mac-address)))

(define (create-pseudo-mac-address)
  "000000000000") ;??

;; Example of "real" use (just use default value of get-network-config
;; parameter):
(get-mac-address)

;; Example of "mock" use (parameterize get-network-config to a
;; function that returns the mock network config data):
network config data):
(parameterize ([get-network-config (lambda () "some hardcoded if config output")])
  (get-mac-address))

Caveat: I'm not sure about all the details, although this approach works in my quick test. After using it more, in practice, I might organize things differently, idk.

Caveat: I don't have much mileage with test mocking philosophies or frameworks. It looks like you're using a mock function, which I don't know about. Maybe there is a preferred way to organize mocks, that's not "just" raw Racket parameters, idk.

Anyway I think my suggestion would be, basically, to focus the real vs. mock stuff to the most specific level, and have it work in terms of some function that returns a string (not using ports).

3

p.s. After some searching, now I'm guessing you're using the mock package?

Although I haven't digested that, obviously a lot of thought has gone into its design, it offers some great ways to organize things, and I'm sure @notjack has a lot of experience with testing using mocks, as well as with Racket.

So I think my suggestion (FWIW) is, of course use mock, but maybe still focus on mocking the smallest possible chunk of functionality, the "get network config" part. That should be simpler to get working?

2 Likes
4

So I think my suggestion (FWIW) is, of course use mock, but maybe still focus on mocking the smallest possible chunk of functionality, the "get network config" part. That should be simpler to get working

See also the "Functional Core, Imperative Shell" architecture: I would have a non-IO program that manipulates the response from ifconfig et al., plus a shim that grabs that output and feeds it to the rest of the program.

Then you can have 2 sets of tests:

  1. Does my pure program work given these inputs?
  2. Does the ifconfig program give me outputs that look like what I expect?

The latter is useful to detect changes in the program's output and only makes sense with ifconfig around.

Here's a video with code (https://www.destroyallsoftware.com/screencasts/catalog/functional-core-imperative-shell), and the same author's overview of the pattern (https://www.youtube.com/watch?v=yTkzNHF6rMs)

3 Likes
5

Yes, I am using mock and yes, it is a really nice package :slight_smile:

It's an early version of the code, so I'm open to a good refactor. :slight_smile: However, I'm starting to think I dragged in a little too much context. As you say, there's something wrong with the pipes, and I'm most interested in that.

Having said that, I'm intrigued as to why you used parameters in your code?

6

It struck me as a good way to vary the behavior of a function, without needing to supply it an explicit argument.

I imagined this might be convenient when running tests. For example maybe the function to vary is called by some higher level functions, and you wouldn't want to pass this in through the whole chain of function calls, especially not solely for the sake of tests.

It looks like the mock package has a with-mock-behavior macro that uses parameters.

1 Like
7

Hi, @robertpostill.

I have no insight on the philosophy of the thing, but I think--if I am understanding correctly--that the parameterization of the current-output-port is the issue, in the sense that no swapping is necessary.

I ran this on a linux-box I have to hand.

#lang racket/base

(require
  mock rackunit
  racket/system
  racket/port
  racket/string
  racket/function)

(define IFCONFIG-OUTPUT
#<<$$$
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 10.100.10.104  netmask 255.255.255.128  broadcast 10.100.10.127
        inet6 fe80::2372:f133:f24d:ed75  prefixlen 64  scopeid 0x20<link>
        ether 00:0c:29:10:55:86  txqueuelen 1000  (Ethernet)
        RX packets 12789  bytes 18594238 (17.7 MiB)
        RX errors 0  dropped 222  overruns 0  frame 0
        TX packets 8968  bytes 857556 (837.4 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1000  (Local Loopback)
        RX packets 8  bytes 480 (480.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 8  bytes 480 (480.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
$$$
  )

(define (create-pseudo-mac-address) "000000000000")

(define (get-mac-address #:ifconfig-invoker [system system])
  (define mac-address-regex #px"ether.+?\n")
  (define network-config (with-output-to-string (lambda () (system "/sbin/ifconfig"))))
  (displayln network-config)
  (let ([mac-output (regexp-match mac-address-regex network-config)])
    (if mac-output
        (string-replace (string-replace (string-trim (car mac-output)) "ether " "") ":" "")
        (create-pseudo-mac-address))))

(test-case
 "it generates the mac from ifconfig output"
 (define system-mock
   (mock #:behavior (thunk* (write IFCONFIG-OUTPUT))))
 (define result (get-mac-address #:ifconfig-invoker system))
 (check-equal? result (create-pseudo-mac-address)))

Output when setting the #:ifconfig-invoker to the system-mock:

"eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500\n        inet 10.100.10.104  netmask 255.255.255.128  broadcast 10.100.10.127\n        inet6 fe80::2372:f133:f24d:ed75  prefixlen 64  scopeid 0x20<link>\n        ether 00:0c:29:10:55:86  txqueuelen 1000  (Ethernet)\n        RX packets 12789  bytes 18594238 (17.7 MiB)\n        RX errors 0  dropped 222  overruns 0  frame 0\n        TX packets 8968  bytes 857556 (837.4 KiB)\n        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0\n\nlo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536\n        inet 127.0.0.1  netmask 255.0.0.0\n        inet6 ::1  prefixlen 128  scopeid 0x10<host>\n        loop  txqueuelen 1000  (Local Loopback)\n        RX packets 8  bytes 480 (480.0 B)\n        RX errors 0  dropped 0  overruns 0  frame 0\n        TX packets 8  bytes 480 (480.0 B)\n        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0"

Output when setting the invoker to the normal system:

eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 10.100.10.104  netmask 255.255.255.128  broadcast 10.100.10.127
        inet6 fe80::2372:f133:f24d:ed75  prefixlen 64  scopeid 0x20<link>
        ether 00:0c:29:10:55:86  txqueuelen 1000  (Ethernet)
        RX packets 39157  bytes 44694777 (42.6 MiB)
        RX errors 0  dropped 1068  overruns 0  frame 0
        TX packets 24502  bytes 5595740 (5.3 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1000  (Local Loopback)
        RX packets 8  bytes 480 (480.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 8  bytes 480 (480.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0


--------------------
it generates the mac from ifconfig output
. FAILURE
name:       check-equal?
location:   tmp.rkt:48:1
actual:     "000c29105586  txqueuelen 1000  (Ethernet)"
expected:   "000000000000"
--------------------

When using with-output-to-string, it automatically parameterizes the current-output-port, so whatever you write in the thunk, stays in the thunk, so to speak.

Apologies if I have misunderstood, though.

Edit: eh, I think I may have put my foot in my mouth.

So, the regexp as I have it above does not match at all (so the mac-output is always false), but this is because of the write in the mock-call.

If you swap it out for a display things look different.

#lang at-exp racket/base

(require
  mock rackunit
  racket/system
  racket/port
  racket/string
  racket/format
  racket/function)

(define IFCONFIG-OUTPUT
@~a{
 eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 10.100.10.104  netmask 255.255.255.128  broadcast 10.100.10.127
        inet6 fe80::2372:f133:f24d:ed75  prefixlen 64  scopeid 0x20<link>
        ether 00:0c:29:10:55:86  txqueuelen 1000  (Ethernet)
        RX packets 12789  bytes 18594238 (17.7 MiB)
        RX errors 0  dropped 222  overruns 0  frame 0
        TX packets 8968  bytes 857556 (837.4 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1000  (Local Loopback)
        RX packets 8  bytes 480 (480.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 8  bytes 480 (480.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0})

(define (create-pseudo-mac-address) "000000000000")

(define (get-mac-address #:ifconfig-invoker [system system])
  (define mac-address-regex #px"ether.+?\n")
  (define network-config (with-output-to-string (lambda () (system "/sbin/ifconfig"))))
  (displayln network-config)
  (let ([mac-output (regexp-match mac-address-regex network-config)])
    (if mac-output
        (string-replace (string-replace (string-trim (car mac-output)) "ether " "") ":" "")
        (create-pseudo-mac-address))))

(test-case
 "it generates the mac from ifconfig output"
 (define system-mock
   (mock #:behavior (thunk* (display IFCONFIG-OUTPUT))))
 (define result (get-mac-address #:ifconfig-invoker system-mock))
 (check-equal? result (create-pseudo-mac-address)))

 eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 10.100.10.104  netmask 255.255.255.128  broadcast 10.100.10.127
        inet6 fe80::2372:f133:f24d:ed75  prefixlen 64  scopeid 0x20<link>
        ether 00:0c:29:10:55:86  txqueuelen 1000  (Ethernet)
        RX packets 12789  bytes 18594238 (17.7 MiB)
        RX errors 0  dropped 222  overruns 0  frame 0
        TX packets 8968  bytes 857556 (837.4 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1000  (Local Loopback)
        RX packets 8  bytes 480 (480.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 8  bytes 480 (480.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
--------------------
it generates the mac from ifconfig output
. FAILURE
name:       check-equal?
location:   robertpostill-mock-testing.rkt:47:1
actual:     "000c29105586  txqueuelen 1000  (Ethernet)"
expected:   "000000000000"
--------------------

Unless, it is always supposed to be false?

Setting the mac-address to the something like
00:00:00:00:00:00 txqueuelen 1000 (Ethernet)
works for the appropriate psuedo-address, but I am not confident that I follow the logic, anymore.

1 Like
8

You've cracked it!

Once I used display and stopped messing about with the ports it worked a charm. Thanks for your help :folded_hands:

2 Likes