# Mutate: inject bugs into your programs! **URL:** \ **Category:** Show & Tell\ **Tags:** package\ **Created:** [October 15, 2022, 4:59pm UTC](https://racket.discourse.group/t/mutate-inject-bugs-into-your-programs/1388 "2022-10-15T16:59:16Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![llazarek](https://avatars.discourse-cdn.com/v4/letter/l/df705f/32.png) [@llazarek](https://racket.discourse.group/u/llazarek)\ **Post date:** [October 15, 2022, 4:59pm UTC](https://racket.discourse.group/t/mutate-inject-bugs-into-your-programs/1388/1 "2022-10-15T16:59:16Z")
Recently added to the package catalog, documented here: [Mutate](https://docs.racket-lang.org/mutate/index.html) This is a library for mutating programs, i.e. injecting possible bugs by making small syntactic changes to the program syntax. There's [an example in the docs prologue](https://docs.racket-lang.org/mutate/Prologue.html#%28part._.A_full_example%29). It might be used to create a corpus of known-buggy programs (that's why I wrote it). Or to evaluate the quality of a test suite through [mutation testing](https://en.wikipedia.org/wiki/Mutation_testing). Or something else? I'm posting it here in the hope that it can be useful to someone else as well! [Source is here](https://github.com/LLazarek/mutate) ---
**Author:** ![spdegabrielle](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/spdegabrielle/32/95_2.png) [@spdegabrielle](https://racket.discourse.group/u/spdegabrielle)\ **Post date:** [October 16, 2022, 12:32pm UTC](https://racket.discourse.group/t/mutate-inject-bugs-into-your-programs/1388/2 "2022-10-16T12:32:31Z")
Thank you @llazarek This is new to me but I'm finding your documentation very helpful. best regards Stephen ---
**Author:** ![dstorrs](https://avatars.discourse-cdn.com/v4/letter/d/898d66/32.png) [@dstorrs](https://racket.discourse.group/u/dstorrs)\ **Post date:** [October 18, 2022, 4:27pm UTC](https://racket.discourse.group/t/mutate-inject-bugs-into-your-programs/1388/3 "2022-10-18T16:27:34Z")
This is very cool, but I'm curious about the underlying reasoning. The documentation covers the 'how' but not the 'why'; can you give me a tl;dr so that I don't need to immediately dig into the literature? The example mutator changes `(if c t f)` into `(if (not c) t f)` which is essentially guaranteed to break the software. Is the point simply to verify that you have test cases that will catch this, or is there more to it? ---
**Author:** ![llazarek](https://avatars.discourse-cdn.com/v4/letter/l/df705f/32.png) [@llazarek](https://racket.discourse.group/u/llazarek)\ **Post date:** [October 19, 2022, 4:01pm UTC](https://racket.discourse.group/t/mutate-inject-bugs-into-your-programs/1388/4 "2022-10-19T16:01:27Z")
Thanks @spdegabrielle and @dstorrs! Yeah you're exactly right @dstorrs that the if-swap very likely breaks the program, and that one way to use mutation is to check that a test suite catches such bugs. That's the rationale behind mutation testing: a good test suite should be able to catch bugs injected into the code-to-be-tested. On the other hand, I can elaborate on my use case as a different example of why one might want mutants. I've been using mutation to analyze the debugging information different tools (like contracts) provide when programs go wrong. To do that, I need a corpus of buggy programs where I know for each one exactly where the bug is. So I use mutation to turn a set of programs that aren't known to be buggy into a huge number of mutants, each with a different potential bug that serves as a different scenario for my analysis. ---
**Author:** ![jbclements](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/jbclements/32/11_2.png) [@jbclements](https://racket.discourse.group/u/jbclements)\ **Post date:** [October 20, 2022, 9:21pm UTC](https://racket.discourse.group/t/mutate-inject-bugs-into-your-programs/1388/5 "2022-10-20T21:21:40Z")
> [@llazarek](#): > > To do that, I need a corpus of buggy programs where I know for each one exactly where the bug is. Though... you do need to verify that the change actually introduces a bug, right? Certainly one possible scenario is that you make a change in dead code. I'm sure you've already thought about that. ---
**Author:** ![llazarek](https://avatars.discourse-cdn.com/v4/letter/l/df705f/32.png) [@llazarek](https://racket.discourse.group/u/llazarek)\ **Post date:** [October 21, 2022, 2:53pm UTC](https://racket.discourse.group/t/mutate-inject-bugs-into-your-programs/1388/6 "2022-10-21T14:53:50Z")
Yes indeed. And that can be a tricky thing to check! In my use-case we take the simple and conservative choice to filter for mutants that cause the program to crash. ---
**Author:** ![amirouche](https://avatars.discourse-cdn.com/v4/letter/a/f475e1/32.png) [@amirouche](https://racket.discourse.group/u/amirouche)\ **Post date:** [June 13, 2023, 9:06pm UTC](https://racket.discourse.group/t/mutate-inject-bugs-into-your-programs/1388/7 "2023-06-13T21:06:26Z")
> [@jbclements](#): > > you do need to verify that the change actually introduces a bug, right? Certainly, one possible scenario is that you make a change in dead code. In the mutation testing program I built, I only mutate code that is covered by test cases. Mutating dead-code according to the test suite is not useful, and slows down the process of making the test suite robust. Here is a related scenario: Given one expression that is green on coverage, the associated tests are also green, but when you remove that said expression, the tests are still green ⇒ the given expression is not tested. That can happen when side effects are not tested. Mutation testing will verify the test suite is robust. I think it can be summarized like: ```scheme (for-each (lambda (mutated-program) (assert (not (program-check mutated-program)))) (mutate program)) ``` Where `program-check` runs `program`'s test suite against the mutated program. In case of timeout, program-check must return `#f`. Dead-code will not be exercised by the test suite, hence whatever mutation is done to it, `program-check` will return the nominal result `#t` which is an error according to mutation testing: the test suite is not robust enough. It is known beforehand because of the coverage. The problem of mutating dead code according to the test suite, is that it yields many "candidate mutations" that are noise. I struggled with the following: - For all mutations, the program test suite fails ⇒ mutation testing success ⇒ test suite is robust; - It should^W must be possible to run the test suite concurrently; otherwise it is far too costly to run; - The more knowledge you can infer from the code, the more interesting mutations it is possible to infer; it is possible to filter out mutations that will always fail, such as replacing `+` with `list`, and avoid a combinatorial explosion. Mutation testing is fuzzing for code.