# How does in-list "provide better performance"? **URL:** \ **Category:** Questions & Answers\ **Tags:** question\ **Created:** [November 9, 2021, 3:10pm UTC](https://racket.discourse.group/t/how-does-in-list-provide-better-performance/61 "2021-11-09T15:10:30Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![badkins](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/badkins/32/99_2.png) [@badkins](https://racket.discourse.group/u/badkins)\ **Post date:** [November 9, 2021, 3:10pm UTC](https://racket.discourse.group/t/how-does-in-list-provide-better-performance/61/1 "2021-11-09T15:10:31Z")
The [documentation](https://docs.racket-lang.org/reference/sequences.html#%28def._%28%28lib._racket%2Fprivate%2Fbase..rkt%29._in-list%29%29) for `in-list` states: > An in-list application can provide better performance for list iteration when it appears directly in a for clause. How is this accomplished? It _seems_ counter-intuitive to increase performance by adding an application. ---
**Author:** ![samth](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/samth/32/3_2.png) [@samth](https://racket.discourse.group/u/samth)\ **Post date:** [November 9, 2021, 3:23pm UTC](https://racket.discourse.group/t/how-does-in-list-provide-better-performance/61/2 "2021-11-09T15:23:47Z")
The `for` forms recognize `in-list` (or really they expand `in-list` which describes what to do) which produces code that calls `car` and `cdr` directly. If you don’t use `in-list`, then the `for` expansion uses more general operations such as `make-sequence` to handle the sequence. ---
**Author:** ![badkins](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/badkins/32/99_2.png) [@badkins](https://racket.discourse.group/u/badkins)\ **Post date:** [November 9, 2021, 3:26pm UTC](https://racket.discourse.group/t/how-does-in-list-provide-better-performance/61/3 "2021-11-09T15:26:48Z")
I just looked at the source for `in-list`, and I was struck by this clause: ```auto [[(id) (_ lst-expr)] ``` I didn’t realize a macro wasn’t required to be in the outermost form, or maybe I’m totally misunderstanding this. The \_ is a placeholder for `in-list`, right? And yet the clause refers to elements outside of the `(in-list ...)` form. What is going on here? 🙂 Here’s the full version for context: ```auto (define-sequence-syntax *in-list (lambda () #'in-list) (lambda (stx) (syntax-case stx (list) [[(id) (_ (list expr))] #'[(id) (:do-in ([(id) expr]) #t () #t () #t #f ())]] [[(id) (_ lst-expr)] #'[(id) (:do-in ;;outer bindings ([(lst) lst-expr]) ;; outer check (unless-unsafe (check-list lst)) ;; loop bindings ([lst lst]) ;; pos check (pair? lst) ;; inner bindings ([(id) (unsafe-car lst)] [(rest) (unsafe-cdr lst)]) ; so `lst` is not necessarily retained during body ;; pre guard #t ;; post guard #t ;; loop args (rest))]] [_ #f]))) ``` ---
**Author:** ![samth](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/samth/32/3_2.png) [@samth](https://racket.discourse.group/u/samth)\ **Post date:** [November 9, 2021, 4:20pm UTC](https://racket.discourse.group/t/how-does-in-list-provide-better-performance/61/4 "2021-11-09T16:20:38Z")
`in-list` isn’t a regular macro, it’s a special form that’s handled by the `for` forms, which is why it’s defined with `define-sequence-syntax`. The `for` expansion recognizes `in-list` on the right hand side of a `for` clause, and then calls the macro with the entire clause, not just the right hand side. That’s why the patterns look the way they do here. ---
**Author:** ![benknoble](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/benknoble/32/16_2.png) [@benknoble](https://racket.discourse.group/u/benknoble)\ **Post date:** [November 9, 2021, 9:04pm UTC](https://racket.discourse.group/t/how-does-in-list-provide-better-performance/61/5 "2021-11-09T21:04:03Z")
Following up on this, if I use `in-list` outside of a for-form, can the for-form still take advantage of improved performance? Or are bindings “opaque” to this sort of thing? That is, in my boggle post, I have something like ```auto (define (find-words-at board maximum-word-length c) (in-generator …)) (define (find-words board maximum-word-length) (apply in-sequences (map (curry find-words-at board maximum-word-length) (hash-keys board)))) (for ([word (find-words board 8)]) …) ``` I know the generator stuff isn’t the same, but are potential performance gains impeded by `define` here? ---
**Author:** ![samth](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/samth/32/3_2.png) [@samth](https://racket.discourse.group/u/samth)\ **Post date:** [November 9, 2021, 9:22pm UTC](https://racket.discourse.group/t/how-does-in-list-provide-better-performance/61/6 "2021-11-09T21:22:04Z")
No, the performance benefit is only when it’s used _directly_ in the `for` form. ---
**Author:** ![kablooey](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/kablooey/32/226_2.png) [@kablooey](https://racket.discourse.group/u/kablooey)\ **Post date:** [November 10, 2021, 4:56am UTC](https://racket.discourse.group/t/how-does-in-list-provide-better-performance/61/7 "2021-11-10T04:56:39Z")
So if in the example above, `find-words` directly evaluated to `(in-generator ...)`, then we would get the performance boost, right? ---
**Author:** ![samth](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/samth/32/3_2.png) [@samth](https://racket.discourse.group/u/samth)\ **Post date:** [November 10, 2021, 2:31pm UTC](https://racket.discourse.group/t/how-does-in-list-provide-better-performance/61/8 "2021-11-10T14:31:38Z")
No, it’s about being syntactically inside the for form, not about what it evaluates to. ---
**Author:** ![soegaard](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/soegaard/32/19_2.png) [@soegaard](https://racket.discourse.group/u/soegaard)\ **Post date:** [November 10, 2021, 3:33pm UTC](https://racket.discourse.group/t/how-does-in-list-provide-better-performance/61/9 "2021-11-10T15:33:21Z")
If you wonder about the inner workings of `for` then the `:do-in` reveals how `for` works inside. Inside a `for` form `:do-in` will expand into a loop of the form: ![image](https://global.discourse-cdn.com/free1/uploads/racket/original/1X/37e20dc284dfe2602e2e1fcc2e7040c2686c243c.png) The loop is a typical named let loop. Consider how `(for ([x (in-list '(1 2 3))]) (displayln x))` could be written in this style. It becomes more or less: ![image](https://global.discourse-cdn.com/free1/uploads/racket/original/1X/725b97a41b3c9a6028a8c0821d5c1f28edc8be34.jpeg) The form `:do-in` is used like this: ```auto (:do-in ([(outer-id ...) outer-expr] ...) outer-check ([loop-id loop-expr] ...) pos-guard ([(inner-id ...) inner-expr] ...) pre-guard post-guard (loop-arg ...)) ``` This tells us how to implement `in-list` using `:do-in`: ![image](https://global.discourse-cdn.com/free1/uploads/racket/original/1X/82ff69df4c07302c658018941db75e9e19e639ff.jpeg) Consider now the case ```auto (for ([x (in-list '(1 2 3))] [y (in-list '(a b c))]) (list x y)) ``` This is a parallel loop, where x and y runs over the lists `'(1 2 3)` and `'(a b c)` in parallel. The `for` form must therefore find the pieces from the two loops and fuse them together. E.g. combine `([xs initial-xs])` and `([ys initials-ys])` into `([xs initial-xs] [ys initials-ys])` . This process explains why `for` needs to see the `in-list` in the rhs of the clause. If at compile time, the `in-list` is present `for` can generate the various pieces needed to loop through the elements and put these pieces together with pieces from other iterations. If on the other hand the `in-list` is missing, then it can’t do anything better than output something that at runtime must check whether it needs to loop through a list, a vector, a string or similar. [Documentation for :do-in](https://docs.racket-lang.org/reference/for.html#%28form._%28%28lib._racket%2Fprivate%2Fbase..rkt%29._~3ado-in%29%29)