# How to hilight non-racket code with Scribble? **URL:** \ **Category:** General\ **Tags:** scribble\ **Created:** [March 24, 2024, 9:52pm UTC](https://racket.discourse.group/t/how-to-hilight-non-racket-code-with-scribble/2832 "2024-03-24T21:52:07Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![janus](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/janus/32/1115_2.png) [@janus](https://racket.discourse.group/u/janus)\ **Post date:** [March 24, 2024, 9:52pm UTC](https://racket.discourse.group/t/how-to-hilight-non-racket-code-with-scribble/2832/1 "2024-03-24T21:52:07Z")
As a continuation to [this StackOverflow question](https://stackoverflow.com/questions/74394913/include-non-racket-source-in-scribble-files), I now want to hilight Idris2 code using Scribble. I like Scribble for the styling and the programmability in general, but the code of my documentation is not Scheme/Racket. So I've modified Katla, which is a Idris code formatter, to output Scribble code with Racket syntax. But my issue is that I think the code is really weird, and I had to do a bunch of hacks. My generated code looks like this. First, a preamble where I map some Idris syntactic concepts to some racket colors. These mappings make no sense, I have based it on whether the colors looked nice. ```scheme #lang racket (require scribble/decode scribble/core scribble/manual "katla.scr") (define IdrisData racketidfont) (define IdrisType racketresultfont) (define IdrisBound racketparenfont) (define IdrisFunction racketkeywordfont) (define IdrisKeyword racketoutput) ``` Then, named generated code snippets are emitted: ```scheme (provide 8main) (define 8main (let [(lines (list (list IdrisKeyword "partial") 'nl (list IdrisFunction "main") " " (list IdrisKeyword ":") " " (list IdrisType "IO") " " (list IdrisType "Unit") 'nl (list IdrisFunction "main") " " (list IdrisKeyword "=") " " (list IdrisFunction "H2.runHttp2WithPostgres") " " (list IdrisFunction "okReplyAndBody2") 'nl ))] (if (empty? lines) (error "code block is empty") (make-nested-flow (make-style 'code-inset '(box-mode)) (decode-flow (list (apply verbatim (append-map wp lines)))) ) ))) ``` which is the highlighted version of ```scheme partial main : IO Unit main = H2.runHttp2WithPostgres okReplyAndBody2 ``` That function can then be used in a regular Scribble document. My issue is that all this `make-nested-flow` and `decode-flow` stuff is derived my trial-and-error, and I don't understand it at all. Isn't there a nicer way to tell Scribble to just include some monospaced code that I have hilighted myself? I understand that Scribble is designed first and foremost for Scheme, but I thought it would make sense if there is a lower-level API I could use. What I am currently doing technically works, but I haven't shown the worst part, which is moving around white space because it seems that Scribble doesn't like highlighted white space. ---
**Author:** ![soegaard](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/soegaard/32/19_2.png) [@soegaard](https://racket.discourse.group/u/soegaard)\ **Post date:** [March 24, 2024, 10:45pm UTC](https://racket.discourse.group/t/how-to-hilight-non-racket-code-with-scribble/2832/2 "2024-03-24T22:45:14Z")
After looking at the source code for `codeblock` I have an idea for you. The way `codeblock` works is: 1. Use `get-tokens` to get a list of tokens. 2. Call `typeset-code`. So I think, you can do this: 1. Make Katla produce tokens. That is, convert the file into a file containing s-expressions of the form: `(list T natural natural natural)` 2. Implement an `idris2-get-tokens` in Racket (that reads the tokens from a file). 3. Implement `idris2-codeblock` by calling `idris2-get-tokens` followed by a call to `typeset-code`. Explanation of tokens and `get-tokens`: > The source for `codeblock` and `typeset-code`: > ---
**Author:** ![janus](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/janus/32/1115_2.png) [@janus](https://racket.discourse.group/u/janus)\ **Post date:** [March 26, 2024, 10:00am UTC](https://racket.discourse.group/t/how-to-hilight-non-racket-code-with-scribble/2832/3 "2024-03-26T10:00:26Z")
This looks promising, but `typeset-code` calls `get-tokens` on line 64, which makes sense since it takes a string, not a list of tokens. Maybe I can make a patch for `manual-code.rkt` that extracts the call to `get-tokens` to the callers of `typeset-code`... ---
**Author:** ![LiberalArtist](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/liberalartist/32/151_2.png) [@LiberalArtist](https://racket.discourse.group/u/LiberalArtist)\ **Post date:** [March 30, 2024, 3:35am UTC](https://racket.discourse.group/t/how-to-hilight-non-racket-code-with-scribble/2832/4 "2024-03-30T03:35:34Z")
One approach would be to make a stub `#lang idris2-for-scribble`, give it a [color lexer](https://docs.racket-lang.org/tools/lang-languages-customization.html#%28part._.Syntax_.Coloring%29) that calls Katla, and then use it from Scribble via something like: ```scheme (define (idrisblock . strs) (apply typeset-code #:keep-lang-line? #f "#lang idris2-for-scribble\n" strs)) ``` The color lexer protocol is a bit awkward, but, especially if you don't care about interactive use, extending this advice to parse the whole file at once might make it a lot easier: > [@Color lexer backup distances and \`dont-stop\`](https://racket.discourse.group/t/color-lexer-backup-distances-and-dont-stop/2170/3): > > I think @greghendershott 's advice is good, tho. Processing an entire line at once and putting the processing result into the mode and then returning a sequence of tokens via `dont-stop` until the end of the line seems like a (relatively?) easy way to get something going. _Beautiful Racket_ has a good introduction to color lexers: [Beautiful Racket: Level up: jsonic revisited](https://beautifulracket.com/jsonic-2/syntax-coloring.html). Depending on how much effort you want to put in, your `#lang` could also take care of whatever running you want to do, or even make automatic identifier links work. The Zuo documentation is one example of how to make that work for a language not actually implemented in Racket: [zuo/zuo-doc at main · racket/zuo · GitHub](https://github.com/racket/zuo/tree/main/zuo-doc) (rendered at [Zuo: A Tiny Racket for Scripting](https://docs.racket-lang.org/zuo/)). Actually, Idris2 has a Racket backend, doesn't it? I haven't looked at its implementation, but that might put developing a real, working `#lang idris2` a relatively reasonable project to undertake.