How to hilight non-racket code with Scribble?

General
1

As a continuation to this StackOverflow question, I now want to hilight Idris2 code using Scribble. I like Scribble for the styling and the programmability in general, but the code of my documentation is not Scheme/Racket.

So I've modified Katla, which is a Idris code formatter, to output Scribble code with Racket syntax. But my issue is that I think the code is really weird, and I had to do a bunch of hacks.

My generated code looks like this. First, a preamble where I map some Idris syntactic concepts to some racket colors. These mappings make no sense, I have based it on whether the colors looked nice.

#lang racket                                                                        
(require scribble/decode scribble/core scribble/manual "katla.scr")                 
(define IdrisData       racketidfont)                                               
(define IdrisType       racketresultfont)                                           
(define IdrisBound      racketparenfont)                                            
(define IdrisFunction   racketkeywordfont)                                          
(define IdrisKeyword    racketoutput)

Then, named generated code snippets are emitted:

(provide 8main)                                                                      
(define 8main                                                                       
(let [(lines (list                                                                      
(list IdrisKeyword "partial")  'nl                                                  
(list IdrisFunction "main") " " (list IdrisKeyword ":") " " (list IdrisType "IO") " " (list IdrisType "Unit")  'nl    
(list IdrisFunction "main") " " (list IdrisKeyword "=") " " (list IdrisFunction "H2.runHttp2WithPostgres") " " (list IdrisFunction "okReplyAndBody2")  'nl
))]                                                                                 
(if (empty? lines)                                                                  
    (error "code block is empty")                                                       
    (make-nested-flow (make-style 'code-inset '(box-mode))                          
      (decode-flow (list (apply verbatim (append-map wp lines))))                       
    )                                                                               
)))

which is the highlighted version of

partial
main : IO Unit
main = H2.runHttp2WithPostgres okReplyAndBody2

That function can then be used in a regular Scribble document.

My issue is that all this make-nested-flow and decode-flow stuff is derived my trial-and-error, and I don't understand it at all. Isn't there a nicer way to tell Scribble to just include some monospaced code that I have hilighted myself? I understand that Scribble is designed first and foremost for Scheme, but I thought it would make sense if there is a lower-level API I could use. What I am currently doing technically works, but I haven't shown the worst part, which is moving around white space because it seems that Scribble doesn't like highlighted white space.

2 Likes
2

After looking at the source code for codeblock I have an idea for you.

The way codeblock works is:

  1. Use get-tokens to get a list of tokens.
  2. Call typeset-code.

So I think, you can do this:

  1. Make Katla produce tokens. That is, convert the file into a file containing s-expressions of the form: (list T natural natural natural)

  2. Implement an idris2-get-tokens in Racket (that reads the tokens from a file).

  3. Implement idris2-codeblock by calling idris2-get-tokens followed by a call to typeset-code.

Explanation of tokens and get-tokens:

The source for codeblock and typeset-code:

1 Like
3

This looks promising, but typeset-code calls get-tokens on line 64, which makes sense since it takes a string, not a list of tokens. Maybe I can make a patch for manual-code.rkt that extracts the call to get-tokens to the callers of typeset-code...

4

One approach would be to make a stub #lang idris2-for-scribble, give it a color lexer that calls Katla, and then use it from Scribble via something like:

(define (idrisblock . strs)
  (apply typeset-code
         #:keep-lang-line? #f
         "#lang idris2-for-scribble\n"
         strs))

The color lexer protocol is a bit awkward, but, especially if you don't care about interactive use, extending this advice to parse the whole file at once might make it a lot easier:

Beautiful Racket has a good introduction to color lexers: Beautiful Racket: Level up: jsonic revisited.

Depending on how much effort you want to put in, your #lang could also take care of whatever running you want to do, or even make automatic identifier links work. The Zuo documentation is one example of how to make that work for a language not actually implemented in Racket: zuo/zuo-doc at main · racket/zuo · GitHub (rendered at Zuo: A Tiny Racket for Scripting).

Actually, Idris2 has a Racket backend, doesn't it? I haven't looked at its implementation, but that might put developing a real, working #lang idris2 a relatively reasonable project to undertake.

1 Like