Unable to use parser-tools/lex::lexer with in-port

Questions & Answers
1

Hi,

I may be missing something, but I think this fails when it should yield a list. This expression makes racket enter an infinite loop, due to an apparent conflict between in-port and lexer:

(require parser-tools/lex)
(for/list ([t (in-port
               (lexer [#\1 'one] [(eof) 'end])
               (open-input-string "1"))])
 t)

What do you think?

Thanks!

2

in-port never passes eof to the function. It's not obvious to me what you want to do, but in-producer might be what you're looking for.

1 Like
3

This works for me:

(require parser-tools/lex)
(for/list ([t (in-port
               (lexer [#\1 'one] [(eof) eof])
               (open-input-string "1"))])
  t)

The doc of in-port states that it stops when it encounters eof, so the solution is to produce an eof.

4

Thank you for this explanation. I didn't understand the necessity to pass eof. I am folding over a sequence of tokens and I had trouble doing that with the parser-tools/yacc parser, so I cooked up something of my own.