Parser-tools output position token's position only has offset?

Questions & Answers
1

I want to parse some s-expression with position information by parser-tools

(require parser-tools/lex
         (prefix-in : parser-tools/lex-sre))

(define-empty-tokens symbol
  (|(| |)|))
(define-tokens datum
  (ID
   STR
   NUM))
(define-empty-tokens end (EOF))

(define l
  (lexer-src-pos
   [(eof) (token-EOF)]
   ["(" (token-|(|)]
   [")" (token-|)|)]
   [(:+ alphabetic)
    (token-ID lexeme)]
   [(:seq "\"" (:* (:~ #\")) "\"")
    (token-STR lexeme)]
   [(:+ numeric)
    (token-NUM lexeme)]
   [whitespace (return-without-pos (l input-port))]))

But what I get from the following code only has offset in position, line and col are #f

(define f "test.ss")
(parameterize ([file-path f]
               [current-input-port (open-input-file f)])
  (define (next) (l (current-input-port)))
  (println (next)))
1 Like
2

You need to enable line counting with port-count-lines!.

https://docs.racket-lang.org/reference/linecol.html

2 Likes
3

oh wow, this is an unexpected place......

4

The documentation of lexer also mentions it, but it is easy to overlook since it's mentioned in the end
of a long paragraph.

image
image1406×188 30.5 KB

1 Like