Possible bug in parser-tools lexer-srcloc

Using both Drracket and Emacs racket-mode, the error highlighting is sometimes incorrect in my custom lang(#lang magic). I finally found that the issue is that a literal '\r' in the file's text leads to what appears to be incorrect positions in the srcloc's after that. This causes DrRacket to highlight the wrong text for the error. The srcloc line field is also incorrect but the column is fine.

Screenshot with correct highlighting:

and incorrect highlighting:

Notice that the adding the '\r' to the end of line 14 leads to an increase in the error srcloc position of 3 instead of 2.

Is the '\r' being converted to '\r\n' somehow? If anyone thinks this could be a Racket bug, let me know.

EDIT: I can reproduce this with any escape sequence like '\\' or '\b', but only if they occur at the end of a line. I'll need to check if this is something I'm doing in my reader, but I'm not sure how it might interact with srcloc. I'm using lexer-srcloc from brag/support.

EDIT2: After some more digging, I expect this issue resides somewhere in the parser-tools used by brag. Probably related to calculating the end-pos of a lexeme.

I've also determined that reproducing it may be a little more complicated than what I describe above. Not all lines will trigger the issue with a literal '\' 'r' appended to the end.

1 Like