You could definitely propose something like that.
I notice there's an out of date GitHub issue about this too: Typed Racket pretends mutable sets don't exist, and yet it assumes all hashtables could be mutable · Issue #32 · racket/typed-racket · GitHub