How to measure an object size accurate

I make a draft macro called size which takes an object and measures the memory use:

(define-syntax (size stx)
  (syntax-parse stx
    [(_ obj)
     #'(let ([before-use (current-memory-use)])
         obj
         (define after-use (current-memory-use))
         (printf "size: ~a bytes~n" (- after-use before-use)))]))

(size (thread (λ () (let loop () (thread-receive) (loop)))))
(size 1)
(size 2)

Above program prints:

size: 8672 bytes
size: 32 bytes
size: 32 bytes

Does there any way to make it more accurate?

1 Like

First naive suggestion: call (collect-garbage) three times before measuring memory use. (The 3x may be a relic from the deep past, no idea if this is still the right heuristic.)

2 Likes

Ooh... except that if the value doesn't survive, that call could actually collect it. So I would return it after checking the memory use, to make sure it doesn't die. Even then, you run the risk of optimizers cleverly discovering that it's not actually needed.

but if you bind it to a top-level identifier, I would imagine that it definitely wouldn't get collected.

With 8.2 CS, in DrRacket I got

size: 20112 bytes
size: 0 bytes
size: 0 bytes

My guess is 32 is the size of the temporal variables,. fixnums like 1 or 2 don't allocate memory neither in CS or BC.

I made another version with void/reference-sink because otherwise the compiler may notice that it's unused and just avoid allocating it. (And there are a few more optimization that may remove the reference before you expect it.)

For example try

 (size (list 0 1 2 3 4 5 6 7 8 9))

Also, I added (sleep 1). I never heard it's necessary but trying the program a few times I get more consistent results with a small pause (???).

With my version I get size that is like x10 the number I get with your version.

size: 182768 bytes
size: 0 bytes
size: 0 bytes

I'm not sure it's correct, but my version is:

#lang racket
(require (for-syntax syntax/parse))
(require ffi/unsafe)

(define-syntax (size stx)
  (syntax-parse stx
    [(_ obj)
     #'(let ()
         (sleep 1)
         (collect-garbage)
         (collect-garbage)
         (collect-garbage)
         (define before-use (current-memory-use))
         (define thing obj)
         (define after-use (current-memory-use))
         (void/reference-sink thing)
         (printf "size: ~a bytes~n" (- after-use before-use)))]))

(size (thread (λ () (let loop () (thread-receive) (loop)))))
(size 1)
(size 2)
2 Likes

This sounds make sense, what I actually would like to know the representation size in machine of any value, but seems it only count the allocation

You're pointing out that this doesn't measure the size of other existing values that are linked to from this one? So, for instance, in

(define a (list 3 4 5 6))
(define b (cons 44 a))

You'd like the size of 'b' to include the size of 'a'?

I don't see an easy way to compute this. I'm also not sure how useful it is, because it may include large values that are not collectible.

I guess I should ask: how would you like to use this information?

1 Like

My original scenario is I have many long-live threads, they will exchange messages to complete different jobs, and each has its own local storage. I want to measure the accurate memory usage so that I can know the limitation of the design, to know whether I should add something into the thread or I should put them somewhere else.

You may be able to use current-memory-use with a custodian in that case. For example:

#lang racket/base

(define cust (make-custodian))
(define thd
  (parameterize ([current-custodian cust])
    (thread
     (lambda ()
       (let loop ([data null])
         (loop (cons (thread-receive) data)))))))

(collect-garbage)
(define old (current-memory-use cust))
(define (display-delta)
  (collect-garbage)
  (define new (current-memory-use cust))
  (define delta (- new old))
  (set! old new)
  (printf "delta: ~a~a old: ~a new: ~a~n" (if (> delta 0) "+" "") delta old new))

(display-delta)

(thread-send thd (vector 1 2 3))
(display-delta)
(thread-send thd (make-bytes 10240))
(display-delta)
(thread-send thd (make-bytes 10240))
(display-delta)

(define unreachable (make-bytes 10240))
(display-delta)

(thread-send thd (make-bytes 10240))
(display-delta)

The above prints

delta: -32 old: 3472 new: 3472
delta: +104 old: 3576 new: 3576
delta: +10272 old: 13848 new: 13848
delta: +10272 old: 24120 new: 24120
delta: 0 old: 24120 new: 24120
delta: +10272 old: 34392 new: 34392

on my machine with a recent build of Racket CS.

1 Like