tokfit/bench
Throughput measurement.
time gleam run -m tokfit/bench
Not a test. gleam_stdlib 1.0.5 exposes no clock, so the timing is done
by the shell. This prints the two numbers worth diffing: the input size and
the token count. Divide the first by the wall clock to get bytes per
second.
The vocabulary is loaded once and reused, which is the point: the same run with a fresh handle per pass would measure the 199,998-entry parse instead of the tokenizing.