user avatar
Tune tokenization/detokenization flow for speed-up
Jingning Han authored
This commit optimizes the tokenization and detokenization operational
flow for speed-up. It makes the coding process about 0.3% faster at
speed 0.

Change-Id: I28008df7482874e4b5f237f2d418ff82a249dd56
455f2de2