Metadata-Version: 2.1
Name: mult-tokenizer
Version: 0.1.37
Summary: Multigramm Tokenization package
Home-page: https://github.com/OksanichenkoFedor/MultigrammTokenizer
Author: Oksanichenko Fedor
Author-email: okssolotheodor@gmail.com
License: MIT
Platform: UNKNOWN
Requires-Dist: numpy (>=1.21.5)
Requires-Dist: tqdm (>=4.64.0)

A Python implementation of ngram-tokenization algorithm.

