Skip to content

text-splitters[minor]: Ability to Passing a Tokenizer directly to TokenTextSplitter and updating from_tiktoken_encoder, from_huggingface_tokenizer #31428

text-splitters[minor]: Ability to Passing a Tokenizer directly to TokenTextSplitter and updating from_tiktoken_encoder, from_huggingface_tokenizer

text-splitters[minor]: Ability to Passing a Tokenizer directly to TokenTextSplitter and updating from_tiktoken_encoder, from_huggingface_tokenizer #31428

cd libs/text-splitters  /  poetry run pytest -m compile tests/integration_tests #3.12

succeeded Oct 10, 2024 in 14s