x86_64-linux · 151fa4e8ddfd · generated 2026-10-07T10:25:31Z

python313Packages.pytorch-tokenizers 1.5.0 single

C++ implementations for various tokenizers (sentencepiece, tiktoken, etc.)

Name
python3.13-pytorch-tokenizers-1.5.0
Package set
python313Packages
Homepage
https://github.com/meta-pytorch/tokenizers
License
BSD-3-Clause
Source
pkgs/development/python-modules/pytorch-tokenizers/default.nix:103
Derivation
/nix/store/q3yxqb3b2p5nm76s6fcn4jjmxpkpgyli-python3.13-pytorch-tokenizers-1.5.0.drv
Repology
not reported as outdated · look up (as of 2026-10-07T10:23:38Z)

Maintainers 1

HandleNameContactVia
GaetanLepage Gaetan Lepage GitHub · gaetan@glepage.com listed on package

Dependencies 23

PackageAsMaintainersUsed by (transitive)Status
python313 3.13.15 propagatedBuildInputs
mweinelt
 13,660 team-only outdated
python313Packages.sentencepiece 0.2.1 propagatedBuildInputs
pashashocky
 46 single outdated
python313Packages.tiktoken 0.12.0 propagatedBuildInputs
happysalada
 161 single outdated
python313Packages.tokenizers 0.23.2 propagatedBuildInputs
GaetanLepage
 137 single
ensureNewerSourcesForZipFilesHook 26.05pre-git nativeBuildInputs
74,596 unmaintained setup hook
python313Packages.cmake 4.4.3 nativeBuildInputs
5,231 unmaintained outdated
python313Packages.pybind11 3.0.4 nativeBuildInputs
dotlambda, yuriaisaka
 4,642 maintained outdated
python313Packages.pypaBuildHook 26.05pre-git nativeBuildInputs
11,598 unmaintained setup hook
python313Packages.pypaInstallHook 26.05pre-git nativeBuildInputs
11,625 unmaintained setup hook
python313Packages.pytestCheckHook 26.05pre-git nativeBuildInputs
10,182 unmaintained setup hook
python313Packages.pythonCatchConflictsHook 26.05pre-git nativeBuildInputs
11,609 unmaintained setup hook
python313Packages.pythonImportsCheckHook 26.05pre-git nativeBuildInputs
11,642 unmaintained setup hook
python313Packages.pythonMetadataCheckHook 26.05pre-git nativeBuildInputs
dotlambda
 11,597 single setup hook
python313Packages.pythonNamespacesHook 26.05pre-git nativeBuildInputs
11,634 unmaintained setup hook
python313Packages.pythonOutputDistHook 26.05pre-git nativeBuildInputs
74,564 unmaintained setup hook
python313Packages.pythonRemoveBinBytecodeHook 26.05pre-git nativeBuildInputs
74,584 unmaintained setup hook
python313Packages.pythonRemoveTestsDirHook 26.05pre-git nativeBuildInputs
11,634 unmaintained setup hook
python313Packages.pythonRuntimeDepsCheckHook 26.05pre-git nativeBuildInputs
11,611 unmaintained setup hook
python313Packages.setuptools 83.0.0 nativeBuildInputs
mweinelt
 11,500 team-only outdated
python313Packages.transformers 5.17.0 nativeBuildInputs
GaetanLepage, happysalada, pashashocky
 89 maintained outdated
python313Packages.wrapPython 26.05pre-git nativeBuildInputs
11,634 unmaintained setup hook
bashNonInteractive 5.3p15 other input
andir, balsoft, infinisil, pyrox0
 102,441 maintained outdated
stdenv 26.05pre-git other input
90,330 team-only

Used by 1 directly, 1 transitively

PackageAsMaintainersUsed by (transitive)Status
python313Packages.executorch 1.5.0 propagatedBuildInputs
GaetanLepage
 0 single outdated