staging · x86_64-linux · 8e1af4ab17b2 · generated 2026-11-10T01:02:05Z

python313Packages.pytorch-tokenizers 1.5.0 single outdated

C++ implementations for various tokenizers (sentencepiece, tiktoken, etc.)

Name
python3.13-pytorch-tokenizers-1.5.0
Package set
python313Packages
Homepage
https://github.com/meta-pytorch/tokenizers
License
BSD-3-Clause
Source
pkgs/development/python-modules/pytorch-tokenizers/default.nix:102
Derivation
/nix/store/6qf2p3qyyyghzmi34d2g78is0izrqj29-python3.13-pytorch-tokenizers-1.5.0.drv
Repology
outdated, newest version is 1.6.0 · python:pytorch-tokenizers (as of 2026-11-10T01:00:17Z)

Maintainers 1

HandleNameContactVia
GaetanLepage Gaetan Lepage GitHub · gaetan@glepage.com listed on package

Dependencies 23

PackageAsMaintainersUsed by (transitive)Status
python313 3.13.18 propagatedBuildInputs 14,168 team-only
python313Packages.sentencepiece 0.2.1 propagatedBuildInputs 49 single outdated
python313Packages.tiktoken 0.12.0 propagatedBuildInputs 162 single outdated
python313Packages.tokenizers 0.23.3 propagatedBuildInputs 138 single
ensureNewerSourcesForZipFilesHook 26.05pre-git nativeBuildInputs
74,250 unmaintained setup hook
python313Packages.cmake 4.4.4 nativeBuildInputs
5,870 unmaintained
python313Packages.pybind11 3.1.0 nativeBuildInputs 3,786 single
python313Packages.pypaBuildHook 26.05pre-git nativeBuildInputs
11,689 unmaintained setup hook
python313Packages.pypaInstallHook 26.05pre-git nativeBuildInputs
11,717 unmaintained setup hook
python313Packages.pytestCheckHook 26.05pre-git nativeBuildInputs
10,197 unmaintained setup hook
python313Packages.pythonCatchConflictsHook 26.05pre-git nativeBuildInputs
11,701 unmaintained setup hook
python313Packages.pythonImportsCheckHook 26.05pre-git nativeBuildInputs
11,734 unmaintained setup hook
python313Packages.pythonMetadataCheckHook 26.05pre-git nativeBuildInputs 11,689 single setup hook
python313Packages.pythonNamespacesHook 26.05pre-git nativeBuildInputs
11,726 unmaintained setup hook
python313Packages.pythonOutputDistHook 26.05pre-git nativeBuildInputs
74,218 unmaintained setup hook
python313Packages.pythonRemoveBinBytecodeHook 26.05pre-git nativeBuildInputs
74,238 unmaintained setup hook
python313Packages.pythonRemoveTestsDirHook 26.05pre-git nativeBuildInputs
11,726 unmaintained setup hook
python313Packages.pythonRuntimeDepsCheckHook 26.05pre-git nativeBuildInputs
11,703 unmaintained setup hook
python313Packages.setuptools 83.0.0 nativeBuildInputs 11,547 team-only outdated
python313Packages.transformers 5.22.1 nativeBuildInputs 90 maintained outdated
python313Packages.wrapPython 26.05pre-git nativeBuildInputs
11,727 unmaintained setup hook
bashNonInteractive 5.3p20 other input 102,212 maintained
stdenv 26.05pre-git other input
90,021 team-only

Used by 1 directly, 1 transitively

PackageAsMaintainersUsed by (transitive)Status
python313Packages.executorch 1.5.1 propagatedBuildInputs 0 single outdated