python314Packages.pytorch-tokenizers 1.5.0 single

C++ implementations for various tokenizers (sentencepiece, tiktoken, etc.)

Name: python3.14-pytorch-tokenizers-1.5.0
Package set: python314Packages
Homepage: https://github.com/meta-pytorch/tokenizers
License: BSD-3-Clause
Source: pkgs/development/python-modules/pytorch-tokenizers/default.nix:103
Derivation: /nix/store/qc4l5rz2c7mx0zsnp4xnhs88mbl2ni1l-python3.14-pytorch-tokenizers-1.5.0.drv
Repology: not reported as outdated · look up (as of 2026-10-07T10:23:38Z)

Maintainers 1

Handle	Name	Contact	Via
GaetanLepage	Gaetan Lepage	GitHub · gaetan@glepage.com	listed on package

Dependencies 23

Package	As	Maintainers	Used by (transitive)	Status
python3 3.14.7	propagatedBuildInputs	mweinelt	72,537	team-only outdated
python314Packages.sentencepiece 0.2.1	propagatedBuildInputs	pashashocky	56	single outdated
python314Packages.tiktoken 0.12.0	propagatedBuildInputs	happysalada	209	single outdated
python314Packages.tokenizers 0.23.2	propagatedBuildInputs	GaetanLepage	181	single
ensureNewerSourcesForZipFilesHook 26.05pre-git	nativeBuildInputs		74,596	unmaintained setup hook
python313Packages.pythonOutputDistHook 26.05pre-git	nativeBuildInputs		74,564	unmaintained setup hook
python313Packages.pythonRemoveBinBytecodeHook 26.05pre-git	nativeBuildInputs		74,584	unmaintained setup hook
python314Packages.cmake 4.4.3	nativeBuildInputs		12,371	unmaintained outdated
python314Packages.pybind11 3.0.4	nativeBuildInputs	dotlambda, yuriaisaka	10,851	maintained outdated
python314Packages.pypaBuildHook 26.05pre-git	nativeBuildInputs		71,473	unmaintained setup hook
python314Packages.pypaInstallHook 26.05pre-git	nativeBuildInputs		71,694	unmaintained setup hook
python314Packages.pytestCheckHook 26.05pre-git	nativeBuildInputs		70,075	unmaintained setup hook
python314Packages.pythonCatchConflictsHook 26.05pre-git	nativeBuildInputs		71,690	unmaintained setup hook
python314Packages.pythonImportsCheckHook 26.05pre-git	nativeBuildInputs		71,715	unmaintained setup hook
python314Packages.pythonMetadataCheckHook 26.05pre-git	nativeBuildInputs	dotlambda	71,460	single setup hook
python314Packages.pythonNamespacesHook 26.05pre-git	nativeBuildInputs		71,713	unmaintained setup hook
python314Packages.pythonRemoveTestsDirHook 26.05pre-git	nativeBuildInputs		71,713	unmaintained setup hook
python314Packages.pythonRuntimeDepsCheckHook 26.05pre-git	nativeBuildInputs		71,662	unmaintained setup hook
python314Packages.setuptools 83.0.0	nativeBuildInputs	mweinelt	71,381	team-only outdated
python314Packages.transformers 5.17.0	nativeBuildInputs	GaetanLepage, happysalada, pashashocky	114	maintained outdated
python314Packages.wrapPython 26.05pre-git	nativeBuildInputs		71,719	unmaintained setup hook
bashNonInteractive 5.3p15	other input	andir, balsoft, infinisil, pyrox0	102,441	maintained outdated
stdenv 26.05pre-git	other input		90,330	team-only

Used by 1 directly, 1 transitively

Package	As	Maintainers	Used by (transitive)	Status
python314Packages.executorch 1.5.0	propagatedBuildInputs	GaetanLepage	0	single outdated