python313Packages.tokenizers 0.23.2 single
Fast State-of-the-Art Tokenizers optimized for Research and Production
- Name
python3.13-tokenizers-0.23.2- Package set
- python313Packages
- Homepage
- https://github.com/huggingface/tokenizers
- License
- Apache-2.0
- Source
- pkgs/development/python-modules/tokenizers/default.nix:147
- Derivation
/nix/store/0kdpgfqa6lq4456li1flfxx8iqxa47dv-python3.13-tokenizers-0.23.2.drv- Repology
- not reported as outdated · look up (as of 2026-10-07T10:23:38Z)
Maintainers 1
| Handle | Name | Contact | Via |
|---|---|---|---|
| GaetanLepage | Gaetan Lepage | GitHub · gaetan@glepage.com | listed on package |
Dependencies 28
| Package | As | Maintainers | Used by (transitive) | Status |
|---|---|---|---|---|
| python313 3.13.15 | propagatedBuildInputs | 13,660 | team-only outdated | |
| python313Packages.huggingface-hub 1.30.0 | propagatedBuildInputs | 173 | maintained outdated | |
| openssl 3.5.8 | buildInputs | 76,863 | maintained outdated | |
| cargo 1.98.1 | nativeBuildInputs | 28,718 | team-only | |
| ensureNewerSourcesForZipFilesHook 26.05pre-git | nativeBuildInputs | 74,596 | unmaintained setup hook | |
| pkg-config 0.29.2 | nativeBuildInputs | 78,016 | unmaintained | |
| python313Packages.datasets 4.5.0 | nativeBuildInputs | 148 | single outdated | |
| python313Packages.numpy 2.5.2 | nativeBuildInputs | 5,137 | single outdated | |
| python313Packages.pypaBuildHook 26.05pre-git | nativeBuildInputs | 11,598 | unmaintained setup hook | |
| python313Packages.pypaInstallHook 26.05pre-git | nativeBuildInputs | 11,625 | unmaintained setup hook | |
| python313Packages.pytest-asyncio 1.4.0 | nativeBuildInputs | 8,369 | single | |
| python313Packages.pytestCheckHook 26.05pre-git | nativeBuildInputs | 10,182 | unmaintained setup hook | |
| python313Packages.pythonCatchConflictsHook 26.05pre-git | nativeBuildInputs | 11,609 | unmaintained setup hook | |
| python313Packages.pythonImportsCheckHook 26.05pre-git | nativeBuildInputs | 11,642 | unmaintained setup hook | |
| python313Packages.pythonMetadataCheckHook 26.05pre-git | nativeBuildInputs | 11,597 | single setup hook | |
| python313Packages.pythonNamespacesHook 26.05pre-git | nativeBuildInputs | 11,634 | unmaintained setup hook | |
| python313Packages.pythonOutputDistHook 26.05pre-git | nativeBuildInputs | 74,564 | unmaintained setup hook | |
| python313Packages.pythonRemoveBinBytecodeHook 26.05pre-git | nativeBuildInputs | 74,584 | unmaintained setup hook | |
| python313Packages.pythonRemoveTestsDirHook 26.05pre-git | nativeBuildInputs | 11,634 | unmaintained setup hook | |
| python313Packages.pythonRuntimeDepsCheckHook 26.05pre-git | nativeBuildInputs | 11,611 | unmaintained setup hook | |
| python313Packages.requests 2.34.2 | nativeBuildInputs | 6,686 | single | |
| python313Packages.setuptools-rust 1.13.0 | nativeBuildInputs | 5,058 | unmaintained | |
| python313Packages.tiktoken 0.12.0 | nativeBuildInputs | 161 | single outdated | |
| python313Packages.wrapPython 26.05pre-git | nativeBuildInputs | 11,634 | unmaintained setup hook | |
| rustc 1.98.1 | nativeBuildInputs | 30,080 | team-only outdated | |
| writableTmpDirAsHomeHook 26.05pre-git | nativeBuildInputs | 66,113 | unmaintained setup hook | |
| bashNonInteractive 5.3p15 | other input | 102,441 | maintained outdated | |
| stdenv 26.05pre-git | other input | 90,330 | team-only |
Used by 23 directly, 137 transitively
| Package | As | Maintainers | Used by (transitive) | Status |
|---|---|---|---|---|
| python313Packages.transformers 5.17.0 | propagatedBuildInputs | 89 | maintained outdated | |
| python313Packages.litellm 1.102.1 | propagatedBuildInputs | 38 | single outdated | |
| python313Packages.cohere 7.0.4 | propagatedBuildInputs | 8 | single outdated | |
| python313Packages.sentence-transformers 6.1.0 | propagatedBuildInputs | 8 | unmaintained | |
| python313Packages.faster-whisper 1.2.1 | propagatedBuildInputs | 6 | single | |
| python313Packages.chromadb 1.5.9 | propagatedBuildInputs | 3 | maintained | |
| python313Packages.curated-transformers 2.0.1 | propagatedBuildInputs | 2 | single | |
| python313Packages.vllm 0.24.0 | propagatedBuildInputs | 2 | maintained | |
| python313Packages.pytorch-tokenizers 1.5.0 | propagatedBuildInputs | 1 | single | |
| python313Packages.unclecode-litellm 1.81.13 | propagatedBuildInputs | 1 | single | |
| python313Packages.crawl4ai 0.9.1 | nativeBuildInputs | 0 | single outdated | |
| python313Packages.crewai 1.15.18 | propagatedBuildInputs | 0 | single outdated | |
| python313Packages.exllamav3 1.5.4 | propagatedBuildInputs | 0 | single broken | |
| python313Packages.fastembed 0.8.0 | propagatedBuildInputs | 0 | single outdated | |
| python313Packages.guidance 0.3.1 | nativeBuildInputs | 0 | single | |
| python313Packages.langchain-huggingface 1.2.2 | propagatedBuildInputs | 0 | maintained | |
| python313Packages.langchain-mistralai 1.1.6 | propagatedBuildInputs | 0 | single | |
| python313Packages.mflux 0.15.4 | propagatedBuildInputs | 0 | single outdated | |
| python313Packages.pocket-tts 3.3.0 | propagatedBuildInputs | 0 | single | |
| python313Packages.pyopenjtalk-plus 0.4.1-post9 | nativeBuildInputs | 0 | single | |
| python313Packages.torchtitan 0.3.0 | propagatedBuildInputs | 0 | single | |
| python313Packages.transformers_4 4.57.6 | propagatedBuildInputs | 0 | maintained broken | |
| vllm 0.24.0 | propagatedBuildInputs | 0 | maintained |