python313Packages.tokenizers 0.23.3 single
Fast State-of-the-Art Tokenizers optimized for Research and Production
- Name
python3.13-tokenizers-0.23.3- Package set
- python313Packages
- Homepage
- https://github.com/huggingface/tokenizers
- License
- Apache-2.0
- Source
- pkgs/development/python-modules/tokenizers/default.nix:148
- Derivation
/nix/store/kspfc23v8mhybrk1a7a0d36qss8hg15r-python3.13-tokenizers-0.23.3.drv- Repology
- not reported as outdated · look up (as of 2026-11-10T01:00:17Z)
Maintainers 1
| Handle | Name | Contact | Via |
|---|---|---|---|
| GaetanLepage | Gaetan Lepage | GitHub · gaetan@glepage.com | listed on package |
Dependencies 29
| Package | As | Maintainers | Used by (transitive) | Status |
|---|---|---|---|---|
| python313 3.13.18 | propagatedBuildInputs | 14,168 | team-only | |
| python313Packages.huggingface-hub 2.4.0 | propagatedBuildInputs | 175 | maintained | |
| oniguruma 6.9.10 | buildInputs | 4,279 | single | |
| openssl 3.5.9 | buildInputs | 76,574 | maintained | |
| cargo 1.99.1 | nativeBuildInputs | 29,289 | team-only | |
| ensureNewerSourcesForZipFilesHook 26.05pre-git | nativeBuildInputs | 74,250 | unmaintained setup hook | |
| pkg-config 0.29.2 | nativeBuildInputs | 77,688 | unmaintained | |
| python313Packages.datasets 5.1.0 | nativeBuildInputs | 149 | single outdated | |
| python313Packages.numpy 2.5.4 | nativeBuildInputs | 5,321 | single | |
| python313Packages.pypaBuildHook 26.05pre-git | nativeBuildInputs | 11,689 | unmaintained setup hook | |
| python313Packages.pypaInstallHook 26.05pre-git | nativeBuildInputs | 11,717 | unmaintained setup hook | |
| python313Packages.pytest-asyncio 1.4.1 | nativeBuildInputs | 8,226 | single | |
| python313Packages.pytestCheckHook 26.05pre-git | nativeBuildInputs | 10,197 | unmaintained setup hook | |
| python313Packages.pythonCatchConflictsHook 26.05pre-git | nativeBuildInputs | 11,701 | unmaintained setup hook | |
| python313Packages.pythonImportsCheckHook 26.05pre-git | nativeBuildInputs | 11,734 | unmaintained setup hook | |
| python313Packages.pythonMetadataCheckHook 26.05pre-git | nativeBuildInputs | 11,689 | single setup hook | |
| python313Packages.pythonNamespacesHook 26.05pre-git | nativeBuildInputs | 11,726 | unmaintained setup hook | |
| python313Packages.pythonOutputDistHook 26.05pre-git | nativeBuildInputs | 74,218 | unmaintained setup hook | |
| python313Packages.pythonRemoveBinBytecodeHook 26.05pre-git | nativeBuildInputs | 74,238 | unmaintained setup hook | |
| python313Packages.pythonRemoveTestsDirHook 26.05pre-git | nativeBuildInputs | 11,726 | unmaintained setup hook | |
| python313Packages.pythonRuntimeDepsCheckHook 26.05pre-git | nativeBuildInputs | 11,703 | unmaintained setup hook | |
| python313Packages.requests 2.34.2 | nativeBuildInputs | 6,780 | single | |
| python313Packages.setuptools-rust 1.13.0 | nativeBuildInputs | 5,174 | unmaintained | |
| python313Packages.tiktoken 0.12.0 | nativeBuildInputs | 162 | single outdated | |
| python313Packages.wrapPython 26.05pre-git | nativeBuildInputs | 11,727 | unmaintained setup hook | |
| rustc 1.99.1 | nativeBuildInputs | 32,538 | team-only | |
| writableTmpDirAsHomeHook 26.05pre-git | nativeBuildInputs | 64,758 | unmaintained setup hook | |
| bashNonInteractive 5.3p20 | other input | 102,212 | maintained | |
| stdenv 26.05pre-git | other input | 90,021 | team-only |
Used by 23 directly, 138 transitively
| Package | As | Maintainers | Used by (transitive) | Status |
|---|---|---|---|---|
| python313Packages.transformers 5.22.1 | propagatedBuildInputs | 90 | maintained outdated | |
| python313Packages.litellm 1.108.0 | propagatedBuildInputs | 38 | maintained | |
| python313Packages.cohere 7.0.4 | propagatedBuildInputs | 8 | single outdated | |
| python313Packages.sentence-transformers 6.2.0 | propagatedBuildInputs | 8 | unmaintained | |
| python313Packages.faster-whisper 1.2.1 | propagatedBuildInputs | 6 | single | |
| python313Packages.chromadb 1.5.9 | propagatedBuildInputs | 3 | maintained | |
| python313Packages.vllm 0.33.0 | propagatedBuildInputs | 3 | maintained | |
| python313Packages.curated-transformers 2.0.1 | propagatedBuildInputs | 2 | single | |
| python313Packages.pytorch-tokenizers 1.5.0 | propagatedBuildInputs | 1 | single outdated | |
| python313Packages.unclecode-litellm 1.81.13 | propagatedBuildInputs | 1 | single | |
| python313Packages.crawl4ai 0.9.1 | nativeBuildInputs | 0 | single outdated | |
| python313Packages.crewai 1.15.18 | propagatedBuildInputs | 0 | single outdated | |
| python313Packages.exllamav3 1.6.6 | propagatedBuildInputs | 0 | single broken | |
| python313Packages.fastembed 0.8.0 | propagatedBuildInputs | 0 | single outdated | |
| python313Packages.guidance 0.3.1 | nativeBuildInputs | 0 | single | |
| python313Packages.langchain-huggingface 1.2.3 | propagatedBuildInputs | 0 | maintained | |
| python313Packages.langchain-mistralai 1.1.6 | propagatedBuildInputs | 0 | single | |
| python313Packages.mflux 0.15.4 | propagatedBuildInputs | 0 | single outdated | |
| python313Packages.pocket-tts 4.0.1 | propagatedBuildInputs | 0 | single | |
| python313Packages.pyopenjtalk-plus 0.4.1-post9 | nativeBuildInputs | 0 | single outdated | |
| python313Packages.torchtitan 0.3.0 | propagatedBuildInputs | 0 | single outdated | |
| python313Packages.transformers_4 4.57.6 | propagatedBuildInputs | 0 | maintained broken | |
| vllm 0.33.0 | propagatedBuildInputs | 0 | maintained |