tokenizers BpeBuilder Buffer Overflow via merge token
Summary
| CVE | CVE-2026-85670 |
|---|---|
| State | PUBLISHED |
| Assigner | VulnCheck |
| Source Priority | CVE Program / NVD first with legacy fallback |
| Published | 2026-09-04 15:17:44 UTC |
| Updated | 2026-09-04 18:18:06 UTC |
| Description | tokenizers (Hugging Face) is affected by an out-of-bounds buffer access in BpeBuilder::build (tokenizers/src/models/bpe/model.rs). When loading a tokenizer.json via Tokenizer::from_file/from_str, the builder sizes a scratch buffer to the longest vocabulary key, then writes each concatenated merge rule into it. A merge whose concatenated token exceeds the longest vocabulary key overruns the buffer, which Rust turns into a panic that aborts the process in Rust and FFI embeddings. This occurs at load time with no encoding required, so an attacker who supplies a crafted tokenizer.json can cause a denial of service. A secondary defect at the same location can cause a usize underflow (panic in debug, potential memory corruption in release) when continuing_subword_prefix is set and a merge token is shorter than the prefix. Observed in version 0.23.1. |
Risk And Classification
Primary CVSS: v4.0 7.1 HIGH from [email protected]
CVSS:4.0/AV:N/AC:L/AT:N/PR:N/UI:P/VC:N/VI:N/VA:H/SC:N/SI:N/SA:N/E:X/CR:X/IR:X/AR:X/MAV:X/MAC:X/MAT:X/MPR:X/MUI:X/MVC:X/MVI:X/MVA:X/MSC:X/MSI:X/MSA:X/S:X/AU:X/R:X/V:X/RE:X/U:X
Problem Types: CWE-787 | CWE-787 Out-of-bounds Write
| Version | Source | Type | Score | Severity | Vector |
|---|---|---|---|---|---|
| 4.0 | [email protected] | Secondary | 7.1 | HIGH | CVSS:4.0/AV:N/AC:L/AT:N/PR:N/UI:P/VC:N/VI:N/VA:H/SC:N/SI:N/SA:N/E:X/C... |
| 4.0 | CNA | CVSS | 7.1 | HIGH | CVSS:4.0/AV:N/AC:L/AT:N/PR:N/UI:P/VC:N/VI:N/VA:H/SC:N/SI:N/SA:N |
| 3.1 | [email protected] | Secondary | 6.5 | MEDIUM | CVSS:3.1/AV:N/AC:L/PR:N/UI:R/S:U/C:N/I:N/A:H |
| 3.1 | CNA | CVSS | 6.5 | MEDIUM | CVSS:3.1/AV:N/AC:L/PR:N/UI:R/S:U/C:N/I:N/A:H |
CVSS v4.0 Breakdown
Attack Vector
NetworkAttack Complexity
LowAttack Requirements
NonePrivileges Required
NoneUser Interaction
PassiveConfidentiality
NoneIntegrity
NoneAvailability
HighSub Conf.
NoneSub Integrity
NoneSub Availability
NoneCVSS:4.0/AV:N/AC:L/AT:N/PR:N/UI:P/VC:N/VI:N/VA:H/SC:N/SI:N/SA:N/E:X/CR:X/IR:X/AR:X/MAV:X/MAC:X/MAT:X/MPR:X/MUI:X/MVC:X/MVI:X/MVA:X/MSC:X/MSI:X/MSA:X/S:X/AU:X/R:X/V:X/RE:X/U:X
CVSS v3.1 Breakdown
Attack Vector
NetworkAttack Complexity
LowPrivileges Required
NoneUser Interaction
RequiredScope
UnchangedConfidentiality
NoneIntegrity
NoneAvailability
HighCVSS:3.1/AV:N/AC:L/PR:N/UI:R/S:U/C:N/I:N/A:H
Vendor Declared Affected Products
| Source | Vendor | Product | Version | Platforms |
|---|---|---|---|---|
| CNA | Huggingface | Tokenizers | affected 0.23.2 semver | Not specified |
References
| Reference | Source | Link | Tags |
|---|---|---|---|
| github.com/huggingface/tokenizers | [email protected] | github.com | |
| github.com/huggingface/tokenizers/issues/2094 | 134c704f-9b21-4f2e-91b3-4a467353bcc0 | github.com | |
| github.com/huggingface/tokenizers/blob/v0.23.2/tokenizers/src/models/bpe... | [email protected] | github.com | |
| www.vulncheck.com/advisories/tokenizers-bpebuilder-buffer-overflow-via-merge-token | [email protected] | www.vulncheck.com | |
| CVE Program record | CVE.ORG | www.cve.org | canonical |
| NVD vulnerability detail | NVD | nvd.nist.gov | canonical, analysis |
Vendor Comments And Credit
Discovery Credit
CNA: George Chen (en)
There are currently no legacy QID mappings associated with this CVE.