Skip to main content

Detection Summary

The model’s tokenizer.json contains indicators that it has been modified. Multiple tampering techniques exist which can alter model behavior, depending on the tokenizer strategy employed. BPE tokenizers may exhibit signs of a broken merge chain. A valid BPE tokenizer is built by merging base tokens according to a merge list. Violations of merge chain integrity are an indicator of tampering. BPE or WordPiece tokenizers may include two or more tokens that share the same token ID. Tokenizers assume each ID maps to a single token. An attacker can add a malicious token with the same ID as an existing one, giving them control over what text the model outputs for that ID. The original token remains in the vocabulary to prevent the tokenizer from failing on load. The attacker’s token is then used when the model’s output is converted back to text. Unigram tokenizers are may also exhibit evidence of tampering including duplicate tokens or impossible values in the vocab list.

Security Impact

Token manipulation enables deterministic output substitution. The model generates token IDs with no awareness of the tampered vocabulary, and the attacker’s token is used when converting those IDs back to text. This can be used to redirect URLs through attacker-controlled proxies, inject tool calls, or substitute commands. In security-sensitive or policy-enforced contexts (e.g., safety classifiers, content filters, or access control), this can undermine the integrity of model outputs used for classification, filtering, or generation.

Risk Considerations

This finding in a published model indicates deliberate tampering with the tokenizer artifact. It is a supply chain integrity concern. Consumers of the model should treat duplicate token IDs as a security issue, especially where the model is used for classification, filtering, or generation in high-assurance or policy-enforced contexts. Some lower-severity findings, such as orphaned tokens, may be an acceptable risk absent any other evidence of manipulation.
  • Restore the tokenizer: Remove any entry that does not belong to the original model’s vocabulary. The original token must be preserved to maintain valid merge rules. If the original tokenizer file is available from the base model’s source, replace the tampered file entirely.
  • Validate after changes: After restoring the tokenizer, verify that tokenization and decoding produce expected results, including round-trip checks.
  • Inspect the full model: Tokenizer tampering is a supply chain attack and may indicate that other model artifacts have also been compromised. Review other artifacts, including weights and configuration files, for signs of further tampering.
General mitigations:
  • Use defense-in-depth when deploying the model.
  • Include tokenizer integrity checks in red-team or robustness testing.