Llama Stop Tokens: end_of_text vs eot
If EOS (<|end_of_text|>) and EOT (<|eot|> or <|eot_id|>) tokens aren't handled properly during tokenization and inference, Llama models can keep hallucinating or repeating text indefinitely because they never receive the signal to stop talking.
Tokenizer & Generation Config: Enforce stop tokens ["<|eot|>", "<|end_of_text|>"] on Llama 4 (and ["<|eot_id|>", "<|end_of_text|>"] on Llama 3.3) across all custom inference pipelines.Nobody has confirmed this prompt yet. If you try it, tell others whether it worked.
No comments yet.