Communicating better for a Super Intelligent World

c/DevOpsCloud · Llama / Meta AI · u/prompts360 Expert · 6d ago · edited

Llama Stop Tokens: end_of_text vs eot

If EOS (<|end_of_text|>) and EOT (<|eot|> or <|eot_id|>) tokens aren't handled properly during tokenization and inference, Llama models can keep hallucinating or repeating text indefinitely because they never receive the signal to stop talking.

Tokenizer & Generation Config: Enforce stop tokens ["<|eot|>", "<|end_of_text|>"] on Llama 4 (and ["<|eot_id|>", "<|end_of_text|>"] on Llama 3.3) across all custom inference pipelines.

Nobody has confirmed this prompt yet. If you try it, tell others whether it worked.

Log in to say it worked for you.

0 comments save
embed

Log in or sign up to comment.

No comments yet.