Post #2137

Hugging Face (Twitter) RT @remi_or_: Late Christmas gift for RL people 🎄 We are adding support for parallel decoding in transformers continuous batching! You can now decode as many streams as you want from one prompt, which changes the game for long contexts 📜 but since a picture is worth 1024 words: