TGTGInsightintelligence telegramLIVE / telegram public index
← Hugging Face
Vedi originale
Hugging Face avatar

TGINSIGHT POST

Post #2019

@huggingface

Hugging Face

Visualizzazioni13Numero di visualizzazioni
Pubblicato18 dic18/12/2025, 17:27
Contenuto del post

Contenuto

Hugging Face (Twitter) RT @itazapo: Transformers v5 redesigns tokenization. In this blog post we talk about: > tokenization crash course > tokenizers and transformers - the bridge > v5 tokenizers backend Major shoutout to @karpathy for his BPE video that got me interested in tokenization in the first place.