BART
Corrupt the text, learn to rebuild it — summarization's workhorse
Latest: BART base/large (Oct 2019)
Denoising sequence-to-sequence pretraining (October 2019): corrupt text with noise (deletion, infilling, shuffling), train a full encoder-decoder Transformer to reconstruct it. BART became the go-to model for summarization and generation tasks, and its recipe influenced a generation of seq2seq models.
Why it matters
BART's corrupt-and-reconstruct pretraining made a full encoder-decoder Transformer the go-to for summarization and generation; bart-large-cnn was the standard summarization baseline for years and still handles over a million monthly Hugging Face downloads. Its denoising recipe shaped the seq2seq models that followed it.
Facts
- BART-large fine-tuned on CNN/DailyMail was the standard summarization baseline for years; distilled variants (DistilBART) still serve production summarizers.
Try it yourself
Summarize text with bart-large-cnn ↗ bart-large on Hugging Face ↗ Read the paper ↗