Self-Alignment with Instruction Backtranslation

We present a scalable method to build a high quality instruction followinglanguage model by automatically labelling human-written text with correspondinginstructions. Our approach, named instruction backtranslation, starts with alanguage model finetuned on a small amount of seed data, and a given webcorpus. The seed model is used to construct training examples by generatinginstruction prompts for web documents (self-augmentation), and then selectinghigh quality examples from among these candidates (self-curation). This data isthen used to finetune a stronger model. Finetuning LLaMa on two iterations ofour approach yields a model that outperforms all other LLaMa-based models onthe Alpaca leaderboard not relying on distillation data, demonstrating highlyeffective self-alignment.