Preference Optimization for Robust Instruction Following in Parameter-Efficient Language Models. JAAIR 2026, 5 (1).