Model Promotion: Using EMA to Balance Learning and Forgetting in IIL

This article introduces a novel Knowledge Consolidation strategy for IIL that utilizes Exponential Moving Average to transfer learned knowledge from the student to the teacher model.