Your LLM Has Amnesia - And We Built the System That Keeps It That Way (or Almost According to a16z)

So a16z dropped a piece about continual learning last week, and I've been thinking about it obsessively since, in the way I used to think about the Transformer paper back in 2017 when I first read it three times in a row on a flight and arrived at my destination genuinely confused about what year it was