The jump from llama2 to llama3 had something to do with meta downloading every textbook ever published and using it as training data.

The arguments by meta so far in that court case are absolutely terrible and I'm half expecting to see the world's first trillion dollar copyright infringement award.