Too Much Change
Another reason why Brady should have retired earlier is that the Bucs entered another transitional period. Bruce Arians said that he quit because Brady came back. He did this as a compliment to the quarterback because he wanted Todd Bowles to inherit his talents. But Arians had already mentally checked out after winning the Super Bowl.

Meanwhile, there was a significant player turnover and this affected Brady. He lost Rob Gronkowski to retirement again as well as multiple receivers. This meant that he didn’t have many elite weapons apart from Mike Evans. It affected Brady’s game because he was trying to forge new on-field relationships but it didn’t always work (via Glamour).