“I never said I was perfect.”
Criticizing someone does not imply that you are questioning their level of perfection. Discard this platitude like the piece of trash it is.
Besides, you probably never assumed that they were perfect. Especially someone stupid enough to say something like “”I never said I was perfect.”
Point this out to them, too.
