When Patriots are pushed too far