'There's always going to be critics,' says US Olympian Lyles