The Sussex's were heckled having left early and told to go back to Britain