Australia minister steps aside over affair allegations