Lee Anderson cautions Work will 'ask' to rejoin EU as Lammy neglects 'number one need'