Despite previous promise, Biden rarely lunches with VP Harris