You need to take responsibility for your end of your partner's affair