Now you’ll either have to let them go (stressful for you and them) or tolerate the bad fit (stressful for you, them, and everyone else on the team). One stress breeds another stress.
It's bettsr to get clear with arguments and conclude with one wherewould be less cons. Decision would be tough, but in the end the continuity would be much problematic.