What to do? What to do? Help?

I think you should first ask your partner what he thinks about asking them to move out, because if they make you uncomfortable, I bet your partner would completely understand. Just talk to him.