With no agreement in place, the other person can go after half the equity in the house. In my opinion, wanting their down payment back seems reasonable.