Both answers so far seem to have overlooked that your ex-fiance has been paying for some or all of the car, even though you are the only one on the title.
Ask him to propose a plan for him to take full ownership of the car and its loan. Make sure that he understands that you no longer want to share the car. You would like him to take full responsibility or ...
An alternative to involving the police...though if this isn't an option, please involve the police.
If he lives with his mother and she is understanding, there's nothing wrong (read: illegal) by swinging by their house while he's out, her letting you in the house, and her giving you the keys. Then you get in the car and drive home.
Is your ex on the loan? ...
You go down to the police station where he lives and tell them you are repossessing a vehicle you own. Make sure they have a copy of the title and whatever useful information you can find (loan/receipt).
They will then either tell you to go find vehicle and call them or they will follow up where he lives and get the car handed over to you.
If the ...
Each state is different, here is the information from [New York state]:
About transferring vehicle ownership and acceptable proofs of ownership:
Cars, Trucks, and Motorcycles Model year 1973 or newer
The acceptable proof of ownership is either
the original New York State Certificate of Title – must be the original 1