Good for you for asking the question!
DO NOT provide this person ANY more personal information than you have already provided. DO NOT give them a physical address, if you haven't already. ESPECIALLY DO NOT give them any information about your financial accounts. Not even the name. Nothing.
The potential for scamming and stalking is legitimate. Every bit of additional information you provide increases your risk. ESPECIALLY DO not assume that just because you don't see the scam that there is no potential for it.
Based on what you've provided, it is EXTREMELY likely he is scamming you.
You ask, "how is he trying to scam me". Impossible to say exactly how this particular person could be trying to scam you, we don't have all the facts, but scammers will gather little bits of personal information and use it together to exploit their victims. It takes very little for a persistent scammer to exploit. As an example, a scammer can give you a check, you deposit it, the scammer asks for a little of the money back for some reason, and then the check turns out to be no good. OR, a scammer gets enough of your bank information, seizes control of your account and cleans it out, and then disappears. There are many many other scenarios and it depends on exactly and specifically all of the information you have provided to the scammer.
Stalkers get cell phones and addresses and once the relationship does not go how they wish, the harassing begins. Up to and including physical danger.
Seriously, disengage. Don't explain, just disengage. Even better, change your number.