It's been a short while since I sold on eBay, but I had a feedback rating of about 4,500 so I've done a lot of transactions.
The trump card is, and always will be, the buyer's ability to contact their credit card company and reverse the charges. PayPal has no policy to stop this even though they claim to "vigorously defend Sellers from chargebacks" on their website. You will lose this case 100% of the time. I don't see how that will change if you have your own terminal. The Buyer can still reverse the charges. Since you know the card number maybe you can contact his credit card company but it's probably not going to do much.
I've found PayPal is more Seller friendly in terms of PayPal claims. For example, the customer has a duty to pay postage to return the product and that's a cost for him. You also have things like online tracking which shows delivery and PayPal has an IP log to see where the payments are coming from. That helped me when a buyer claimed that someone else made the payment. Because people often break into someone's house and make PayPal payments for them....heh.
You really just need to use PayPal. You'll get more customers and better prices and it will offset the losses from scammers. Also, about 99% of buyers are honest people. Consider the scammers a cost of doing business and keep making money off of the good Buyers. If you're just pissed off that people actually scammed you, get over it. Don't cut off your nose to spite your face. It's just part of doing business on eBay.