Here in the U.S., a realtor can act as a "seller's agent" or a "buyer's agent". I think what you are calling a "broker" in the U.S. we call a "buyer's agent", and this may just be a difference in terminology, from your post it sounds like the concept is the same. I am answering from a U.S. perspective, please let me know if something doesn't make sense in the Israeli context.
Here, each typically gets 3% to 3.5% of the sale price (at least in my part of the country). So yes, the buyer's agent has an incentive to get a higher price, even though this is contrary to the interests of the person he is supposed to represent.
On the other hand, the buyer's agent has a strong incentive to find a house at a price that you consider acceptable. If the absolute most you are willing to pay is, say, ₪1,000,000, and he keeps showing you houses that cost ₪1,500,000, he's just wasting his time. (He's wasting your time too, of course, but let's assume he doesn't care about that.) (I don't know what housing prices are in Israel today, just making up numbers.)
Suppose he has two houses that he can show you, one in your price range and one not. If he shows you the first you may buy it and we will very quickly get his commission. If he shows you the second, you probably won't buy it and he'll get zero. If he keeps showing you houses above your price range, he's doing a bunch of work for which he will never be paid.
The worst case from your point of view is if you're thinking that you're expecting and prepared to pay, say, ₪1,000,000 to ₪1,300,000, and you tell the broker that, his incentive is to concentrate on the upper end, maybe even push it a little. But still, if he shows a house that's well within your range so you'll quickly buy, he can get ₪30,000 today, versus trying to push you to go higher so he can maybe get ₪39,000 in a few months. Is the extra ₪9,000 worth several months of extra work? Probably not.
Personally, I've never had a problem with a realtor trying to push me to buy a house more expensive than I said I was prepared to pay. At least not that I noticed. Maybe they were very skillful at it and I didn't realize they were doing it, like showing me houses that were totally run-down dumps until I decided I was willing to pay more.
As to your specific suggestion: I don't know if a realtor would be willing to negotiate an alternative deal from their standard contract. I've never tried to do such a thing. Yes, this would give him an incentive to find the lowest possible price. Arguably this would create a perverse incentive to show you houses of very low quality just because they're cheap. And there would be the problem that he'd have no incentive to show you houses at or just over your stated maximum, as his commission would be zero. (Negative if it goes over slightly?)
What I did on my last house was tell the realtor, I want to start by looking at houses costing under \$X. If I can't find anything I like, I'll go a little higher. By not telling the realtor my maximum, I discouraged her from immediately going for the maximum. At least that was my theory.