What the bank 'should' have done is a question of law, but I will point out that the reason for the rejection was not the bank's error, it was yours / the payee's [which, in the bank's eyes, is still kind of your error].
Consider from the bank's point of view:
(1) You release 50k SEK from your account;
(2) The bank, seeing it is headed to a EUR account, converts it to EUR for you, at their available rate [of course, they will take a cut off of the conversion];
(3) The bank sends the EUR off to the account you requested;
(4) The EUR is rejected due to invalid account;
(5) The bank needs to convert the EUR back to SEK for you [again, they do likely take a cut], at a conversion rate that happens to give you 633 less SEK than what you started with.
If the bank did as you ask, and used the same rate to convert your refund in step 5 as your original conversion in step 2, they might have lost money. ie: if the SEK had strengthened in between step 2 & step 5, then it would have cost the bank more EUR, out of their own pocket, to get you your SEK back. They are not a charity, so they chose to charge you the regular conversion rates in both steps.
Whether they should do as you ask is probably a question of law; whether they might do as you ask is probably a question of customer relationship. That is, if you were worth 633 SEK to them, they might swallow their pride and give you your original value back, if you asked nicely.