

You’re not paying a “fee”. Sure, someone is paying for it, but it isn’t a fee.
Returns are a right and a necessity. Just as you take broken, spoiled, lost (and as you said, stolen) goods into account and “mark up” others to make up those losses, you do the same thing for returns.
It’s a business expense that has to be covered by some means (larger margins). But that’s not a “fee”.
Greedy retailer.
Any retailer worth their salt would include them in their profit margins.
When you run a retail store (or online store) selling physical goods, you are bound by the rules of matter (as oppoes to digital stuff). Stuff breaks. Food spoils. Old car models lose value. PC parts even quicker. Stuff gets lost. Stuff gets broken. An error occurs during manufacturing. These are all sources of loss which you have to take into account, predict and mitigate.
Adding returns to that already large (and by no means exhaustive) list isn’t an unreasonable ask.
You just estimate the number and projected cost of returns and adjust your prices and profit margins accordingly.
A “restock fee” is definitely uncalled for. The store made the decision to order X amount of the product, with a Y margin of loss (lost, broken in transport, stolen,…). These present a loss of item. An item they could’ve sold. However, a return isn’t a loss of item. They get the item back. And charging customers for the priviledge of buying something, getting dissappointed and making a big deal out of it with “restock fees” is a stupid business move - you risk losing the customer. Especially when you consider the fact that a return is the smallest cost out of all the issues mentioned here.
And if your competition doesn’t treat their customers as bad as you do - the risk isn’t small. And even if not, a boycott out of spite, even just one customer, is a much larger loss than the net gain of one “restock” fee.
So, it’s just greedy. And a bad business move if you care about customer retention. Not doing it while others do is a smart move, since these things are bound to happen. And when they do at a competitor, who knows? Maybe the customer tries you next and just… Remains loyal. Although when you say “customer loyalty” today, people think of gimmicks like loyalty cards.