I think there are/will be real problems with a very small number of specific species of fish, and in-particular anemones.
But for the vast majority of fish, the population doubling times is low enough that the real impact of removing them is small.
And as far as corals go, by far the most destructive practise is limestone mining for building material.
Then there is global warming. Which has yet to actually show any real long term destructive effect on the reefs. All the hype about the reefs disappearing in the next 50 years, seems to be just that - hype (as convenient as it is if you want to get funding for research). They won't disappear, they just won't be in the same places as they are now. They will (and are) moving. They've done it before, and there's nothing to suggest they won't do it again.
The hobby isn't killing the reefs. There are a multitude of other practises which are causing significantly more damage the fish and coral collection.
Sure it's not helping, but for the most part, it's a drop in the ocean relative to the other reef killers out there.