Yes, good thinking.
by the pic, anyway, you have got quite a bit of rock in there, opening it up by removing some may look good.
Also there is debate, and contradicting anecdotal evidence about toxins, but for me anyway I would tend not to have such a large leather relative to water volume. There may be no discernable effect on other corals, but it may just simply slow growth of them generally.
Something I've been wondering, due to the conflicting stories about this, perhaps a coral will not manufacture much toxins if all is well, but may start to do so in response to certain triggers, such as detecting encroachment of competing corals?
Just a theory , but I know for certain that some land dwelling plants do it that way.