Under the circumstances, it sounds like a very sensible precautions to remove the elephant ear mushroom coral from your seahorse system.
That was a very good idea to swap Mr. Big Lips for a colorful tree sponge too. They are veritable seahorse magnets and the ponies really do love them. Tree sponges are usually brightly colored (red and orange shades are common) and their shape and texture seem to make them irresistible to seahorses as hitching posts. Very often, all the seahorses in the tank can be found clinging to the same tree sponge together, eschewing other nearby holdfasts that appear every bit as comfy and attractive to human eyes.
Positioning the tree sponge where it will receive brisk water flow is also the right thing to do. The strong water movement will help discourage the growth of algae, which can easily smother a live sponge.
A bright red tree sponge like yours may well result in color changes at some point. Collectors will often find bright red or vivid orange seahorses living in beds of colorful sponges in the wild. Leslie Leddo has some great photos of a bright orange H. barbouri and an orange Pinto perched on a red-orange tree sponge in her aquarium, and the seahorses blend in perfectly with their bright orange hitching post.
Best wishes with all your fishes (and invertebrates), Carrie!