Any weight past the string nocks is parasitic.  The end of the bow should be like a fishing rod, not a golf club.  The more mass at the end, the more energy is shunted through the string when it comes to a stop after release.  That energy has to go somewhere. If there is too much mass on the ends, the arrow has already left the string when the bow comes to rest and the remainder of the energy is dumped through the wood.

Thin that antler down to the outside diameter of where the string rests and taper it thinner towards the end.  On the belly side, taper the wood back also. You may also want to reduce the length past the nocks by about 1/3.

Check the tiller.