Tell. I have played around with a fair amount of elk antler and had the same thoughts however, don't think you could do as well as Buffalo horn, it is more brittle then horn because it is antler not horn. From some of the testing I've done with sheds it is harder to form then horn. There are some documents of antler bows having been built from a strip of the main beams and then sinewed, but I doubt they performed very well. Hate to see ya ruin a nice set trying.
I made a two piece take down with a center section as the handle. One end glued in and the other limb removeable similar to the brass sleeves but it eventually cracked under stress