If you've tillered the bow to bend thru the handle, even a little bit, and then you try to glue on a handle riser, it really doesn't matter what glue you use. It won't stay on.
Leave the handle riser off, and make it a bendy handle bow. This type of bow will have a little more hand shock, all else being equal, than a non-bendy handle bow. Hand shock is also intensified with heavy tips, or a poorly tillered bow. Have you put a grip on yet? When you do, the hand shock will go way down, especially if it's a leather grip with a bit of padding under it.
I have used TB3 and epoxy with ipe for risers. Make sure you degrease first.