It looks to me like there are some serious limb twist issues there. Other than that, I would agree that there's too much bending going on in the handle. What I try to do with bendy handle bows is intentionally leave the handle stiff until I'm about 4" from my target draw length. Then I start to work the handle area to get it moving. Even so It is very easy to overstress a bow like this in the center. I'm not sure you'll be able to even out the bend in the limbs before losing more draw weight than you want, especially if you have limb twist due to uneven wood removal. Still, give it a go and see what you get. Every bow teaches us something.
Good luck,
George