It seems to me that it is either that you formed the handle out of alignment with the limbs or, like Pat suggested, the limbs are uneven thickness causing the miss alignment when stressed(bracing adds stress). Just because the handle feels right in your hand doesn't mean the alignment is correct.