Cool bow, Aaron!  I have been working some dogbane for a chokecherry bow I hope to build if it ever warms up enough to get out and find some wood.  I find that, as I see-saw the fibers between my fingers to get the pith out of the stalks, most of the outer bark breaks off too.  It's a tedious job--I work it while watching TV with the family or listening to music--but not too hard.  Has that been your experience too?  A lot of primitive-skills people on youtube mention scraping the bark off or otherwise struggling with it as you've said; other people echo my experience the most of the bark comes off fairly easily as you work the fibers.  I suspect this has something to do with regional differences or different subspecies of the plant.  I find it a lot easier than working sinew.
Whatever the case, it's cool to see somebody actually doing this.  Does the db add draw weight/snap to the bow, do you think?
Thanks.  The db added about 3lbs to this one, but I didn't have it in reflex when I glued it on.  I'd like to try and see if that would work, might add a bit more that way.
I do like you with getting the pith out, but most of mine the outer bark wants to hang on.  I get a few stalks worth and rub the bundle between my hands and most falls off that way, but you get little pieces all through the stack.  When I twist cord it tends to all come out as I twist, but for the bow it was just mixed in with the fibers.  I've tried combing, but I feel like I loose too many fibers, so I quit doing that.
Good luck with your bow, I'll be excited to see your experience with db.