it is rather diffucult to drill straight. get a piece of pipe inside diameter of your arrow, and a LONG drill bit. Take a bit of shaft material, about 1/2 inch, drill a hole perfectly centered through it, then glue that into the top. take the shaft you want to use, drill about 1/8 into the end perfectly centered. then slip about 1" into pipe, carefully insert bit through top guide, then move the bit around until you can feel it slip into that little indent/hole you made earlier, then drill your desired depth.
It requires a long bit, at least 12". Not super expensive.
Its an easy jig to make, best done from alumnium or steel tube(it comes in small enough diameters).
I use something similar to this(but with bigger bits) to make blowguns from solid wood stock. And drill bit extensions. I actually have a 36" 1/4 drill bit I use to drill a guidance hole, and a 32" 1/2 inch bit to drill the main bore(for a .50 cal blowgun). actual twist bits, not spade bits. They are only fluted for the last 3 inches.
