DC, it could be normal. Just realize, what's normal in one bow design may not be normal in the next. Much about how a bow acts on the tree, and in the hand, during the draw depends on its design. Is the bow you're talking about of symmetrical design or asymmetrical? Symmetricals tend to have a bow hand fulcrum that slides more under the hand as the bow is drawn, and they take longer in the draw cycle to 'correct'. They're also more prone to tipping initially in the draw. But if the bow is held stable in the tree, this sliding fulcrum can reveal itself in how the hook drifts relative to the line. It can very well be that properly tillered in the end, it drifts away from the line a bit, then back nearer as it approaches full draw.
BUT, it seems odd to me that you're seeing it happen so early in the draw. Asymmetrical bows tend to tow the line earlier, but even they don't go back and forth in just 6" of draw. If I were you, I'd make double/tripple sure nothing is tipping initially, even a hair. Watch the handle section very, very closely when you first pull the rope. Clamp it down if you can.