From what you have said I think it could only be one thing
When you look at a feather one side is shiny and smooth - the top of the feather as it sits in the wing. The other side is rough - the underside of the wing. Doesn't matter left or right wing BUT you must have all three from the same wing. Eg. all rough on the same side.
The offset you put on the feather has to be towards the rough side.
Eg the rough side is catching the air as the arrow flys.
If you offset to the wrong side eg. the smooth side catching the air then your arrows will fly......not very well
Highgly unlikely to be form etc if you can group the bare shafts and it is only when you fletch them that they start acting funny. If it was form then you certainly wouldn't be able to group bare shafts
Once you have your fletching sorted out then leave one arrow unfletched. Shoot it with your other arrows and it should group with them.