I often hear people's scrapers/knife blades make noise as they come in contact with the wood. If you hear a pop or click as your tool comes down, you just cut a ding in the wood where you started scraping. *click*scrape*click*scrape*
That's a cut mark going ACROSS the grain. So every scrape leaves another tool mark.
Instead, lay the scraper/knifeblade on the limb well above the tool mark and rotate it up to scraping position as you move it towards the tool mark you intend to take out. It's kinda like a wiping motion. And that is an apt description because it wipes out tool marks.
And with your rasps and such, try changing the angle of the rasp to the limb. All of my rasps have a point where the rank and file of the teeth line up and it leaves virtually no toolmarks.
Lastly, SIDEWALK CHALK, and the cheapest you can find. Or a charcoal briquette.
No really. Rub this up and down the limbs and wipe with a rag. Pops those tool marks out so they are easier to find.