As long as you compensate for the knot in the layout, you won't have an issue with durability. The old timers knew this, and left a bit of extra width at the spot, or a bump on the belly. Fill any loose knots with Dutchmen plugs made from yew.
People sometimes have problems when they shave through knots,(both in tension and compression) pretending they aren't there. Some people get away with it, especially if its otherwise overbuilt. A highly stressed bow, with compromised knots will blow sooner rather than later.