I've never done it, but looks like you left plenty of extra meat around that knot to eliminate any concern. I'd drill that bad boy out! Probably will just take longer than pushing out a dead branch because you will have to clean up the inside of the hole on the belly side. Again I've never tried but my suspicion is that's it not difficult, just more time consuming.