If you’re using those dowel cutters, why not do the wider hole first? The “brad point” will leave an indentation at the bottom of the hole in the centre, is that not enough to centre the narrower cutter?