Preheat the oven to 350°F (175°C). Grease and flour a 9x5-inch loaf pan.
In a medium bowl, whisk together the flour, baking powder, cinnamon, and nutmeg. If using, stir in the dried cranberries to coat them in flour.
In a large bowl, use an electric mixer to beat the softened butter and granulated sugar until light and fluffy, about 2-3 minutes. Add the eggs one at a time, beating well after each.
Mix in the eggnog, rum extract, and 1 teaspoon of vanilla extract on low speed. The batter may look curdled at this point, which is normal.
Add the dry ingredients to the wet ingredients in two batches. Mix on low speed until just combined. Be careful not to overmix.
Pour the batter into the prepared loaf pan and smooth the top with a spatula. Bake for 55 to 60 minutes.
The bread is done when a wooden skewer inserted into the center comes out clean or with a few moist crumbs. The internal temperature should read 200°F (93°C).
Let the bread cool in the pan on a wire rack for 10 minutes. Then, carefully turn the loaf out of the pan to cool completely on the rack.
For the glaze, whisk the confectioner's sugar, 2 tablespoons of eggnog, and 1/2 teaspoon of vanilla extract in a small bowl until smooth. Add more eggnog, a teaspoon at a time, to reach a pourable consistency. Drizzle over the cooled loaf and let the glaze set before slicing.