In a large bowl, use an electric mixer to beat the softened butter and granulated sugar on medium speed until light and fluffy, about 3 minutes.
Add the eggs one at a time, mixing well after each. Beat in the eggnog, vanilla extract, and optional rum until combined. The mixture may look slightly curdled, which is normal.
In a separate bowl, whisk together the flour, baking powder, ground nutmeg, and salt.
With the mixer on low speed, gradually add the dry ingredients to the wet ingredients, mixing just until a soft dough forms. Do not overmix.
Cover the bowl with plastic wrap and chill the dough in the refrigerator for at least 30 minutes, or until firm enough to handle.
Preheat your oven to 350°F (175°C). Line two large baking sheets with parchment paper.
On a well-floured surface, roll the chilled dough to 1/4-inch thickness. Use floured cookie cutters to cut out shapes and place them about 1 inch apart on the prepared baking sheets.
Bake for 10-12 minutes, or until the edges are lightly golden. Let the cookies cool on the baking sheets for a few minutes before transferring them to a wire rack to cool completely.
To make the icing, combine the powdered sugar and softened butter in a small bowl. Add the eggnog and mix until smooth and spreadable. If it's too thick, add more eggnog, 1 teaspoon at a time. Spread the icing on the cooled cookies and sprinkle with nutmeg.