Preheat your oven to 350°F (175°C) and grease and flour two 9x5x3-inch loaf pans.

In a large mixing bowl, combine diced apples with brown sugar, buttermilk, vegetable oil, and eggs, and mix until combined.

Sift in the flour, baking soda, cinnamon, salt, and nutmeg, and gently fold until just combined.

Divide the batter evenly between the prepared loaf pans and smooth the tops.

Bake for 45 minutes or until a toothpick inserted comes out clean with some moist crumbs.

Cool in pans for 10 minutes before transferring to wire racks to cool completely.

To make the glaze, melt the butter in a saucepan, stir in brown sugar, and bring to a boil for about 2 minutes.

Mix in milk until incorporated, remove from heat, and let cool slightly before stirring in powdered sugar.

Drizzle the caramel glaze over the cooled loaves and allow it to set.