In a skillet, melt butter over medium heat. Add brown sugar and stir until bubbly. Add sliced bananas and cook for 1–2 minutes until soft and caramelized. Pour this mixture into the bottom of a greased 9x13-inch baking dish.
Add cubed bread on top of the banana mixture, spreading evenly.
In a large bowl, whisk together eggs, milk, cream, brown sugar, vanilla, cinnamon, nutmeg, and salt until fully combined.
Pour the custard evenly over the bread cubes, pressing slightly to help the bread absorb the liquid. Cover and refrigerate for at least 1 hour or overnight for best results.
Preheat oven to 350°F (175°C). Remove the casserole from the fridge and let it sit at room temperature for 15–20 minutes. Bake uncovered for 40–45 minutes or until the top is golden brown and the center is set.
Let cool slightly before serving. Dust with powdered sugar, drizzle with maple syrup, or sprinkle with chopped nuts if desired.