Cut the buttermilk cake loaf into small cubes. Set aside.
In a medium bowl, combine the cold almond milk and cinnamon pudding mix. Use an electric mixer or whisk to blend until thickened, about 2-3 minutes. Let the mixture sit for an additional 5 minutes to firm.
Layer the trifle bowl starting with cake cubes at the bottom covering the surface well.
Add a layer of the pudding over the cake cubes, then spread half of the apple puree over the pudding.
Drizzle half of the ginger caramel sauce over the apple layer, followed by sprinkling 1 cup of chopped walnuts.
Repeat the process with the remaining cake cubes, pudding, and apple puree. Hold off on adding the remaining caramel and walnuts until later.
Cover the trifle with plastic wrap and refrigerate for at least 3 hours or overnight.
Before serving, top the dessert with whipped cream evenly over the top layer of apple puree.
Drizzle the remaining ginger caramel sauce over the whipped cream and sprinkle with the remaining chopped walnuts.
Notes
For deeper flavors, use spiced apple puree featuring cloves and cardamom. Toasting the walnuts will elevate their flavor. Consider garnishing with grated nutmeg or cinnamon sticks for a seasonal twist.