In a large bowl, whisk together the lemon pudding mix and cold milk for about 2 minutes until thickened. Gently fold in the whipped topping until fully combined and smooth.
In a 9×13-inch baking dish, place a single layer of graham crackers, breaking them as needed to fit. Spread half of the lemon pudding mixture evenly over the graham crackers. Add another layer of graham crackers, followed by the remaining pudding mixture. Finish with a final layer of graham crackers on top.
In a small bowl, whisk together powdered sugar, lemon juice, melted butter, and milk until smooth. Pour the frosting over the top layer of graham crackers and spread evenly.
Cover and refrigerate for at least 4 hours, or overnight, to allow the layers to soften and the flavors to blend. Slice and enjoy!