Chill a 9-by-13-inch (23-by-33-cm) baking dish and a large mixing bowl for 10 minutes if time allows. This helps the cream whip more easily.
Beat the cold heavy cream in the chilled bowl with an electric mixer on medium-high speed until soft peaks form, about 2 to 3 minutes. Do not overbeat.
With the mixer on low speed, gradually add the sweetened condensed milk and vanilla. Beat just until the filling is smooth and holds medium peaks.
Arrange a single layer of graham crackers over the bottom of the baking dish, breaking a few crackers as needed to cover gaps.
Spread about one-third of the cream filling over the crackers. Add about one-third of the mango pieces in an even layer.
Repeat with two more layers of graham crackers, cream filling, and mangoes. Finish with mangoes on top. Cover the dish tightly.
Refrigerate for at least 6 hours, or preferably overnight, so the graham crackers soften into cake-like layers. Slice cold and serve.