Place the potatoes in a large pot and cover with cold water by 1 inch (2.5 cm). Salt the water generously and bring it to a boil over high heat. Lower the heat to maintain a steady simmer and cook until the potatoes are tender when pierced with a knife, about 15 to 20 minutes depending on their size.
Drain the potatoes and let them stand until cool enough to handle. Peel if desired, then cut them into 1/2-inch (1.25-cm) slices.
While the potatoes cook, add the queso fresco, evaporated milk, ají amarillo paste, crackers, garlic, oil, lime juice, salt, and pepper to a blender. Blend until completely smooth, stopping to scrape down the sides as needed.
Check the sauce consistency. It should be thick but pourable. Blend in 1 tablespoon (15 ml) water at a time only if needed. Taste carefully and add a little more ají amarillo paste for extra heat or a pinch of salt if needed.
Arrange lettuce leaves on a large serving platter. Layer the sliced potatoes over the lettuce and spoon the sauce generously across the top. Garnish with egg halves and black olives. Serve at room temperature or lightly chilled.