Cut the mamey in half, being careful not to get the large center seed. Discard the seeds and scoop the pulp directly into the blender. Add the ice, milk, vanilla paste, and sugar. Blend everything together on high speed until smooth, about 30 seconds to 1 minute.