Neandertal ancestors left between 500,000 and 300,000 years ago, followed by humans around 50,000 years ago.