The attack took place in the village of Gawa Qanqa in the country’s Oromia region, with suspected members of the Oromo Liberation Army (OLA) targeting ethnic Amharas.
The killings occurred shortly after federal troops left the area “without explanation”, according to the human rights organisation.
Survivors, who had escaped the attackers and taken refuge in a nearby forest, later found the bodies in a school. Most of the deceased were women, children and the elderly, who the militants had rounded up and shot.
One man told Amnesty International that his father, his sister and his wife’s grandfather were among the dead.
“This senseless attack is the latest in a series of killings in the country in which members of ethnic minorities have been deliberately targeted,” said Deprose Muchena, Amnesty International’s regional director for East and Southern Africa.
“The fact that this horrendous incident occurred shortly after government troops abruptly withdrew from the area in unexplained circumstances raises questions that must be answered,” he added.
Mr Muchena called on Ethopian authorities to investigate the killings and to prosecute the perpetrators, as well as to protect minority groups across the country.
The Ethiopian Human Rights Commission (EHRC) also condemned the massacre of civilians by a group of up to 60 attackers, with its chief commissioner Daniel Bekele calling the deaths “unconscionable”.
The EHRC has urged the government to launch an independent inquiry into the killings, which would also “shed light” on why the military withdrew from an area “long known to be vulnerable to attacks”.
Around 200 families have now fled the area, according to Ararsa Merdasa, the head of the Oromia region police commission.
The incident is the latest in a spate of massacres over the last month in Ethiopia, which have left several dozen dead.
Referring to the most recent massacre, Dessalegn Chanie, a senior politician in the opposition National Movement of Amhara Party, accused the government of failing to protect its own citizens.
“Ethnic Amharas residing outside of the Amhara region are being labelled as outsiders and are exposed to repeated attacks,” he said.
Amharas are the second largest ethnic group in Ethiopia after Oromos.
Additional reporting from AP.