Pakalitha Mosisili, heads a coalition government that took power after snap elections in February 2015.
Monyane Moleleki, who quit the government earlier this week, said the landlocked country needed a broad-based and "strong" government of national unity to replace the "rotten" current administration.
"I invite all parties represented in the national assembly including the opposition to approach us to talk about how we can take this country forward," he said.
The current prime minister, Pakalitha Mosisili, heads a coalition government that took power after snap elections in February 2015.
On Thursday Mosisili sacked four ministers, immediately triggering the departure of four others including Moleleki, while 20 lawmakers from the ruling group announced they were quitting the fragile parliamentary majority.
The departures were officially due to disagreements over economic policy, according to local media.
Moleleki was speaking from his stronghold of Machache, about 40 km from the capital Maseru, in front of several thousand supporters dressed in the red colours of his party, an AFP journalist said.
Lesotho has been gripped by crisis since a failed coup d'etat in June 2014, which led to the elections in early 2015.
The All Basotho Convention (ABC) party of then premier Thomas Thabane was beaten by the Democratic Congress (DC) of former Prime Minister Pakalitha Mosisili, who formed a slim majority with several small parties controlling 65 seats in the 120-member parliament.
On Sunday several ABC members joined the DC rebel group, confirming that alliances could be completely rebuilt in parliament, threatening the current government.
Completely landlocked by South Africa, Lesotho is one of the world's poorest countries and its economy is heavily dependent on its larger neighbour, to whom it exports water and hydroelectric power.
South Africa, the United States and the Southern African Development Community (SADC) bloc have repeatedly called for political reform in Lesotho.