Recommended articles
The dramatic move came after a tense weekend in which Mnangagwa and Grace Mugabe -- who were seen as the leading candidates to replace the 93-year-old president -- openly traded barbs.
"The vice president has consistently and persistently exhibited the traits of disloyalty, disrespect, deceitfulness and unreliability," Information Minister Simon Khaya Moyo told a press briefing in Harare as he announced the removal.
On Saturday Mugabe openly threatened to fire Mnangagwa, accusing him of fanning factionalism to garner support ahead of the ruling ZANU-PF party's special congress due in December, after his wife was booed at a rally.
She shouted back at the hecklers: "If you have been paid to boo me, boo, go ahead... I don't care, I am powerful."
"We are denigrated and insulted in the name of Mnangagwa. Did I make a mistake in appointing him as my deputy?" the president asked the crowd following the incident.
Mugabe has given no indication that he will step down soon. His ZANU-PF party has already named him as its candidate for next year's presidential election.
'We need someone acceptable'
No one has yet been named to replace Mnangagwa as the government's number two and it is not thought it will happen before the party congress next month. The date for the meeting has yet to be released.
Phelekezela Mphoko continues to serve as second vice president.
But the ZANU-PF's powerful youth league quickly endorsed Grace Mugabe to replace Mnangagwa.
"We need someone who is acceptable and with unquestionable loyalty to the party and its principal, the President Comrade R. G. Mugabe," ZANU-PF youth league leader Kudzai Chipanga told journalists. "The only person possessing such qualities is... the first lady."
The main opposition MDC party said that the move showed the government could not be trusted with the country's future.
"ZANU-PF has been reduced to a Mugabe dynasty... We are urging Zimbabweans to join the big opposition tent and confront this collapsing dictatorship," MDC spokesman Obert Gutu told AFP.
Derek Matyszak, an analyst at the Pretoria-based Institute for Security Studies, said that Mnangagwa's sacking would effectively disqualify him from the race to replace the nonagenarian head of state -- leaving Grace in poll position.
"Two years ago it was inconceivable that Grace Mugabe would become deputy president but right now anything is possible," he told AFP.
He warned that powerful figures in Zimbabwe could seek to destabilise the ruling party in protest at the decision to remove Mnangagwa, who was appointed vice president in 2014.
He took over from Joice Mujuru who was axed after Grace Mugabe launched a campaign accusing her of plotting to topple the president.
Despite being widely touted as the obvious successor to Mugabe who has led the country for 37 years, Mnangagwa had been accused by Mugabe loyalists of undermining the president and fanning factionalism in the ZANU-PF.
Mnangagwa was last month stripped of his role as justice minister, in a cabinet reshuffle widely considered part of a campaign to reduce his powers and quash opposition within the government.
Grace Mugabe -- 41 years younger than her husband -- has become increasingly active in public life in what many say is a political grooming process to help her eventually take the top job.