Celebrations erupted in the capital Mogadishu as Mohamed Abdullahi Farmajo was declared winner.
"This is the beginning of unity for the Somali nation, the beginning of the fight against Shabaab and corruption ..." a triumphant Farmajo said after being declared the winner in a long, drawn-out election process in the conflict-wracked nation.
After six hours and two rounds of voting, Farmajo failed to win the required two thirds majority, but had 184 votes to Mohamud's 97, prompting the incumbent to drop out of a third round.
Celebrations erupted in the capital Mogadishu, and even in Dadaab, the world's biggest refugee camp populated by Somalis in eastern Kenya, according to an AFP correspondent.
Farmajo, 55, is a father of four who holds joint American and Somali citizenship.
He served as prime minister for only eight months between 2010 and 2011 and was ousted before he had a chance to prove himself in a deal to form a new government and postpone elections that year.