Olsen's departure follows an inquiry into the indirect financing by Lafarge of armed groups in Syria.
His resignation, which has been accepted by the group's board, will be effective on July 15, LafargeHolcim said in a statement.
Olsen's departure follows an inquiry into the indirect financing by Lafarge of armed groups in Syria, a country torn by civil war, to keep one of its cement plants operational.
"My decision is driven by my conviction that it will contribute to addressing strong tensions that have recently arisen around the Syria case," said Olsen.
"While I was absolutely not involved in, nor even aware of, any wrongdoing I believe my departure will contribute to bringing back serenity to a company that has been exposed for months on this case," he said.
In March, LafargeHolcim admitted that it had resorted to "unacceptable practices" to continue operations at one of its now-closed factories in Syria.
The admission comes after sources close to the case told AFP in January that the French government had filed a legal complaint against Lafarge for buying oil in Syria to power the Jalabiya factory, in violation of sanctions.
The factory closed down before French cement maker Lafarge merged with Switzerland's Holcim in 2015.
The plant, located in northern Syria some 150 kilometres (95 miles) northeast of Aleppo, was finally evacuated in 2014.
Lafarge is suspected of sourcing oil locally to operate the factory in defiance of a 2012 EU ban on purchases of Syrian oil as part of a sanctions package targeting the regime of Syrian President Bashar al-Assad.
LafargeHolcim said Monday that the search for Olsen's successor would begin "immediately."
In the meantime, supervisory board chief Beat Hess would take over in the interim.