The bill's withdrawal marks a rare concession to popular opposition by the ruling Justice and Development Party (AKP)
Critics had said the bill -- which would allow the release from jail of sex assault convicts if they marry their victims -- would legitimise rape. Thousands took to the streets in protest at the weekend.
"We are taking this bill in the parliament back to the commission in order to allow for the broad consensus the president requested, and to give time for the opposition parties to develop their proposals," Yildirim said at a news conference in Istanbul.
"This commission will evaluate and take into account all sides and surely a solution will be found," the prime minister added.
In comments overnight, President Recep Tayyip Erdogan had called for a compromise to be found on the bill.
The bill's withdrawal marks a rare concession to popular opposition by the ruling Justice and Development Party (AKP).
Co-founded by Erdogan, the AKP has dominated Turkish politics since coming to power in 2002.
If the bill had passed, it would have permitted the release from prison of men guilty of assaulting a minor if the act was committed without "force, threat, or any other restriction on consent" and if the aggressor "marries the victim".
Opposition parties from across the political spectrum had heavily criticised the bill, which was approved in an initial parliamentary reading on Thursday.
It was expected to be put forward again in parliament on Tuesday but since last week, there have been protests in country in which thousands of people urged the government to withdraw the bill as well.
The main opposition Republican People's Party (CHP) had called for the bill to be withdrawn and vowed to go as far as the constitutional court to block it.
But the Turkish government has insisted it was trying to help families in which the men involved were not rapists or sexual aggressors, and who were unaware of the law.
The legal age of consent in Turkey is 18 but child marriage is widespread, especially in the southeast.
Even as he said the bill would be withdrawn, Yildirim insisted that the proposals sought to rectify the situation for 3,800 families who "are forced to grow up without the love of their father" -- likely imprisoned -- and "paying for the mistakes of their mothers and fathers".
But as well as opposition activists, the bill had also met with criticism from the pro-government Women's and Democracy Association (KADEM), whose deputy chairman is Erdogan's younger daughter Sumeyye Erdogan Bayraktar.
It said in a statement on Friday that one of the bill's biggest problems would be proving legally what constituted force or consent.
Campaigners have long accused the government of failing to do enough to stamp out child marriage and of paying more attention to pushing up the birth rate.
Erdogan in June this year urged women to have at least three children and once described birth control as treason.