Black banners were raised in Tehran and some posters showed the supreme leader and Rafsanjani together smiling.
State television showed people pouring on to the streets around the campus where Ayatollah Ali Khamenei led the eulogies for Rafsanjani despite their "differences".
President Hassan Rouhani, parliament speaker Ali Larijani and his brother judiciary chief Ayatollah Sadegh Larijani accompanied the supreme leader at the prayers.
Former president Mohammad Khatami, an ally of Rafsanjani but long out of favour with the regime, was not part of the official delegation at the service.
The heavyweight politician, who died on Sunday at the age of 82, will be buried inside the crypt of Ayatollah Ruhollah Khomeini, the leader of Iran's 1979 Islamic revolution, at Khomeini's mausoleum is in south Tehran.
Black banners were raised in Tehran and some posters showed the supreme leader and Rafsanjani together smiling. Another poster said "good bye, old combatant".
Free bus or metro travel was provided to the funeral venue.
Since Rafsanjani's death, messages of condolence have poured in both from at home and and abroad.
Even the White House sent a message, unprecedented since the 1979 revolution that led to cutting of ties between Tehran and Washington.
"Former president Rafsanjani was a prominent figure throughout the history of the Islamic Republic of Iran and the United States sends our condolences to his family and loved ones," spokesman Josh Ernest said.
"He was a consequential figure inside Iran. But you know, for what potential impact this could have on Iranian policy, I wouldn't speculate."
One of Iran's most controversial figures in the West, the head of its Revolutionary Guards' foreign operations division, Major General Qassem Suleimani, was seen at the funeral.
Rafsanjani was a father-figure for Iran's moderate and reformist camps.
He fell out of the regime's highest inner circle after the 2009 re-election of hardline former president Mahmoud Ahmadinejad, when he spoke out against the use of lethal force on protesters who claimed the vote was rigged.
Video clips published on social media showed pockets of mourners in the streets around the funeral venue chanting slogans in support of Khatami and fellow reformist Mir Hossein Mousavi.
Mousavi and Mehdi Karoubi, two of the losing candidates in the 2009 election, have been under house arrest since 2011 for leading the so-called Green Movement protests that the regime calls "sedition".
Khatami is under a strict media ban and often prevented from attending public events.
Rafsanjani's son Mohsen invited people to attend the service "in full tranquility".
"Ayatollah Rafsanjani's concern was unity... and we should show off our unity to the world," he said on Monday.