The negotiations mark a new milestone in the Colombian peace process, after President Juan Manuel Santos's government sealed a historic accord with the country's largest rebel group, FARC.
The negotiations mark a new milestone in the Colombian peace process, after President Juan Manuel Santos's government sealed a historic accord with the country's largest rebel group, the Revolutionary Armed Forces of Colombia (FARC), in November.
The government and the ELN are now seeking to bring "complete peace" to Colombia, said the representative for host country Ecuador, Juan Meriguet, as he declared the talks open in a small ceremony at a Jesuit retreat outside the capital Quito.
"We have before us the opportunity to finally turn the page on this war," said the government's chief negotiator, Juan Camilo Restrepo.
But experts warn the ELN will be a tougher negotiating partner than the FARC.
There was friction between the two sides even as they celebrated the formal opening of talks.
Restrepo warned the rebels that if they fail to give up kidnapping, "it will be very difficult to advance."
The ELN's chief negotiator, Pablo Beltran, for his part called on the government to "take responsibility" for its actions during the conflict -- saying the rebels were prepared to do the same.
Colombia is the scene of the last major armed conflict in the Americas.
South America's third economy and the world's biggest cocaine producer, the country has been torn since the 1960s by fighting that has drawn in multiple leftist rebel groups, right-wing paramilitaries, drug gangs and the army.
November's landmark peace accord with the FARC, the oldest and largest rebel group, leaves the ELN as the last active guerrilla insurgency.
It has an estimated 1,500 fighters, mostly in the north and west.
The talks come after three years of secret negotiations and an embarrassing false start in October, when the ELN refused to release their most high-profile hostage, ex-lawmaker Odin Sanchez.
A flurry of behind-the-scenes negotiations followed, leading to Sanchez's release on Thursday in exchange for two ELN prisoners.
In a further goodwill gesture on Monday, the ELN released a soldier they had captured two weeks earlier.
But there will be more bumps in the road, warned Frederic Masse, an expert on the conflict at the Universidad Externado in Bogota.
"The ELN has more fundamentalist demands than the FARC," he said.
"They want much deeper social change."
This marks the fifth effort to make peace with the ELN, after a string of failed attempts in the 1990s and 2000s.
Negotiators will now get down to business on Wednesday, behind closed doors.
Despite Monday's hostage release, the issue of kidnappings remains a touchy subject.
Unlike the FARC, "the ELN has still not renounced kidnapping," long a source of revenue for both rebel groups, said Kyle Johnson of the International Crisis Group.
"They might kidnap someone else in the future and we'll be back in the same difficulties."
Elections in 2018 to decide Santos's successor also threaten to complicate matters.
The peace process faces ongoing resistance from conservative opponents who accuse Santos of granting impunity to rebels guilty of war crimes.
Santos, who won the Nobel Peace Prize last year, had to tweak the initial FARC accord after voters narrowly rejected it in a referendum last October -- a major embarrassment for the government.
The slightly revised version was ratified in Congress, where the president enjoys a majority.
A new poll found that Colombians are growing less optimistic on the chances for peace.
Polling firm Datexco, which interviewed 900 people nationwide on the prospects for peace, found 51.7 percent were optimistic, down from 67.4 percent in October.