Abbas also said the Palestinians may consider reversing recognition of Israel if Trump moves the US embassy to Jerusalem.
Abbas held a private meeting with Pope Francis before heading to inaugurate the diplomatic mission, located in a building facing the Vatican that also houses the embassies of Peru and Burkina Faso.
Speaking briefly to reporters, Abbas reiterated his opposition to the possible transfer of the US embassy to Jerusalem, pledged by US President-elect Donald Trump.
"We cannot say anything yet because it has not happened, but if this does happen it will not help the peace process. I hope it will not happen," said Abbas, speaking in Arabic.
In an interview with French daily Le Figaro on Friday Abbas said the Palestinians may consider "reversing recognition" of Israel if Trump moves the US embassy to Jerusalem.
"I wrote to president(-elect) Trump to ask him not to do it. Not only would this move deprive the United States of all legitimacy in playing a role in conflict resolution, it would also destroy the two-state solution," Abbas was quoted as saying.
Trump promised during the election campaign to recognise Jerusalem as the capital of Israel and move the American embassy there.
Such a step would be a historic break with US policy, and with most of the international community, over the status of Jerusalem, also claimed by the Palestinians as capital of their future state, an issue to be settled by negotiation.
Abbas met with Pope Francis for 20 minutes ahead of the embassy opening.
Among the gifts exchanged, Abbas offered the pontiff a stone from the Church of the Holy Sepulchre in Jerusalem, according to Greg Burke, spokesman for the Holy See.
A Vatican statement regarding Israel and the Palestinians added: "Hope was expressed that direct negotiations between the parties may be resumed to bring an end to the violence that causes unacceptable suffering to civilian populations, and to find a just and lasting solution."
The private audience was the third meeting between Francis and Abbas. The pontiff visited Israel and the Occupied Territories in 2014 and Abbas made a trip to the Vatican the following year for the canonisation of two Palestinian nuns.
Relations between the Vatican and the Palestinian Authority turned a new page in 2015 with the signing of an agreement to create a Palestinian embassy at the Vatican.
The agreement -- two years after the Vatican recognised Palestine as a state -- provoked the ire of Israel, which was also angered when Francis called Abbas "an angel of peace" during their meeting in May 2015.