The England striker, whose last-gasp penalty gave United a stunning victory over PSG here in the last 16 in 2019, smashed home low in the 87th minute as Ole Gunnar Solskjaer's side claimed a well-deserved three points.
They had taken a first-half lead behind closed doors in Paris courtesy of a retaken Bruno Fernandes penalty after United's captain for the night had initially missed.
But it looked as though PSG would escape with a draw despite a poor performance after France striker Anthony Martial headed a Neymar corner into his own net 10 minutes after the restart in this Group H encounter.
It is another fine result for Solskjaer, who was handed the United manager's job on a permanent basis on the back of that victory here two seasons ago and has now seen his side bounce back in style from an embarrassing 6-1 home loss to Tottenham Hotspur just before the recent international break.
He had again left Paul Pogba on the bench at kick-off, just like in Saturday's 4-1 Premier League win at Newcastle United, and the Norwegian's decision to start with three central defenders also paid off.
Axel Tuanzebe, whose last start had come 10 months ago, came into the back line with Harry Maguire missing, while there was a debut as a left wing-back for Alex Telles following the Brazilian's arrival from Porto.
PSG, meanwhile, looked out of sorts, perhaps struggling more than their visitors to adapt to the bizarre experience of playing such a game without their fans behind them.
In different times the Parc des Princes would have been buzzing with anticipation prior to kick-off, ahead of the first game of another Champions League campaign, under the lights.
Instead it was empty, the 9:00 pm local time start of this match coinciding with the start of the nighttime curfew recently introduced in the French capital as part of attempts to slow the spread of the coronavirus.
PSG off colour
PSG were hoping to get off to a winning start to their latest bid to win this competition, two months after Thomas Tuchel's side lost to Bayern Munich in the final.
The French champions were also seeking revenge against United for that defeat in the last 16 in 2019.
If that night it was a controversial late penalty by Rashford that took United through, this time the visitors went ahead from another spot-kick awarded after Martial was fouled as he tried to spin away from Abdou Diallo.
Having missed a penalty in the weekend win at Newcastle, Fernandes -- wearing the captain's armband in the absence of Maguire -- stepped up and saw his kick stopped by Keylor Navas.
However, the Costa Rican was off his line when he made the save so referee Mateu Lahoz ordered a retake. This time Fernandes kept his nerve to score.
PSG had come close to taking an early lead when an Angel Di Maria curler was tipped around the post by David de Gea, but they looked off colour and missed Marquinhos and Marco Verratti.
Tuchel therefore changed things around at half-time, sending on Everton loanee Moise Kean and changing formation.
The alterations had the desired effect. After a fantastic De Gea save from Kylian Mbappe and a Layvin Kurzawa cross that landed on the crossbar, Paris equalised in the 55th minute as Neymar's corner from the left was headed into his own net by Martial.
Pogba was then sent on by Solskjaer, and the action swung from end to end. Yet it was Rashford who had the best chances, twice being denied by Navas before eventually finding the net.
The result sets United up nicely for next week's home clash with RB Leipzig, who also won on Tuesday, beating Istanbul Basaksehir 2-0 in Germany.