A seriously depleted United looked to have given themselves too much to do after losing 2-0 in the first leg of their last-16 tie last month, yet a Romelu Lukaku brace in the first half at the Parc des Princes gave them hope.
Even so, Juan Bernat had netted in between for PSG, and it looked as though they would stumble on until the drama at the death.
Slovenian referee Damir Skomina awarded a spot-kick after reviewing the images when he had been alerted of a possible handball by PSG defender Presnel Kimpembe in the box.
Rashford duly beat Gianluigi Buffon from 12 yards, as United won a European tie after losing the first leg at home for the first time in their history.
Their improbable victory -- following Ajax's similarly stunning turnaround against holders Real Madrid 24 hours earlier -- takes them through to the last eight for the first time since 2014.
Their remarkable revival under Ole Gunnar Solskjaer continues, with the Norwegian now having overseen nine consecutive away wins, a club record.
While they celebrate, for PSG this defeat will feel like groundhog day.
For the third year running they fail to make the quarter-finals, after their defeat at the hands of Real Madrid a year ago and their humiliating collapse against Barcelona in 2017.
With what came before, this will feel even more painful for them and their Qatari owners.
Electric at Old Trafford in the first leg, Kylian Mbappe was poor here and Neymar was again missing, watching from the stands as he recovers from injury.
United were missing many more players, with Paul Pogba suspended and nine more sidelined due to fitness problems.
Despite that, they pulled off one of their greatest European results, up there with their 3-2 win at Juventus in the 1999 semi-finals, which came after they fell two goals behind early on.
Solskjaer was on the bench that night, and the Norwegian watched from the sideline here -- curiously looking like a substitute with a bib on over his jacket apparently due to a colour clash with PSG's kit -- as his side somehow went in front inside two minutes.
A dreadful back-pass attempt by PSG defender Thilo Kehrer fell into no-man's land, and Lukaku pounced on the loose ball before rounding Gianluigi Buffon and scoring. Surely PSG were not about to blow it again?
They did pull themselves together to equalise in the 12th minute, with Dani Alves releasing Mbappe in the box. As the away defence slept, Mbappe's ball across goal was turned in at the far post by Bernat, the defender scoring his third Champions League goal this season.
Now it looked as though their start to the game would prove to be nothing more than a blip.
The depleted visitors, with Eric Bailly at right-back and Fred, Scott McTominay and Andreas Pereira all in midfield, were incapable of keeping the ball. However, they were gifted another goal half an hour in.
Not closed down 25 yards out, Rashford tried a shot that was powerful but straight at Buffon.
Yet the veteran Italian, still hoping to win the Champions League for the first time aged 41, spilled the ball, and Lukaku followed in to score. It was a sixth goal in three games for the Belgian.
PSG -- so slick in France -- were wobbling again at the business end of the Champions League.
Di Maria had a goal ruled out for offside in the 56th minute and Mbappe then managed to fall when clean through late on, with Bernat sending the loose ball against the post.
It still looked as though they would hang on, but then VAR intervened, and Rashford sent United into ecstasy.