The teams scored three tries each but Scotland were 17-12 ahead at half-time and after regaining the lead midway through the second period.
The teams scored three tries each but Scotland were 17-12 ahead at half-time and after regaining the lead midway through the second period, they held on gamely for the win.
It meant Scotland finally tasted victory against the Wallabies after going down by just one point in their previous two meetings.
In a tense finish, the Wallabies tried desperately to conjure the converted try needed for victory but the Scottish defence refused to yield.
"It's really a proud day for us all," said coach Gregor Townsend, who has now won his first two games in charge after last week's victory over Italy in Singapore.
"For everyone involved in Scottish rugby to blow away those frustrations of the last couple of seasons and come away with such an important win is a credit to the players.
"It's down to the work over the last couple of seasons and the way the players have stuck to their task at training, they've learned from close defeats."
It was a triumph for captain John Barclay, who has been on the wrong end of close recent results against Australia.
The fifth-ranked Scots' last success against the Wallabies, currently the world number three, was also in Australia when they edged a wet, try-less game 9-6 on their last visit in 2012.
"For some reason whenever we've played each other in the last couple of years it's been very close. For us it's nice to end up on the right side of it," Barclay said.
The Wallabies were their own worst enemies in a ragged first half, gifting the Scots two tries to trail at half-time with two signature tries by fullback Israel Folau keeping the hosts in the contest up to half-time.
"I thought we were half-a-yard away from the game all the match," Wallabies coach Michael Cheika said.
Skipper Michael Hooper added: "They're a scruffy team to play. They get their feet into the rucks, get their hands into the rucks and we couldn't find our shape well tonight.
"I'm disappointed we couldn't get over the line... You look at Izzy's (Folau's second) try, there's potential across the whole park for us. It just didn't click tonight."
Australia made a sloppy start and a poor pass from scrum-half Will Genia to hooker Tatafu Polota-Nau was snapped up by inside-centre Duncan Taylor, who ran over from 30 metres.
Fly-half Finn Russell's conversion gave the Scots a surprise 10-0 lead after 15 minutes.
The Wallabies hit back minutes later when Bernard Foley found Folau with a huge gap out wide for the fullback to storm over.
But yet again Scotland seized on a Wallabies mistake when the influential Russell charged down Genia's clearing kick for a gift try to give the visitors a 17-7 lead after 27 minutes.
The Wallabies answered with a breathtaking second try for Folau, who easily out-jumped fullback Greig Tonks to score off Foley's pinpoint kick to the right wing.
That left the Wallabies trailing 17-12 at half-time but they returned with more intent and hit the front 19-17 when Genia burrowed over for a try from a ruck after 57 minutes.
But the Scots roared back with a well-taken try to openside flanker Hamish Watson, after winger Lee Jones leapt over an attempted tackle, as Scotland outnumbered the Australian defence.
In a tight finish, Scotland held out the Australians who tossed away another opportunity when lock Rory Arnold lost the ball in a lineout close to Scotland's line.