Rosberg joined an exclusive club by becoming only the second son of a former champion to take the title.
The 31-year-old Rosberg concluded the longest season in the sport's history with a nail-biting second place under intense pressure after leader and ultimate race winner Hamilton had slowed the field to create a difficult finish.
Hamilton reeled off his 10th win of the season - the most by any driver not to win the title -and the 53rd of his career, but his gamesmanship upset his team and his team-mate in the closing laps.
His final lap was nine seconds slower than the lap that carried him to pole and was clearly designed to push Rosberg into a vulnerable position as Sebastian Vettel closed in in his Ferrari.
Hamilton ignored two instructions from the team to increase his speed at the front, but was unable to create a situation in which Rosberg could be attacked and passed.
Rosberg joined an exclusive club by becoming only the second son of a former champion to take the title, his father Keke having won the championship 34 years ago with the Williams team.
Briton Damon Hill did the same when he won the title with Williams, following his father Graham's achievements, in 1996.
Rosberg also became Germany's third champion.
The tension was palpable as the drivers headed for the podium after the race in which Vettel finished third for Ferrari ahead of Dutchman Max Verstappen and his Red Bull team-mate Daniel Ricciardo.
Rosberg celebrated by giving F1 supremo Bernie Ecclestone a massive bear hug, lifting the diminutive octogenarian high into the air, as Hamilton stared glumly at the floor.
Rosberg reliving the final tense laps, said: "With the guys (Vettel and Verstappen) coming up behind me the end was not the most enjoyable.
"I'm very proud to have done the same feat as my Dad achieved."
On the podium Hamilton's good grace returned and he gave his former teenage go-carting companion a handshake and a hug.
"Big congratulations to Nico, good job man," said Hamilton.
Of his own attempted spoiling tactics, Hamilton said simply: "We were fighting for the championship, I was in the lead so I control the pace. That is the rules."
Briton Jenson Button, the 2009 champion, and Brazilian Felipe Massa made emotional exits after competing in their final races of long and successful careers.
Button was forced into retirement after only 12 laps while Massa placed ninth.
As the sun descended at the end of a hot, dry and dusty day at the Yas Marina circuit, Hamilton made a perfect start from his 61st pole position to lead away from Rosberg.
In a frantic opening lap, Verstappen spun after touching the Force India car of German Nico Hulkenberg, dropping him to 17th as he rejoined and advanced through the pack.
After a series of pit stops and mid-field scraps Hamilton was left in control by lap 34 ahead of Rosberg by 1.3 seconds with Verstappen third and Vettel, setting fastest laps, climbing back to fifth to launch himself in pursuit of the two Red Bulls.
"Ok, Lewis, we really need to pick up the pace," Hamilton is told by his team. "Vettel is a threat." For the defending champion, it was the news he wanted to hear, even if for Mercedes it presented a dilemma.
With 10 laps remaining, it was obvious Hamilton was driving as cautiously as he could to compress the field and create a threat behind Rosberg.
"Ok, Lewis, this is an instruction we need 45.1 this is for a win, said the team on Hamilton's radio on lap 46. "You should just let us race," he replied.
Mercedes technical team chief Paddy Lowe followed on the radio. "Lewis, this is Paddy. We need you to pick up the pace to win this race. That's an instruction."
"I am actually in the lead right now," said Hamilton. Lowe's face on the pit wall fell in despair.
On lap 51, Vettel passed Verstappen for third with a bold move to increase the tension, leaving Hamilton backing Rosberg back towards the charging Ferrari, but he survived to take his title.