The reigning champions looked to dominate the opening proceedings without really creating anything concrete.
It was Sweden who took the lead against the run of play as Ola Toivonen lifted the ball of Manuel Neuer after half an hour.
Mario Gomez was introduced at half time and immediately made an impact as he set up Marco Reus for the equaliser.
Germany looked like they were nearing a World Cup exit until Toni Kroos whipped in a 95th-minute free-kick to give them a last-gasp win.
Here are our player ratings from Germany 2-1 Sweden.
Goalkeeper: Manuel Neuer
Made two fantastic saves to bookend a disappointing first half for the Germans. Arguably could have done better for Sweden's goal. 7
Right-back: Joshua Kimmich
A constant threat down the right as he capitalised on the lack of width from Sweden, but defensively open. 7
Centre-back: Jerome Boateng
A shocking day all round. Fortunate not to concede a penalty when bringing down Marcus Berg in the opening exchanges. Constantly strayed from position and allowed Sweden to get in too easily. Sent off with 10 minutes to go. 4
Centre-back: Antonio Rudiger
Made a couple of crucial headed clearances, but struggled to contain the Sweden attackers. 6
Left-back: Jonas Hector
Tasked with providing width on the left, Hector did a good job of stretching play. 6
Midfield: Sebastian Rudy
Operated efficiently from deep before being taken off. Had a couple of chances to let fly with a shot but never quite capitalised. 6
Midfield: Toni Kroos
Dictated play from deep, being allowed space to move the ball around. Left the defence overly-exposed leading to trouble at the back. Pulled it out the bag with a wicked free-kick in the 95th minute. 7
Right wing: Marco Reus
An effective presence as he switched flanks and rotated with the other Germany attackers to cause constant problems. Did well to get Germany's equaliser. 7
Attacking midfield: Thomas Muller
Always an incredibly wily player - using his experience and anticipation to make sure he's in the right place at the right time. 5
Left wing: Julian Draxler
Had an opportunity to take the lead in the opening minutes but couldn't convert from close range. 6
Striker: Timo Werner
Used his pace to cause problems as he floated out of his number nine position. Struggled to make an impact against a largely resolute Sweden defence. 5