Lars Stindl rifled his side into a deserved lead in the 57th minute after a mistake from veteran defender Kolo Toure.
Celtic's Champion League hopes were in tatters on Wednesday as Borussia Monchengladbach produced a clinical performance to inflict a 2-0 home defeat on the Scottish champions.
The Germans arrived in Glasgow with a horrendous injury list and struggling for form following defeats by Manchester City and Barcelona in their opening Group C fixtures.
However, the Bundesliga side dominated the match from the start as they played with the kind of intensity that had helped Celtic claim a thrilling 3-3 draw with Manchester City on match day two.
Following a goalless first half, captain Lars Stindl rifled his side into a deserved lead in the 57th minute after a mistake from veteran defender Kolo Toure.
The former Liverpool defender was caught out again 20 minutes later as Andre Hahn pounced to grab the second and inflict only Celtic's fourth home defeat in 26 Champions League games.
Monchengladbach move to within one point of Manchester City in second, who lost 3-0 to Group C leaders Barcelona thanks to a Lionel Messi hat-trick on Pep Guardiola's return to the Nou Camp, while Celtic are rooted to the bottom on one point.
Celtic manager Brendan Rodgers said he had no complaint about the result.
"The best team won- I don't think there is any argument about that," Rodgers said.
"My players gave everything but tonight they were up against a top side and the only disappointment was the manner of the two goals."
Monchengladbach coach Andre Schubert was delighted to see his side pick up their first points.
"I am extremely happy to get the win and it gives us more security in the group," Schubert said.
"We still have to prove our quality at home but we have taken our first steps and gained our first three points."
Much was made of the visitors injury problems with key forwards Raffael, Thorgan Hazard and Josip Drmic missing, but their replacements were impressive as twice within the opening minutes the Celtic defence was torn apart.
Only a last-ditch block from Toure prevented a Hahn shot from reaching Craig Gordon, but the keeper didn't have to wait long to be tested.
Ibrahima Traore's effort had the Hoops stopper at full-stretch to make a fingertip save before he smothered the danger as Hahn looked to pounce on the loose ball.
The Bundesliga side seemed to cut through Celtic with ease and Jonas Hoffman was next to try his luck but his rising shot failed to trouble Gordon.
The Hoops looked to be in trouble as Stindl latched on to a poor backward header by Bitton, but the Foals' captain over hit his pass to Hahn whose shot from a tight angle was tipped over by Gordon.
The Celtic fans, who seemed to drive their side forward against Man City with their raucous support, were silenced as their side struggled to cope with the relentless pressure from the Germans.
However, they came alive again just before the break as Rogic's clever pass behind the Borussia defence played in Sinclair but the English forward sent his effort sailing over the bar.
The home support tried their best to rouse their side at the start of the second half but the Germans' slick one-touch passing play stifled their opponents.
The visitors took a deserved lead in bizarre circumstances. Toure was caught trying to usher the ball out inside the six-yard box but Hahn's perseverance allowed him to dig the ball back into ball for the advancing Stindl who lashed the ball through the legs of Gordon.
Another costly mistake from Toure allowed the visitors to increase their advantage. The experienced defender was caught dithering on the ball by Hahn who robbed him of possession near the halfway line and raced away to blast a strike into the top right-hand corner of Gordon?s net.