A new political party set up by former TV anchorwoman Yuriko Koike, elected Tokyo governor in a landslide vote last year, was able to capitalise on this to seize 49 seats out of 127, becoming the leading group in the capital's assembly in Sunday's election.
"We have to take (the result) seriously as a severe criticism against our party the LDP," a humbled Abe told reporters Monday morning, after its seat count shrunk from 57 to just 23, a record low.
"I'm determined to reshape the party to work together and win back trust among the public through realising achievements," he said.
Abe was elected prime minister in late 2012 with a mandate to revitalise the world's third-largest economy.
But the 62-year-old is under fire over claims he showed favouritism to a friend in a business deal, which the prime minister has denied.
Those came a few months after he was forced to deny connections to the controversial director of a school that had purchased government land at a huge discount -- and counted Abe's wife as its honourary principal.
Other problems have included the perception his government has railroaded sensitive legislation through parliament and gaffes by members of his cabinet.
Just days before the election, Abe's controversial defence minister Tomomi Inada -- in the name of her ministry and the Self-Defence Forces, Japan's military -- called on Tokyo voters to support the LDP in the vote.
The comments drew widespread condemnation and the outcry forced Inada to retract the remark.
Koike, 64, a former member of parliament and defence minister, is widely seen as having ambitions to return to national politics and has been touted as a strong candidate to be Japan's first female prime minister.
After the election, her Tomin First party increased its assembly seats to 55 by admitting six further candidates who had run as independents.
And Komeito -- a moderate party backed by a Buddhist-linked group -- has long sided with Abe in national politics, but is cooperating with Koike's local party in Tokyo.
Its 23 seats, plus another belonging to a smaller party, give Koike and her allies effective control of 79 seats in the assembly, a comfortable majority.
"The defeat comes from the Abe administration's missteps, and (the Tokyo vote results) would encourage non-Abe factions inside the LDP to move towards seizing power,"Sadafumi Kawato, professor of politics at the University of Tokyo, told AFP.
"This could threaten the existence of the Abe administration," Kawato added, noting he faces a party leadership vote in September next year, and must call an election for the powerful lower house of parliament by December 2018.
His plans to make the first-ever changes to Japan's post-war pacifist constitution by 2020 "may be delayed significantly without high public support for his government," Kawato said.
Leading national dailies blamed the administration for Sunday's rout, pointing to a lack of accountability amid the scandals and the forcing through of a parliamentary vote last month on a controversial anti-terror bill.
"The government would never win back public trust... unless it overhauls its arrogant nature", the conservative mass circulation Yomiuri daily said on its front page.
Despite Koike's surge and talk of her possibly transforming national politics, media pointed out that she herself faces significant challenges.
The liberal Asahi daily noted in an editorial that Koike has "few tangible achievements" so far and must steer the city through its hosting of the 2020 Tokyo Olympics, which have been plagued by scandals and cost overruns.
