Warnings that ballooning costs could hit a staggering $30 billion have cast a shadow over Tokyo's preparations.
IOC vice president John Coates, who heads the coordination commission for the Tokyo Olympics, insisted Tokyo governor Yuriko Koike and local organising committee head Yoshiro Mori could find common ground despite clashing over proposed venue changes.
"Yes, I'm confident that governor Koike and president Mori and the two organisations that they lead can work together," Coates told a news conference after concluding a two-day visit to Tokyo.
"They wouldn't have participated in the four-party political working group could they not."
Koike and IOC president Thomas Bach last month agreed to hold discussions involving the IOC, local organisers, the Tokyo metropolitan government and the Japanese central government to explore ways to save money.
Warnings that ballooning costs could hit a staggering $30 billion -- four times the initial estimate, and almost triple that of the 2012 London Olympics -- have cast a shadow over Tokyo's preparations.
But a pledge this week by Japanese organisers to put an $18 billion cap on the overall costs did not go far enough, according to Coates.
"The IOC are not in a position yet to accept a budget of $20 billion," he said.
"This was a ceiling budget, a consolidated budget. The IOC just isn't going to sign off on a budget which we think exceeds the cost of the budget the Games can be staged for."
Coates warned that Tokyo's snowballing costs could scare off potential host cities considering bids for future Olympics.
"(Accepting such a budget) would be giving the wrong impression," he said. "It would also not help us in terms of other candidate cities prospectively so there is still work to be done.
"It's clear we all have a common objective of saving costs."
Koike earlier this week dropped a proposal to move the 2020 Olympic canoeing venue outside Tokyo, promising instead to cut construction costs on that and another facility.
Her proposal would have taken the event 400 kilometres (250 miles) north of Tokyo to Miyagi prefecture, one of the areas hardest hit by Japan's 2011 tsunami disaster.
The idea was shot down by Mori and Tokyo 2020 organisers, who argued that such a move was impractical and could even increase costs.
Asked about her apparent climb-down, Koike gave a cryptic reason for escalating costs.
"Look, in the end the taxpayer will end up footing the bill and I have a responsibility to them," she said.
"There are lots of black-headed mice involved here," she added, in a veiled reference to the failure to curb costs.
Koike also said Tokyo will build a new aquatic centre with 15,000 seats costing less than the original plan for a 20,000-seat arena.
However, a decision on whether to build a new volleyball venue in Tokyo or use an existing one in Yokohama, south of the capital, was postponed until later this month.