The ANC will hold a conference in December to pick a new party leader who will be its candidate for the 2019 elections.
Gordhan -- widely seen as a bulwark against government corruption in South Africa -- also appeared to back deputy president Cyril Ramaphosa to succeed Zuma.
"I think there are many of us who are extremely worried that if we continue as we are in the African National Congress (ANC), we are likely to lose the 2019 elections," Gordhan told CNN late Thursday.
"If we, like any business organisation or political organisation, fail to muster the right leadership, stick by the right kind of values and do the right thing... then of course we will, as an organisation, fail."
The party slipped to 55 percent of the vote in last year's local elections -- its worst-ever result.
The ANC will hold a conference in December to pick a new party leader who will be its candidate for the 2019 elections -- and Gordhan gave his clearest indication yet that he will back Ramaphosa to succeed Zuma.
"Mr Ramaphosa has been extremely brave. He has spoken truth to power at difficult times," said Gordhan.
"He has demonstrated that he is a leader of great mettle that would serve South Africa well into the future."
Zuma, 75, is due to step down as head of the ANC in December, and as president ahead of the 2019 general election.
Among many corruption scandals, he has been accused of being in the sway of the wealthy Gupta business family, allegedly granting them influence over government appointments and contracts.
His midnight cabinet reshuffle last month unleashed a rebellion within the ANC with senior party figures, including Ramaphosa, speaking out against Gordhan's sacking.
The cabinet overhaul cleared out many of Zuma's critics and placed loyalists in key positions.
"We have had a report... last year which mapped out some of the kind of corruption between certain families and certain state institutions it would appear is taking place," said Gordhan.