The party had ordered him to leave office, but the country’s constitution prevented them from being able to force him from office.
However, he would have faced a vote of no confidence by parliament if he had stayed, scheduled to take place tomorrow.
Mr Zuma, 75, made the announcement in an address to the nation at the end of a rambling speech.
He said he was indebted to the ‘glorious’ ANC but disagreed with the way the party had acted towards him and ranted at the ill treatment he said he had received.
“As I leave I will continue to serve the people of South Africa as well as the ANC, the organisation I have served all of my life,” he said.
The ANC said it has accepted Mr Zuma’s resignation and that it has provided “certainty to the people of South Africa”.
Mr Zuma had earlier proposed to resign with a notice period of three to six months, according to ANC secretary general Ace Magashule, but the party had requested he stepped down “urgently… for the sake of the country”.
World news in pictures
A string of corruption allegations have dogged Mr Zuma’s two terms as president, relating specifically to a 1999 arms deal, a rape accusation, scalding black people “who become too clever”, and his relationship with the powerful Gupta family.
A report published in 2016 alleged that the Guptas had tried to influence the appointment of cabinet ministers and were unlawfully awarded state tenders.
He and the Guptas have denied any wrongdoing.
Despite once being an anti-apartheid hero, Mr Zuma resigns as a symbol of corruption leaving behind a weakened South African economy.
The man who is expected to replace Mr Zuma is Cyril Ramaphosa, a confidante of the late Nelson Mandela and unionist turned business leader, who immediately became acting president following the resignation.
He was named the leader of the ANC after a close vote in December, defeating Mr Zuma’s ex-wife Nkosazana Dlamini-Zuma.
Mr Ramaphosa said last week that uncertainty over Mr Zuma’s future as president had damaged the ANC, the party which led South Africa out of apartheid.