Driver was the first celebrity to quit the global charity as accusations over staff paying for prostitutes during crises widened.
"All I can tell you about this awful revelation about Oxfam is that I am devastated," the Oscar-nominated actress wrote on Twitter.
"Devastated for the women who were used by people sent there to help them, devastated by the response of an organisation that I have been raising awareness for since I was nine years old."
A spokesperson for the charity said Driver's decision "saddens us deeply, but we also understand and respect her choice," according to a statement quoted by The Telegraph newspaper.
Oxfam has denied a cover-up over the use of prostitutes by senior aid workers in Haiti following the country's devastating 2010 earthquake.
Since the scandal first broke last week further allegations have been made about the organisation's mishandling of sexual assault in South Sudan.
Oxfam deputy chief Penny Lawrence resigned on Monday in the wake of the scandal, while the British government has called the charity to account for its actions.
Eyewitness? Submit your stories now via social or:
Email: eyewitness@pulse.ng
