Williams rose unmarked to head home Ross Barkley's corner in a major disappointment for the visitors.
Wales defender Williams rose unmarked to head home Ross Barkley's corner in a major disappointment for the visitors, who had taken the lead through Alexis Sanchez's 12th Premier League goal of the season.
After Seamus Coleman's headed equaliser, tempers flared at half-time when players from both sides had to be separated as they exchanged angry words and pushes on their way down the Goodison Park tunnel.
In a frantic finale, Everton defender Phil Jagielka was sent off for a second bookable offence and, with goalkeeper Petr Cech attacking in the home area, Arsenal saw two shots heroically cleared off the line.
Arsenal would have taken over from Chelsea on goals scored if they had won, but they remain second, while Ronald Koeman's Everton climb to seventh after only their second win in 11 league outings.
Everton made a poor start to the contest and fell behind after 20 minutes when a catalogue of errors involving Williams gifted Sanchez the opening goal.
After Barkley lost the ball on the edge of his own area, Williams accidentally injured team-mate Idrissa Gueye with a lunging tackle that forced Jagielka to trip Francis Coquelin inches outside the box.
Sanchez took the set-piece, a powerful shot that deflected wickedly off Williams's left calf and wrong-footed goalkeeper Maarten Stekelenburg, who could only help the ball into his own goal.
But a minute before the interval Everton were rewarded for their response when Coleman headed home the equaliser.
Fellow full-back Leighton Baines swung over a deep cross from the left wing and Coleman, inexplicably unmarked six yards from goal, glanced a superb header past Cech for his third goal of the season.
Arsenal, aiming for a 15th unbeaten league outing, which would have represented manager Arsene Wenger's best run since 2011, had looked content to hit their hosts on the break during a strong opening.
After just six minutes, an overlapping sprint by Nacho Monreal saw the full-back pull the ball into the Everton area, but fractionally behind Mesut Ozil, who could only sky his shot high over the goal.
But after falling behind, Everton showed resilience, with Aaron Lennon, starting just his third game of the season, particularly threatening.
A good run and shot was well blocked by Gabriel, Romelu Lukaku blasted over from Barkley's through ball and then Lennon, again, shot over from close range after a free-kick was deflected to him at high speed.
Ozil appeared to be at the centre of the scuffle that broke out at the interval and was quickly in action again after the restart, feeding Sanchez, who chipped into the box for Theo Walcott to head wide.
Ozil should have restored his team's lead in the 54th minute when he latched onto a pass from Sanchez, but curled a shot well over from just inside the area.
Ozil was then almost played clean through on goal by Hector Bellerin, but Stekelenburg was sharp to the danger and raced from his goal to smother the ball, earning a kick for his troubles.
But Everton were not lacking in confidence after their strong end to the first half and, just before the hour, Lukaku chested down Jagielka's long ball for Barkley to drill a shot narrowly wide.
A cross from Enner Valencia also caused momentary problems in the visiting defence, the ball taking a deflection off Gabriel and forcing Cech into action.
Lukaku was becoming more of a factor and a strong run, shaking off Gabriel down the left wing, ended with a shot into the side-netting.