Taiwan's railways administration confirmed 22 people had been killed and said 171 people had been injured, 10 seriously, in the accident in northeastern Yilan county.
Authorities have not confirmed if any passengers remain trapped alive inside the train. An AFP reporter at the scene said that more bodies were being brought out of the wrecked carriages.
Images from the site showed the Puyuma Express train completely derailed and lying zig-zagged across the track.
All of the train's eight carriages had derailed and five had flipped onto their side near Xinma station, the railways administration said.
"There are four carriages that were overturned at 90 degrees and the worst casualties were in those carriages," Jason Lu, head of the Taiwan Railways Administration told reporters.
"The exact cause of the accident is pending investigation," he added.
Chang Yi-long, 62, told AFP that he had "heard a loud bang" as he was working at a construction site nearby.
When he arrived at the scene he saw local residents helping train passengers to escape.
Train 'shaking intensely'
One passenger told Apple Daily Taiwan that the train had put on its "emergency brake" twice on the journey and that the electricity supply had been patchy.
Hong Kong's broadcaster RTHK reported another passenger saying the train had been preparing to accelerate before overturning and had been shaking intensely, adding that some passengers were asleep at the time of the accident.
Taiwan authorities said the accident had happened at 4:50 pm local time (0850 GMT).
The railways administration said there had been 366 people on board the service en route to the southern city of Taitung.
The defence ministry said it had sent in 120 soldiers to help the rescue.
President Tsai Ing-wen described the accident as a "major tragedy" in a Tweet Sunday evening.
"My thoughts are with all the victims and their families," she added.
The crash was the worst rail accident in Taiwan since 1991 when 30 passengers were killed and 112 injured after two trains collided in Miaoli.
Thirty were also killed in 1981 after a truck collided with a passenger train at a level crossing and sent coaches over a bridge in Hsinchu.
The worst crash in recent years was in 2003 when 17 died and 156 were injured after a train on the Alishan mountain railway plunged into a chasm at the side of the track.
Eyewitness? Submit your stories now via social or:
Email: eyewitness@pulse.ng
