The attack is said to have taken place as the child was cycling towards a train station with her mother.
The girl was riding her bicycle a short distance in front of her mother when the man approached, and the mother was quoted by the TVBS broadcaster as saying she initially thought he was going to help her cycle up the hill.
When he pulled out a large knife and attacked the girl. Her mother tried to fight the attacker off, but was unable to do so.
People passing by went to help her, but the girl had already been decapitated, according to Apple Daily.
The girl’s grandmother, identified only as Mrs Liu, told a local TV channel: ‘I could not believe it. How can something so cruel happen?’
Police told the Taiwanese Central News Agency (CNA) that the attacker named only as Wang, 33 had a history of mental illness.
Hours later, an angry crowd gathered outside the police station where the man was taken.
The government-run Central News Agency said the 33-year-old had an arrest record for drug crimes and had been treated for mental illness.
A politician has called for the death sentence for the man.
READ: Musician decapitated girlfriend, had sex with her head while wearing her dress