The blaze broke out on Wednesday afternoon as worshippers gathered near the town of Medina Gounass in the southeastern region of Tambacounda, a senior official with the firefighting service told AFP. The cause is as yet unknown.
Resident Aziz Thierno Belly Ba recounted witnessing the pilgrims' ordeal to the Observateur newspaper.
"The fire burned through everything in its path. Only the modern tents reserved for the marabouts (religious leaders) were relatively spared from the fury of the flames," he said.
Images of billowing smoke, the charred corpses of animals and burnt-out cars circulated online.
While some victims were badly burned others were hurt in the panicked stampede triggered by the blaze, according to the firefighters' official.
Around 20 of the injured are in a serious state and are being treated in hospital in Tambacounda city, about 80 kilometres (50 miles) away.
Senegal's President Macky Sall paid his respects to victims in a press conference late Wednesday and is expected at the site on Friday following an imminent visit by his interior minister.
The fire ripped through straw shelters constructed for the multi-day event, according to Senegal's local press, allowing it spread rapidly through an open-air gathering packed with people.
Members of Senegal's powerful Tijaniyya Muslim brotherhood gather in their hundreds, even thousands, to worship at the site every year. Pilgrims were weighed down with food and luggage, the local press reported.
Around 95 percent of Senegal's population is Muslim and most men join Sufi brotherhoods that combine Islam with distinctive local beliefs.
An incident at the same site in 2010 caused the deaths of six people, Le Quotidien newspaper reported on Thursday.
Senegal's poor record on fire safety was also thrown into the spotlight in 2013 when a fire in a Koranic school killed nine children, triggering an outcry and calls for tighter regulation.
Eyewitness? Submit your stories now via social or:
Email: news@pulselive.co.ke
