Modi's remarks come just days after a Muslim teenager was stabbed to death on a train after being accused of carrying beef.
Modi's remarks -- his first on vigilantism in nearly a year -- come just days after a Muslim teenager was stabbed to death on a train after being accused of carrying beef.
"Killing people in the name of Gau Bhakti (cow worship) is not acceptable. This not something Mahatma Gandhi would approve," Modi said.
India has been reeling from a spate of vigilante murders in recent months, targeting Muslims and low caste Hindus accused of killing cows or consuming beef.
In the latest high-profile incident, 15-year-old Junaid Khan and three of his brothers were attacked last week in an apparent row over seats as they travelled home by train from New Delhi.
Police have arrested four men over the attack.
One of Khan's brothers said the attackers accused them of carrying beef, a meat popular among many Indian Muslims and low caste Dalits, but shunned by most upper caste Hindus.
Khan's murder drew thousands to the streets across Indian cities on Wednesday, with demonstrators calling for an end to the wave of mob violence under the slogan "not in my name".
Modi expressed his "pain and anguish at the current environment" in India, listing incidents of vigilante violence.
"No person in this nation has the right to take the law in his or her own hands in this country. Violence never has and never will solve any problem," he said.
Rights groups have warned of a culture of impunity for crimes committed against Muslims and urged Modi's Hindu nationalist government to act.
"The pattern of hate crimes committed against Muslims with seeming impunity... is deeply worrying," said Aakar Patel, executive director of Amnesty International India, in a statement this week.
The rights group said at least 10 Muslim men had been lynched or killed in public since April in suspected hate crimes.
Last year Modi criticised the vigilantes and urged a crackdown against groups using religion as a cover for committing crimes.
But critics say vigilantes have been emboldened by the election in 2014 of his right-wing Bharatiya Janata Party, and have urged the government to be more vocal in condemning the attacks.
The slaughter of cows and the possession or consumption of beef is banned in most Indian states, with some imposing life sentences for breaking the law.
There has been a spike in attacks by 'cow protection' vigilante groups, who roam highways inspecting livestock trucks for any trace of the animal.
In April, a Muslim man was beaten to death by a mob in Rajasthan state after they discovered cows in his truck. The man was a dairy farmer transporting milk cows.
The following month, two Muslims were beaten to death on suspicion of stealing cows. In both cases, police were accused of failing to act quickly enough to protect the victims.
In the Rajasthan attack, in which 200 vigilantes set upon trucks transporting cattle along a highway, police first arrested 11 of those beaten for alleged permit violations instead of rounding up the perpetrators.