A community leader said soldiers were taken by surprise when Boko Haram militants fired rocket-propelled grenades.
Boko Haram on Monday claimed it killed 20 soldiers in northeastern Nigeria, in violence marring the celebrations following the release of some of the kidnapped Chibok schoolgirls.
The group, using the name Islamic State West Africa Province (ISWAP), said in a statement that it "stormed a joint position of the armies of Nigeria and Niger" in the town of Ghashghar in northeast Borno State.
It claimed it killed 20 soldiers and wounded dozens of others in "fierce clashes" that erupted during the attack on Sunday, highlighting the threat the group still poses in the region.
A community leader in the Ghashghar area confirmed the raid, saying soldiers were taken by surprise when Boko Haram militants fired rocket-propelled grenades.
"I was called by two of my contacts ... informing of an attack late yesterday on the joint military personnel from Nigeria and Niger stationed in Asaka village, around 15 kilometres (nine miles) from Ghashghar," a community leader from the area said.
"According to them, Boko Haram launched an attack with heavy guns and rocket-propelled grenades on the soldiers. The attackers came in eight vehicles," said the leader, who asked to remain anonymous fearing reprisal from the military.
"The soldiers suffered huge casualties because they were taken off-guard. They never anticipated such an attack," he said, adding that the jihadists stole six military vehicles and weapons.
Boko Haram has been waging a seven-year uprising against the Nigerian state, claiming more than 20,000 lives with the insurgency spilling over the west African nation's borders into neighbouring states.
ISWAP is a faction of Boko Haram, whose leader Abu Musab al-Barnawi was appointed by the Islamic State jihadist group in August.
Boko Haram's longtime chief Abubakar Shekau still claims he is in charge, however, as rival factions vie for control.
News of the raid came less than a week after 21 schoolgirls kidnapped by Boko Haram from the northeast Nigerian town of Chibok in 2014 were released.
Even as town residents celebrated the girls' return, experts warned the nation must brace for more violence as the government negotiates for the others.
The remaining 197 girls appear to be spread out in different jihadist camps, though Shekau has claimed in the past that several died in Nigerian air strikes.
Nigerian presidency spokesman Garba Shehu has said that the ISWAP faction of Boko Haram has claimed to have "83 or more girls to release on negotiation."
"Whenever the Nigerian government will be negotiating with one faction, the other one will be calling for attention," Martin Ewi, a researcher at the Pretoria-based Institute for Security Studies, told AFP.
"You will be hearing about negotiations and you'll be hearing about attacks."