A convoy of about 30 armoured vehicles transporting some 300 Malian and French troops moved into the town early Monday morning without meeting any resistance, an AFP journalist said.
The town, 400 kilometres (250 miles) from the capital Bamako, has been the theatre of air strikes and fighting since it was seized by Islamists a week ago.
A colonel in the Malian army had said earlier that a "fringe of the Diabaly population adheres to the jihadists' theories and we must be very careful in the coming hours".
French television footage from Diabaly has shown charred pick-up trucks abandoned by the Islamists amid mud brick homes.
One resident said the rebels had fled the town which was abandoned by many of its residents, and those remaining lacked food and other essentials.
As French troops began a deployment towards the Islamist-held north, moving into the strategic central towns of Niono and Sevare on Sunday, Paris said the aim of the 11-day old military offensive was total victory over the Islamists.
"The goal is the total reconquest of Mali," French Defence Minister Jean-Yves Le Drian said in televised remarks. "We will not leave any pockets" of resistance.
France swept to the aid of the crippled and weak Malian army on January 11, a day after the hardline Islamists who have occupied the country's vast north since April 2012 made a push south towards Bamako.
In retaliation, a jihadist group run by a former leader of the regional Al-Qaeda franchise Mokhtar Belmokhtar, attacked a gas plant in neighbouring Algeria.
A 72-hour stand-off at the complex ended in scores of deaths Saturday.
Officials have warned that the death toll from the bloody rescue operation could rise.
"I fear that it (the death toll) may be revised upward," Algerian Communications Minister Mohamed Said told a radio station, as details slowly emerged over the hostages who had died.
On Sunday, the assailants, calling themselves "Signatories in Blood", vowed "more operations in all the countries which have taken part in the crusade" against northern Mali, if it did not halt immediately.
The Islamists in Mali, who have proved a well-armed and formidable foe, were reported Sunday to be abandoning some of their positions and converging on the mountainous region of Kidal, their northernmost bastion, 1,500 kilometres (900 miles) from Bamako and near the border with Algeria.
Kidal was the first town seized by an amalgam of Islamist militants and Tuareg separatist groups in March last year. The two sides then had a falling out and the Islamists have since gained the upper hand in the desert north.
'Only 150 African troops have arrived'
Meanwhile the planned deployment of nearly 6,000 African soldiers continued slowly into Bamako, hampered by cash and logistical constraints. Only 150 African troops had arrived by Sunday.
Senegal, Benin and six other west African nations are contributing to the African mission which is expected to take over the baton from France, and Chad has also pledged 2,000 soldiers.
The head of the Commission of regional west African bloc ECOWAS, Desire Kadre Ouedraogo, estimated the cost of an African offensive against the armed Islamist groups at about 500 million dollars (about 375 million euros).
The European Union has pledged 50 million euros to the International Support Mission for Mali (MISMA).
On the ground in Mali, French troops said they were buttressing their positions.
"The deployment towards the north… which began 24 hours ago, is on course with troops inside the towns of Niono and Sevare," Lieutenant Colonel Emmanuel Dosseur told reporters, referring to key frontline outposts.
Niono is about 350 kilometres (220 miles) northeast of the Malian capital and 60 kilometres (40 miles) south of Diabaly.
Sevare has a strategically important airport about 630 kilometres (390 miles) northeast of Bamako that could help serve as a base for operations further north.The western region is home to what are considered the most martial and fanatical Islamists.