Jacques Toubon, France's human rights pointman, said he was "shocked and stunned," announcing a probe into a case which highlighted France's tense relationship with the Roma, most of whom come from eastern Europe.
The bitter row started after reports that the conservative mayor of Champlan, about 23 kilometres (14 miles) south of Paris, had denied the family of the two-month-old a burial on the grounds that the municipal cemetery has "few available plots".
"Priority is given to those who pay their local taxes," Christian Leclerc was quoted as saying by Le Parisien daily on Saturday.
Mayor Christian Leclerc told AFP that he had been misrepresented, saying he was "really sorry" for the controversy and that "at no stage was I opposed to the burial."
Late Sunday he issued a further statement, insisting: "I very much want that the burial should take place in Chmplan, the town where (the family) lives."
The row, touching a raw nerve in France over the treatment of the Roma, came as it emerged that another two-month-old Roma baby died in a shack near the train station of the northern city Lille amid freezing temperatures on Thursday.
Fifty people gathered in silence at the station late Sunday to mark her death.
Leclerc's attempt to defend himself did little to dampen the anger over his reported words.
Prime Minister Manuel Valls took to Twitter to say: "Refusing a child a burial because of its roots is an insult to its memory, an insult to France."
Earlier Sunday, Laurence Rossignol, a junior government minister for the family, said on Twitter the refusal was "an inhumane humiliation" for the family, adding the hashtag "honte", meaning "shame".
The baby, identified only as Maria Francesca, was born on October 14 and died in the early hours of December 26.
"The mother tried to breastfeed her at 5:00 am and the little girl was cold. She was dead," said Marie-Helene Brelaud, a member of the ASEFRR association, which supports Roma families in the region.
Most of France's roughly 20,000 Roma, who are largely from Romania and Bulgaria, have little or no access to basic amenities.
'A question of humanity'
Maria Francesca was rushed to hospital in nearby Corbeil-Essonnes, where she was pronounced dead from sudden infant death syndrome.
The Lille baby was also called Francesca. Roma people sometimes name their children after the place they were born.
The family asked an undertakers in Corbeil-Essonnes to request permission from the authorities to lay the infant to rest but, according to the firm's manager Julien Guenzi, the mayor refused "without explanation".
"He doesn't have to justify himself, but responses like that are very rare," Guenzi told AFP.
Leclerc told AFP: "There was a choice between Corbeil and Champlan. I agreed on Wednesday morning to either scenario. I have a text message proving this."
The mayor of Wissous, a few kilometres away, has since offered to host the burial, telling AFP it was a "a question of humanity".
"The pain of a mother who carried a child for nine months, and lost her after two and a half months must not be worsened," mayor Richard Trinquier told AFP.
ASEFRR said it would cover the costs of the funeral, expected on Monday. The family has already paid for a coffin.
The child's parents are Romanians in their mid-30s who have lived in France for at least eight years, according to supporters.
They have two boys, aged five and nine, who are attending school in Champlan.
The family live in a makeshift settlement on the outskirts of Champlan without electricity or running water, very close to Paris's Orly airport.
Successive governments have drawn fire for demolishing numerous camps and evicting families with children, although some in France have cheered the tough approach.