The baby was baptized in early November with the name of the man who killed seven people in and around the city of Toulouse in March 2012.
The self-proclaimed Al-Qaeda inspired jihadist killed three soldiers before shocking the world when he gunned down three Jewish school children and a rabbi.
“Everyone knows that Mohamed Merah is the name of the person who in 2012 was behind the terrorist acts in Toulouse and Montauban,” Nice prosecutor Jean-Michel Prêtre said on Friday.
“We consider that this would pose difficulties for the child and bring prejudices against him.”
It remains unclear if the family intentionally gave the child the same name as the terrorist, who was killed after a lengthy standoff with police after his rampage.
The prosecutor said that an investigation would be launched to determine the motives of the parents.
The baby naming system in France was only reformed in 1993 so that parents were allowed to name their children as they pleased. Registrars can only step in if they thinks the name is a bit too creative or could subject the child to mockery.
Unusual names have long been ruled out by French courts. In 2009, a family was told that their child couldn't be named after the cartoon character Titeuf, with a judge suggesting that such a name could have both personal and professional ramifications later in life.
Two years ago, two sets of parents in France were told they couldn't name their child Nutella or fraise (strawberry).