The family's car was hit from behind by an empty cattle truck at 14:30 on Wednesday afternoon on the a country road (784 see map below) near Pluguffan, a village near the town of Quimper in rural western Brittany.
It is believed the baby was killed instantly in the crash.
The 37-year-old father was removed from the wreckage with life-threatening injuries and taken to hospital in Quimper, the capital of the Finistere department. But he succumbed to his injuries and died on Thursday.
The mother, 36, who was travelling in the front seat of the British registered car, was given intensive treatment by a medical team at the scene before being airlifted to a hospital in city of Brest. She is believed to be in a serious but stable condition.
The Local understands the baby's grandparents who were on holiday with the couple, were travelling in a separate car in front.
At the scene of the crash there was evidence of tyre tracks where the truck had braked over a distance of 20 metres before it collided with the rear of the car as it was preparing to turn left, sending the vehicle skidding onto the other side of the road, reported the Le Telegramme newspaper.
This Google image shows the crossroads where the accident happened.
“It was a very violent collision,” a member of the Gendarmes in Quimper told The Local. “The back of the car was destroyed.
“We are still investigating the crash and have not yet gathered all the elements together,” said the gendarmes who added that specialist investigators will be looking at the speed the truck was travelling as well as whether the lorry was at a safe distance from the family's car.
The road is not known as an accident black spot.
Christian Bargain the deputy mayor of Pluguffan who visited the scene of the crash told The Local authorities are struggling to understand how the fatal crash could have happened.
'We don't understand how the truck could have hit the tourists' car like that but an investigation is underway,” Bargain said.
The 27-year-old driver of the cattle truck was taken to Quimper hospital where he was being treated for shock.
The public prosecutor's department on Wednesday ordered a judicial assessment to find out the circumstances behind the accident.
The young family had been holidaying in the nearby town of Plogastel-Saint-Germain, an area popular with British holidaymakers.
The high number of road deaths in France has been a constant issue for French governments over the years.
Last year The Local reported that France's road mortality rate had increased in 2015 for the second year in a row, something the country hadn't seen for 35 years. And the fatality rate for 2016 was even higher, with a death toll of 3,469.
The former Interior Minister Bernard Cazeneuve said in January last year that one of the primary reasons for the higher toll was increase in road traffic but also drivers taking risks.
“The rise also reflects an increase in risky behaviour that can lead to serious accident fatalities,” he said.
The former minister also said French drivers were “too relaxed” when it came to following the road rules, and that many motorists continued to drive “at excessive speed”.
A study in April 2015 from BVA found that 75 percent of French people acknowledged that they don't always stick to the speed limit.
“This is not acceptable! This behavior is irresponsible and may be criminal” Cazeneuve had said.