The parents of the three newborns, who died on different days in early December, have filed criminal complaints for manslaughter against the hospital, located in the town of Chambery in southeast France.
Analyses of several unused IV bags from the batch used to give the newborns nutrients while in the hospital's neonatal intensive care ward showed they all contained bacteria, the hospital told the parents.
"We're lodged the complaint to find out what happened. There was a failure in the system and this failure needs to be found so it never happens again," said the 37-year-old father of one of the infants, who gave his first name as Laurent.
Local prosecutors investigating the case said they would make no comment before next week.
France's health minister, Marisol Touraine, announced all the batch that included the affected IV bags had been withdrawn.
She called the deaths "an extremely grave accident".
The director of the hospital, Guy-Pierre Martin, told a news conference that the contaminated IV nutrient bags all came from a French pharmaceutical firm, but he refused to identify it, saying that was up to French government authorities.
He did not identify the lethal bacteria either.
He said the hospital would assume its responsibilities if the investigation found it liable.