The bodies of the two victims were found at the foot of the Dôme de Neige des Écrins in the Hautes-Alpes.
The incident took place on Saturday afternoon while the soldiers were off duty, with rescue teams locating them at 4 am on Sunday morning.
“They were found in an area surrounded by cliffs, but we still do not know for the moment the circumstances of their fall,” the magistrate told the French press.
The two men, “equipped with crampons and climbing equipment” had left for a mountain hike “unrelated to their military function,” he said.
“We are not certain about their course, but we know that they slept in a shelter.”
A friend of one of the two men who was worried when he did not see them return alerted the police shortly after 9 pm.
The search involved a police helicopter which helped locate the bodies but it could not land due to the weather conditions at the time and foot patrol rescue services were called out.
Several hikers have died in the French Alps in recent months.
In August, a 68-year-old Frenchman was killed by falling rocks while hiking on Mont Blanc.
In early August three people from the same group killed just days after four climbers — including an Italian man and two French women — died in the French Alps in three separate accidents.