Eight people were injured in the accident which took place as the train travelled from the coastal city of Nice to the popular tourist town of Digne-les-Bains along a narrow, winding, and sometimes breathtakingly steep track.
A 49-year-old Russian woman was killed along with an 82-year-old Frenchwoman living in the Alps region, said local prosecutor Stephane Kellenberger, adding that 34 people had been on board the train.
The Russian woman's husband, in deep shock, was among at least eight people injured.
Witnesses said an enormous boulder had struck the train, violently throwing one of its two carriages off the track.
Images of the accident show the carriage hanging off the edge of a ravine, with a large crater in its side as the other carriage teeters on the edge.
"It was as if the boulder fell from the sky, like in an earthquake," said traveller Jean-Jacques Messaoud, 47, who said he had seen a victim bleeding heavily.
Another passenger, Floriane Bonnet, was at the back of the train when the accident happened.
"I didn't understand, I was thrown to the left and I saw the first half of the train go down the embankment. I tried to break a window but couldn't manage it so I left through the front."
Firefighters from around the mountainous region were drafted in to help in the rescue operation in a remote and mountainous area difficult to access due to the snow and steep slope.
A total of 110 firefighters and 32 vehicles were deployed, as well as two helicopters.
The railway, known as the Train des Pignes (Pine Cone Train), is hugely popular among tourists, taking them on a picturesque 151-kilometre (90-mile) journey from the sea to the mountains of Haute-Provence.
Over 100 years old, the railway crosses some 30 bridges and viaducts and 25 tunnels through valleys and gorges on an often hair-raising journey at up to 1,000 metres above sea level. The train used for the line is modern, having been put in service in 2011, according to a local transport official.
The train's name is said to come from the fact that it was so slow it allowed travellers to get off and pick up pine cones from the side of the tracks.
Others believe it was so named because pine cones were used as tinder to fire up the original steam engine when the line was opened in 1911.
French Transport Minister Frederic Cuvillier was to visit the scene of the accident afternoon.
In July last year an intercity train derailed as it sped through a station about 25 kilometres (15 miles) south of Paris, leaving seven dead.