"We cannot accept the annexation or the fact that Ukraine would be controlled or invaded," Fabius, who visits the United States this week, said in an interview published on Sunday in The Washington Post.
"On the other hand, we should not go to war with Russia. In between, you have diplomacy and sanctions to exert pressure."
Asked about the prospect of more sanctions against Moscow, Fabius said it would depend on Moscow's reaction to the presidential elections in Ukraine, where Russia is accused of supporting pro-Moscow rebels in the east of the country.
"If actions are taken to make them impossible or to try to delegitimize them, we shall have to increase sanctions," Fabius said.
He admitted that the West cannot read Russian President Vladimir Putin's intentions in Ukraine, where rebels loyal to Moscow have taken over dozens of towns and claimed a massive turnout on Sunday in a vote they held to split east Ukraine into two independent republics.
"It is difficult to know exactly what he has in mind," Fabius told the Post.
"But we have to be clear what we have in mind. It is clear that the annexation of Crimea is unacceptable. You cannot accept that one country annexes another."