Undermined by stubbornly high unemployment and stagnant growth in France, Hollande's party had suffered a string of defeats in by-elections held since he became president in May 2012.
By the time Paris was hit by deadly Islamist attacks last month, Hollande had become the most unpopular president in modern French history. However, his widely-praised handling of the crisis gave him a ratings boost and Sunday's poll was seen as a key test to this new-found popularity.
Local authorities said Socialist Frederic Barbier won 51.4 percent of the run-off vote in the eastern Doubs region, in a close race against National Front (FN) candidate Sophie Montel.
"I'm not rejoicing, I am not boasting," Barbier told his supporters, calling on the post-attacks "spirit of national unity" to continue.
Martial Bourquin, a Socialist senator in Doubs, told reporters the duel between both candidates had been tough.
"It's an extremely close fight with a strong turnout and a strong FN score," he said.
FN claims major victory
The by-election was called after Socialist lawmaker Pierre Moscovici left for Brussels to take office as Economic Affairs Commissioner.
Montel, 45, had come first in the initial round last Sunday with close to a third of the vote.
Barbier, 54, had come second and the main opposition right-wing UMP candidate was knocked out of the race.
Worried, UMP leader and former president Nicolas Sarkozy warned on Tuesday of a real risk of the National Front taking power at a national level in the future.
His party called on its supporters to abstain from voting for either candidate on Sunday.
FN leader Marine Le Pen said the score was still a major victory for her party, amid questions over whether Hollande's newfound popularity is here to stay following the January 7-9 attacks, during which three gunmen killed 17 people.
An opinion poll carried out by the Ifop polling company 10 days after the attacks found Hollande's popularity ratings had doubled to 40 percent.
But official figures published late last month showed unemployment hit a new record in December, casting a cloud on his popularity spike.
And a poll published in the daily Le Parisien on Sunday found that two-thirds of French people still think that Hollande is a "bad president" — down from 77 percent in July.
The importance of the by-election was highlighted last week when both Prime Minister Manuel Valls and Interior Minister Bernard Cazeneuve visited Barbier to pledge their support.
A 'turning point'?
In his victory speech, Socialist Barbier said the by-election could be "a turning point" for Hollande's five-year term.
His party went through a particularly bad year in 2014, when it suffered a drubbing at local polls in March and subsequently went through two government reshuffles.
The Eurosceptic FN, meanwhile, has gone from strength to strength following Marine Le Pen's successful efforts since she took over in 2011 to "de-demonise" a party widely regarded as taboo when her father Jean-Marie Le Pen was at the helm.
The FN won control of 11 towns and more than 1,200 municipal seats nationwide in local elections last year — a huge success for a party which in the last local polls in 2008 had claimed only around 60 seats.
The party's rising popularity is widely seen as a result of unhappiness with Hollande's government, as well as general exasperation at mainstream parties which many believe are unable to resolve problems of unemployment, poor growth and security.