Legislators voted last week to increase from three to four the number of days that an employee in the private sector can be sick and off work without getting paid.
At the same time, public sector workers would go just one day without getting paid.
The combined measures would have saved around €440 million ($593 million) at a time when France is looking to make large cuts in public spending to bring down its level of debt.
"The majority of parliamentarians just don't want it," said Pierre Méhaignerie, president of the social affairs committee in the country's National Assembly.
He said that members of parliament had heard "very strong reactions" in their constituencies from voters. There was a "feeling of injustice: why one day for some and four for others?"
On Monday a group of MPs from the governing UMP party had written to the health minister to call for the fourth day of unpaid leave to be scrapped.
In their letter they said the decision had been taken "without any consultation" and that it would "weigh heavily on private sector employees," particularly those working in small and medium-sized businesses.
In cancelling the move, the prime minister told parliamentarians he wanted them to find the €200 million saving the extra day would have created from elsewhere.