It was "particularly shocking" that the angry protesters tried to block firefighters' access to the burning tax office, the premier said, adding that he "vigorously" condemned Friday's night of violence in the northwestern town of Morlaix.
"Legal action will be taken against the perpetrators," Valls said. "Nothing justifies the use of violent methods that don't resolve anything," he added.
A local resident who witnessed the blaze at the tax office said the farmers used their tractors to keep firefighters away from the scene.
"When the fire truck finally managed to reach the building, they dumped their artichokes to slow the men down," he said. "It was a ballet of tractors that was not spontaneous, it was well organised."
Prosecutors in Brest have opened an inquiry into the incident for damages to public property and hindering the emergency services. No arrests have been made.
Both the Morlaix tax office and the local MSA agricultural insurance building were destroyed in the flames.
"This is an action of some violence that unfortunately reflects an exasperation, a strong distress in the agricultural world today," said Xavier Beulin, head of the FNSEA farmers' federation.
The farmers, who numbered around 100, said they were protesting against heavy fiscal burdens and red tape that have led to a decline in living standards. They also voiced anger over a Russian embargo on European food imports.
Artichoke farmer Gilles Moal said the situation was so bad that he had "the impression of not having the right to a future".
Morlaix mayor Agnes Le Brun also decried the violence but she warned that tensions had been simmering for months, finally brought to boiling point because of "a combination of events: a fall in consumption, overproduction, falling prices, the Russian embargo, increasingly overwhelming payroll costs".