The arrests were made on Wednesday morning in the 18th arrondissement of Paris as well as in Seine-Saint-Denis, to the north of the capital.
"We received information about one individual... that he might commit violent acts in France," Cazeneuve said on French television.
"So this morning he was arrested along with three members of his entourage and checks are under way. I call on everyone to be extremely prudent regarding reports that are circulating about an imminent violent act."
Earlier media reports in France, that quoted sources in the internal intelligence services DGSI, said that the "imminent" attack was to target the "very centre" of Paris and that each of the four were known to have connections to radical Islam.
AFP's own police police sources tried to play down the notion that an imminent terror attack was about to be carried out in a French capital, still recovering after November's carnage.
Police sources told AFP said one was placed under house arrest under the state of emergency that has been in force in France since the November 13 attacks in Paris that claimed 130 lives.
The four had been under surveillance by French intelligence on suspicion of a "possible plan to attack" but the threat has not been confirmed, the sources said.
Police raids have turned up no weapons, but computer hardware and data have been seized for analysisd, they said.
"You can't at this stage talk about a plan of imminent attack," one police source said.
Some reports claimed rounds of Kalashnikov ammunition were found in a house search linked to the suspects.
Two of the suspects are reported to be brothers Aytac et Ercan B, both French citizens with Turkish backgrounds, and the third a 28-year-old Frenchman, Youssef E, who had been sentenced to five years in prison in March 2014 after being stopped at Saint-Etienne airport preparing to travel to Syria for jihad.
However, he was freed from prison in October of 2015 and placed under house arrest under France's state of emergency which was put in place following a series of coordinated terror attacks in November.
The fourth suspect was his girlfriend, who has not been named by French press.