The minister vowed to clamp down on mosques and preachers inciting hatred after a suspected Islamist beheaded his boss during an attack on a gas factory last week.
The attack, which had the hallmarks of a jihadist act but is also believed to have personal motivations, was the second in six months in France which is battling to curb radicalisation that has seen hundreds of citizens leave to wage jihad in Iraq and Syria.
Any "foreign preacher of hate will be deported," said Cazeneuve, adding that several mosques were being investigated for inciting terrorism and if found to be doing so, "will be shut down".
"We have deported 40 preachers of hatred since 2012. Since the beginning of the year we have examined 22 cases, and around 10 imams and preachers of hatred have been expelled," said Cazeneuve.
Yassin Salhi, 35, on Sunday confessed during interrogation to killing his boss and pinning his head to a fence of the Air Products factory near the eastern city of Lyon.
The severed head was discovered flanked by two Islamic flags and it later emerged he had sent a selfie of himself with the head to a number believed to belong to a French jihadist currently in Syria.
While Salhi was known to security services for links to radical Islamists in France, and his crime bore the hallmarks of a jihadist act, sources close to the investigation have suggested a personal dimension after a dispute with his employer two days earlier.
"There is no doubt of the personal motivations but there is a symbolism taken from the most atrocious, abject images of terrorism," said Cazeneuve.