Guéant, who is also responsible for immigration and is known as a hardliner, provoked a storm of controversy with the comments on Saturday.
"Contrary to what the left's relativist ideology says, for us all civilisations are not of equal value," Guéant told a gathering of right-wing students.
"Those which defend humanity seem to us to be more advanced than those that do not," he said.
"Those which defend liberty, equality and fraternity, seem to us superior to those which accept tyranny, the subservience of women, social and ethnic hatred," he said in his speech, a copy of which was obtained by AFP.
He also stressed the need to "protect our civilisation".
"I do not regret (the comments)," Guéant said on Sunday, though he accused critics of taking them "out of context".
The left denounced his speech as an attempt by President Nicolas Sarkozy to woo supporters of the the far-right National Front (FN) ahead of a two-round presidential election in April and May.
Harlem Desir, the number two in the French Socialist Party, slammed "the pitiful provocation from a minister reduced to a mouthpiece for the FN".
Bernard Cazeneuve, a spokesman for Socialist presidential candidate François Hollande, denounced the remarks as "divisive and degrading" while former Socialist candidate Ségolène Royal called them "dangerous."
Sarkozy's allies were quick to defend the minister, however.
Defence Minister Gerard Longuet said it was simply "common sense" to suggest that civilisations could be ranked according to values such as "respecting personal rights, rejecting violence or abolishing the death penalty".
Finance Minister François Baroin accused the left of "exploiting the statements for electoral gain".
Foreign Minister Alain Juppé suggested that his colleague had meant to say that "all ideas, all political systems are not equal".
Speaking on BFM television, Juppé said however one should avoid talking of a shock of civilisations, suggesting the term was "inadequate".
Guéant has repeatedly linked immigration with crime in France and last month said the delinquency rate among immigrants was "two to three times higher" than the national average.
In April, he declared that an increase in the number of Muslim faithful in France posed a "problem".
He has also said that he wants to reduce the number of legal immigrants entering France, including those coming to work legally or to join their families.
His latest comments came as the FN's presidential candidate Marine Le Pen is credited with about 20 percent support in opinion polls.