Sarkozy met the mosque's rector and the French Muslim Council leader and said he told them "he did not want, in this electoral period, some of our compatriots to feel hurt by controversies that have no place here".
"I wanted to say... to our compatriots of the Muslim religion that they naturally have the right to follow their faith as any other citizen has the right to follow his religion," he told reporters.
Sarkozy also inaugurated at the mosque a memorial to Muslim soldiers who died fighting for France.
Sarkozy has been accused of tacking to the right in the run-up to the April 22 first round of the presidential election in order to recruit voters tempted by anti-immigrant candidate Marine Le Pen's platform.
Last week the issue of Muslim immigration and in particular Islamic and Jewish dietary practice surged to the fore in the election debate, upsetting religious leaders.
Jews and Muslims came together to complain they were being used as pawns in the election, after first Le Pen then Sarkozy and finally his prime minister Francois Fillon criticised the production of halal and kosher meat.
France is home to western Europe's largest Muslim minority, officially estimated at least four million, and its largest Jewish community, estimated at up to 700,000.
The country has for years been debating how far it is willing to go to accommodate Islam, now France's second religion, and Sarkozy and Le Pen have both made the matter a central issue in their campaigns.