The IFOP poll published in the weekly Journal du Dimanche showed Sarkozy winning 22 percent of first-round voting intentions, trailing behind the leading opposition Socialist contender Francois Hollande with 26 percent.
Until mid-May, the favourite to win the 2012 French election was Dominique Strauss-Kahn, known as DSK, the Socialist who led the IMF until New York police arrested him on sexual assault and attempted rape charges.
Sunday's poll showed that Hollande, a former leader of the Socialist Party, had gained three points since IFOP's last survey and moved firmly into the lead.
It showed the right-wing president was still facing a threat from Marine Le Pen, leader of the far-right National Front party.
Le Pen would score 21 percent to Sarkozy's 22 in the first round of voting if Hollande runs for the Socialists, and would beat him 22 percent to 21.5 if Socialist leader Martine Aubry ran instead of Hollande, the poll showed.
"Although the DSK affair has flummoxed the public and sparked a burst of feminism, the French do not seem to have decided, at least for now, to offer Nicolas Sarkozy a second presidential term," the newspaper wrote.
France's former president Jacques Chirac, who was Sarkozy's mentor before falling out with him, surprised journalists on Saturday by saying he would vote for Hollande. He later insisted he was joking.