The French leader announced the withdrawal during a surprise visit to meet troops stationed in Sarobi district, northeast of Kabul, and to be briefed on progress against the Taliban by a French general.
"It's necessary to end the war," Sarkozy told journalists at the base. "There was never a question of keeping troops in Afghanistan indefinitely."
France has around 4,000 troops deployed in the country, mostly in Sarobi, Kabul, and in northeastern Kapisa province.
"We will withdraw a quarter of our troops, that's to say 1,000 men, by the end of 2012," he said.
Those remaining in Afghanistan will be concentrated in Kapisa, where they have been deployed since 2008.
"The first group will leave at the end of this year," Sarkozy said, without specifying the magnitude of this "first phase".
That withdrawal will be "in consultation with our allies and with the Afghan authorities," he said, as "the situation allows".
The partial drawdown follows similar announcements by Britain and the United States, as Western leaders look to a final deadline of the end of 2014 to extract all combat troops from an increasingly deadly and costly conflict.
In Kabul, Sarkozy held talks with Afghan President Hamid Karzai, who was in a sombre mood after receiving news shortly before the discussions that his younger brother Ahmed Wali Karzai had been assassinated in Kandahar.
Sarkozy did not rule out that French military advisors and trainers would remain after combat troops leave, "if the Afghan authorities want", adding that civilian cooperation would also continue.
"We must not abandon Afghanistan. We will continue to help Afghanistan. We'll go from military to economic cooperation," Sarkozy said after his meeting with Karzai before flying out of the country.
The French leader earlier met the top US commander on the ground, General David Petraeus, who will oversee the initial drawdown of 33,000 US troops set to leave by the end of next summer -- effectively ending a military "surge" ordered into Afghanistan, principally the south, in late 2009.
Britain has said 500 of its soldiers will leave by the end of next year. Belgium has also announced some of its troops will depart and Canada last week ended its near 3,000-strong combat mission in the southern province Kandahar.
It was Sarkozy's third visit to the battle-scarred country since becoming president and came two days ahead of the Bastille Day French national holiday.
His earlier trips were in December 2007 and August 2008.
His trip came a day after a 22-year-old French soldier was killed in a shooting blamed on "accidental fire" by a fellow French soldier.
France has lost 64 soldiers in the course of the war, according to figures compiled by the independent icasualties.org.
Last month, Sarkozy said "several hundred" French troops would be withdrawn from Afghanistan before the end of the year.
His office had said earlier that France would carry out a progressive pullback of its 4,000 troops "in a proportional manner and in a timeframe similar to the pullback of the American reinforcements".
Sarkozy's visit comes days after that by new US defence chief Leon Panetta and a week after a visit by British Prime Minister David Cameron, with Western leaders focused on efforts to draw down troops and end the long war.
Commanders are now preparing to hand over seven NATO-held areas to Afghan control starting in mid-July, although there is widespread doubt over the ability of Afghan forces to take full responsibility for their own security.
Sarkozy said he shared Obama's belief that security had improved since the killing of Al-Qaeda leader Osama bin Laden in May and that the handover to Afghan troops and police was proceeding smoothly.
Should the situation improve, the pullout of all Western combat troops in 2014 might be "brought forward", he said.
US-led coalition forces have been fighting the Taliban-led insurgency in Afghanistan since they invaded in late 2001 in the wake of the September 11 attacks orchestrated by bin Laden.