A spokesman told AFP that the man was picked up in Berlin late Saturday after flying in from Istanbul and was believed to have spent time in Syria where he allegedly joined the jihadist Islamic State of Iraq and the Levant.
A source close to the investigation in France named him as Tewffik Bouallag, born 30 years ago in Dreux in northern France.
An international warrant for his arrest on charges of "conspiracy to commit terrorist acts" was issued in Paris two days ago, the source added.
German daily Frankfurter Allgemeine Zeitung on Monday reported the man was hurt while fighting in Syria and had posted several pictures of himself with gunshot wounds on Islamic extremist websites.
The report added that German authorities believe the man was returning to Europe to recruit other fighters for Syria or plan an attack.
The spokesman declined to comment, saying that the federal police would release further details on Monday.
"We need to exchange some information with international partners first," he said.
The arrest comes as concerns are mounting in Europe about the growing national security threat posed by jihadists returning from war-torn Syria.
On Saturday France announced that it deported a Tunisian accused of recruiting young jihadists to join the conflict.
Police also suspect a deadly attack on the Jewish Museum in Brussels three weeks ago was carried out by a Franco-Algerian who spent more than a year fighting for jihadists in Syria.
The French government unveiled an anti-terrorism plan in April to prevent the radicalisation of nationals, thwart online recruitment and make it more difficult for aspiring jihadists to leave the country for Syria.
The German domestic security watchdog warned recently of the threat posed by Islamic extremists returning from fighting in the Syrian civil war.