Some placards at the rally, which took place at the same time as Floyd's funeral in Texas, drew parallels between Floyd and those who have recently died at the hands of the French police.
The killing of Floyd, a 46-year-old African American, by white police officers on May 25th has sparked protests across the United States and inspired anti-racism rallies across the world.
French Prime Minister Edouard Philippe, who was not at the demonstration, said there had been a “very big, very legitimate, widely shared” outpouring of emotion after Floyd's death.
“France, the national police, the gendarmerie, are not racist. But every time there is a racist act… it's important that the whole country reacts,” he said in his first public comments on the topic since the rallies began.
The prime minister also called for “respect and trust” towards the police, saying that the public should hold the force to high standards.
Leaders of France's radical left France Unbowed party and other left-wing parties attended Tuesday's gathering, organised by the campaign group SOS Racisme.
Police said there were 2,400 people at the demonstration, while SOS Racisme estimated the crowd size at a much larger 12,000.
“There is a movement forming in the country against the horrible contamination of racism, where we wouldn't want to see it – in an important body, the police,” said France Unbowed leader Jean-Luc Melenchon.
The French protests have rallied around the case of a young black man, Adama Traore, who died in police custody in 2016.
His family have called for more demonstrations in France on Saturday.
Camelia Jordana – a singer and actress of Algerian descent who said last month that thousands of people like her feared being “massacred” by the police – sang “We Shall Overcome” at the meeting.
Interior Minister Christophe Castaner, who rowed with Jordana over her comment last month, has moved to address some concerns about the police.
He banned police from using chokeholds to detain suspects and has promised “zero tolerance” for racism in law enforcement.
Castaner has acknowledged that too many officers “have failed in their Republican duty” in recent weeks, with several instances of racist and discriminatory remarks revealed.
Dominique Sopo, the president of SOS Racisme, welcomed Castaner's comments, saying it was understandable that citizens want the forces of order to be “beyond reproach”.
Last weekend, some 23,000 people protested in several French cities to demand justice for victims of crimes allegedly committed by police.
Tuesday's demonstration went ahead despite the social distancing measures currently in force to prevent any resurgence of the coronavirus.
Castaner acknowledged that worldwide feeling was running so high on this issue as to over-ride such considerations.
Similar, smaller gatherings also took place in other cities across France.