The fighting involving up to 200 people began overnight on Monday-Tuesday and continued on Tuesday afternoon, with some of the migrants armed with metal bars and sticks.
The local government in Calais said in a statement that 20 people had been injured in total, most of them overnight, while the local prosecutor's office said five people had been arrested.
The fighting began overnight after a row caused by a drunk migrant during the distribution of free food by local charity, the Auberge des Migrants.
Calais is a magnet for people wanting to travel onto Britain, a short distance across the Channel.
READ ALSO: France must build more migrant centres, rights watchdog says
Local authorities and the French government are determined to stop camps sprouting up around the port area but were recently compelled to provide basic sanitation facilities.
The state's failure to provide for the migrants' basic needs "exposed them to inhuman and degrading treatment," a court recently found, while local charities regularly denounce the French government's response.
Fighting and attempts to break into trucks heading for Britain are regular problems for the police around Calais, however.
In early July, a brawl between African migrants from Eritrea and Ethiopia left 16 injured.