Two groups, Osez le Féminisme and Les Chiennes de Garde, have joined forces for the campaign which is targeting forms such as tax returns, social security documents and even the train operator SNCF.
"There is no reason for two salutations for women which divide them into two categories: married and unmarried," said Julie Muret, spokeswoman for Osez le Féminisme, to radio station Europe 1.
"Our campaign is aimed at completely removing this sexist title, as it only concerns women," she said.
The groups have set up a website, madameoumadame.fr, and a Facebook page to support the campaign.
Letters are available for download to send to companies and public bodies to demand "the end of the use of mademoiselle" and the term "nom de jeune fille" (maiden name) in all correspondence.
"This salutation does not exist in the civil code and French law does not require that a married woman take the name of her husband. All that is just habit," said Marie-Noëlle Bas of Chiennes de Garde.
The groups will also take up the matter with government ministers and are asking women to write to their members of parliament.
"This double salutation has been abandoned in a large number of companies," said Muret, citing the case of Germany where the use of "Fräulein" has become rare.
Unsuccessful attempts to remove the "mademoiselle" form have been made in the past. As far back as 1972 the justice minister at the time, René Pleven, noted that "no rule insists there should be a choice between madame and mademoiselle."
The blog viedemeuf.fr (life of women) has been collecting stories about the issue, including the grandmother who is still referred to as "mademoiselle" in official letters and a single woman who was refused a bank account when she tried to open it as a "madame."
"To open a bank account, it's impossible to use madame if you're single. It's got to be mademoiselle," wrote the woman on viedemeuf.fr. "For men, is it mister or squire?"