Asking a woman's "maiden name" (or "nom de jeune fille" in French) or "married name" will also be banished from official documents.
Instead, all women will be known as "madame" in future, "just like the equivalent of "monsieur" for men, which does not prejudge their marital status" said the official note.
Instead, the simple "nom de famille" ("family name") will replace masculine terms such as "nom patronymique" and "nom d'époux".
The prime minister has instructed his ministers to get the terms removed "as soon as possible" although officials will be allowed to use up existing stocks of forms so as not to waste public funds.
The move is a surprise success for two feminist groups who launched a campaign to banish "mademoiselle".
Osez le Féminisme and Les Chiennes de Garde launched their campaign to banish the term in September.
"There is no reason for two salutations for women which divide them into two categories: married and unmarried," said Julie Muret, spokeswoman for Osez le Féminisme, at the time.
Their campaign found support from the solidarity minister, Roselyne Bachelot, who said in November she had asked the prime minister to issue the order.
In recent months a number of towns around the country have started to issue the order to dispose of "mademoiselle".
The two feminist organisations which launched the campaign issued a statement on Tuesday welcoming the "concrete results" of the campaign.
They called on "companies and private organisations to follow the move by removing the terms from their own documents."