The Hôtel de la Marine sits on the Place de la Concorde in central Paris, where it has housed senior naval staff for over 200 years. They will move to new headquarters in 2014, leaving vacant a prime piece of Paris real estate with stunning views over the historic square.
Businessman Alexandre Allard has long planned to create an artistic centre in the building but his plans have been received with horror by some of France's leading intellectuals, writers and historians including Pierre Nora and Jacques Le Goff.
The group wrote a letter to Le Monde in which they claimed to be "revolted" by a project that would allow the transformation of a "building which, in its own way, is a lesson in the history of France" into "a commercial Barnum filled with luxury suites."
A host of well-known artists, designers and celebrities hit back with their own letter.
It said that while there may be questions over Allard's project, it had "the immense merit, through the debate it's created, of unmasking the self-importance of unremitting conservatism and to show that there are audacious ways to refashion Paris as the world capital of creativity."
Among the 180 signatories were actors Isabelle Adjani, Gérard Depardieu and Sophie Marceau, singers Johnny Halliday and Yannick Noah and the fashion designer Agnès B.
Ex-president Valéry Giscard d'Estaing was commissioned by current president Nicolas Sarkozy earlier this year to review the future of the building.
In a report published on Monday, he said the Louvre museum should be free to take over the premises to create a new gallery of French treasures, showing the "crown jewels" of French culture.
Works would be displayed from the Louvre and other institutes including those housing decorative arts, furniture and the French national library. Visitors will also have access to the first floor colonnade with its views over the Place de la Concorde.
"This is an emblematic part of our national heritage," the former head of state said in a press conference on Monday afternoon. "It must stay in the hands of the French state and the French people."
President Sarkozy met the former president in the building on Monday where they discussed the report. The president will now make the final decision about the building's future.