Francois Pinault, the luxury goods mogul who also owns the auction house Christie's, is taking over the Bourse de Commerce in the centre of the French capital to show off his $1.4-billion (€1.2-billion) collection of modern masters.
The city's mayor Anne Hidalgo, who negotiated the deal, described the museum as "an immense gift to the heart of Paris".
Pinault, 79, has amassed an enormous trove of work by artists from Mark Rothko to Damien Hirst, which he now shows at his private museums in Venice after failing for decades to find a suitable home for them in Paris.
"I am delighted, it's a big plus for the city," Hidalgo told AFP, pointing out that the new museum is also only a few hundred metres from the Pompidou Centre, Europe's biggest contemporary art collection.
She praised Pinault and his great business rival, France's richest man Bernard Arnault - who opened his own Frank Gehry-designed Louis Vuitton Foundation for his art collection last year - for helping put Paris back on the modern art map.
"It is great to have our captains of industry helping to fly our colours. With this and the FIAC art fair, Paris is regaining its place in contemporary art," she added.
The historic Parisian grain exchange which Pinault is taking over is part of a one-billion-euro urban renewal project to give what Hidalgo calls a "new beating heart" to the city's Les Halles district.
Paris's magnificent 19th-century central market was bulldozed in the 1970s to make way for an airless underground shopping complex and transport hub which most of its residents loathe.
But a vast new steel-and-glass canopy unveiled this month by Hidalgo to put a lid on the problem has also been derided, branded a "custard-coloured flop" by the British daily The Guardian.
Under terms of the deal, Pinault and his family will be given a 50-year lease on the building, which they must also renovate. It was not revealed how much the work would cost or how much rent he will pay.
In 2001, Pinault handed the reins of his empire - which includes the Gucci, Yves Saint Laurent, Puma and Balenciaga brands - to his son Francois-Henri, who is married to the Mexican Hollywood star Salma Hayek.
Since then the man once described as "the most powerful in the art world", has mostly dedicated himself to his art collection, installing it in the Palazzo Grassi in Venice and two other historic buildings there. They will now work in tandem with the new Paris gallery, which will open in 2018, sources close to the collector told AFP.
Pinault had tried for years to build a museum on the site of an old Renault car factory on the Ile Seguin in the middle of the Seine west of Paris, but gave up in despair in 2005 over planning delays.