The silent film won awards including best comedy and best actor for its star Jean Dujardin.
"I think all of us at one time get told no," said Dujardin in his acceptance speech.
"When I was starting out someone told me you will never do movies. Your face is too big, too expressive, too big. I have always followed my instincts to fight for my dreams" he said.
The black and white film was written and directed by Michael Hazanavicius who had difficulty getting financial backing.
It eventually found funding from a collection of French production companies including France 3 Cinema and Canal+. The film was then bought at Cannes by influential Hollywood producer Harvey Weinstein.
The film takes place in the late 1920s as silent cinema is losing ground to the rise of talking pictures.
It focuses on the story or a declining male film star, played by Dujardin, and a rising actress played by Bérénice Bejo.
The Golden Globes are often seen as a good indicator of possible Oscar success and the film's cast have been waging a campaign to drum up support for the film.
Nominations for the 84th Academy Awards will take place on January 24th with the ceremony taking place on February 26th.