The 44-year-old Belgian, who started out designing furniture, will be responsible for Haute Couture, Ready-to-Wear and Women's Accessories, the Paris-based fashion house said in a statement.
He will premiere his first Christian Dior Haute Couture collection in Paris in July, said Dior, one of the crown jewels in Bernard Arnault's LVMH luxury empire.
Dior hailed the Belgian as "one of the biggest contemporary talents", saying that "he will inspire and push into the 21st century the style that Mr Dior launched with the opening of his house and that has changed, since its first collection, the codes of world elegance."
The designer in turn hailed his new employer as "a symbol of absolute elegance."
Flamboyant British designer Galliano was sacked last year after a video emerged of him hurling anti-Semitic slurs at patrons in a Paris bar.
Since then, his former righthand man Bill Gaytten has overseen Dior collections.
Galliano was convicted of anti-Semitism by a French court in September, receiving suspended fines totalling €6,000 ($8,400) after the court accepted his argument that he was sorry for his actions.
The Paris criminal court found him guilty of making anti-Semitic insults in public -- an offence under French law -- when he clashed with bar patrons in the capital's Marais district on two occasions, in 2010 and 2011.
The court ordered Galliano to pay a symbolic euro in damages to each of the victims and to five anti-racism groups that were plaintiffs in the case. He was also told to pay the associations €16,500 in legal costs.
The 50-year-old designer later checked into rehab for two months in Arizona and Switzerland.
Simons, who has always been a darling of the fashion press, had his own eponymous line that has been showing in Paris for a while. He is known for his often ground-breaking, minimalist fashion which he originally designed for men before launching into women's pret-a-porter.
The Flemish Belgian works and lives in Antwerp, and his name had been repeatedly mentioned over recent months as a possible replacement for Galliano.
The Jil Sander fashion house, for which he had worked as creator since 2005, announced his departure in February.
His last Sander fashion show, which was well received in Milan, was awash with sculptural pastel-coloured dresses.
Simons, whose blue eyes, athletic build, and short hair lend him an air of strict authority, was born in Neerpelt, close to the Dutch border.
He studied industrial design before launching his career as a furniture designer for art galleries and private patrons.
Following an internship with Belgian fashion designer Walter Van Beirendonck, he switched to clothes, working under a Belgian tailor, before showing off his mostly classical first men's wear creations in Milan for Raf Simons in 1995.
In 1999-2000, he also designed men's wear for the experimental Ruffo Research fashion house, in association with Veronique Branquinho who worked on women's creations.
But despite good reviews, he declared bankruptcy and took a year's sabbatical saying he felt "stifled and constrained by the commercial aspect of fashion".
In 2000, he started teaching fashion at the Vienna University of Applied Arts, a job he kept for five years. But in 2001 he also resumed his own fashion work with a smaller team and the backing of a Belgian industrial tycoon.
His Paris fashion shows, which played ever more on refining volumes, were keenly watched as indicative of future trends.
Taking his cue from the sub-cultural youth movement and his own predilection for clear cut lines, Simons described his men's wear as meant for "the individualist, an eccentric who doesn't mind being different".
As of 2004, he started designed women's clothing.
In 2008, he opened two shops in Japan and took to designing accessories, including handbags, shoes and glasses, working with partners such as Eastpak, which designed backpacks popular with US students.
In May last year, he presided at the fashion festival in Hyeres, southern France, defending young designers who try to branch out by saying that "if they have a real message, if they have a true story to present, the fashion world will follow them."