Police arrested the man in a nearby street as he was making his escape and recovered the three-kilogram (seven-pound) tusk, museum workers said.
The elephant whose skeleton is preserved in the popular museum was given as a gift in 1668 by the king of Portugal to Louis XIV, who was also known as the Sun King.
The animal's tusks are not the original ones but were added to the skeleton in the 19th century.
Police made no immediate comment about why the man tried to steal the tusk but the incident comes amid a series of thefts in recent years of ivory from European museums and zoos.
The international trade in elephant ivory, with rare exceptions, has been outlawed since 1989 after elephant populations in Africa dropped from millions in the mid-20th century to some 600,000 by the end of the 1980s.
Yet poachers continue to slaughter elephants to lay their hands on their precious ivory and the illegal trade in the white gold continues to boom.