Initial reports suggest that the man entered from the street and walked in to the assembly hall of the La Rochefoucauld primary school late on Thursday morning.
The man, who was not affiliated with the school, forced his way into the institution in central Paris at 11:30 am, pushing aside two adults who tried to stop him, Francois Weil, the city's top educational official said.
The man, who police sources said had a history of family violence, was in an agitated state.
He had some papers with him and a sawn-off shotgun, which he then fired from under his jaw. A part of his face was blown off in front of around a dozen children and one adult.
It is not believed that the man was affiliated with the school as a teacher, staff member or parent.
Some of the children who witnessed the man shoot himself in the head described the ordeal to French media.
"I heard a pistol shot I saw all the blood I also saw that the gentleman:... He fell back when he killed himself. It was horrible," one young girl told Europe1 radio.
Another boy from the nursery said: "I heard the gunshot. They said he had to hide and get away. There were many people who cried like me.
"I was very scared. I thought they were terrorists who had guns and would come into the school. I was behind and I saw the man who was on the floor with all the blood," he told Europe1.
With the school being situated on a busy Paris street, several people witnessed the panic in the aftermath of the terrifying incident.
"I saw a group of children who ran screaming into the street Rue de Grenelle. They were howling. Obviously there was something that had shocked them," a woman named Fabienne, who lives across the street from the school, told Europe1 radio.
Another woman, who works across the street from the nursery school said: "There are people who said that it was like a bomb. Nobody knows any more. There are police and firefighters everywhere."
Police have opened an official investigation into the suicide but so far have not identified the man nor the reasons that prompted him to take his own life at the school.
A psychological support team has been sent to the school to help comfort the children and member of staff who witnessed the shocking incident.
The mother of a student criticised security measures at the school, saying there was only one woman who monitored and controlled visitors to the school.
"Blood had to spill before security measures were undertaken," she told AFP, speaking on condition of anonymity.
The school remained closed on Thursday afternoon with police having cordoned off the whole street.
France's Education Minister Vincent Peillon was returning from Brussels to visit the school and the Paris police commissioner and the mayor of the 7th arrondissement were on their way to the site.
The La Rochefoucauld primary is a private, Catholic school serving children as young as three years of age, and is located on Rue Cler in the 7th arrondissement of Paris, close to both the French parliament and American University of Paris.
*Original reports included in this article said the man had shot himself at a nursery school