Patrice Verry, a father-of-three, was having a barbecue with friends on Saturday evening when he was stung on the hand while trying to swat one of the insects away with a tea towel.
According to police reports he lost consciousness within minutes and could not be resuscitated by emergency services.
The mayor of Lherm, Jean Aycaguer, told local newspaper La Dépêche the number of hornets had "exploded" in the area. "We are powerless," he said.
The second death this year follows that of a 59-year-old woman in June close to Bordeaux.
The Asian hornet is believed to have arrived in France at the start of the decade as part of a delivery of Chinese products by boat, probably at the port of Bordeaux.
Since then it has spread to forty French departments, progressing by 100 kilometres each year. Experts believe it will have reached Paris within two years and is in the process of invading the rest of Europe.
The Asian hornet attacks bees and is not known to target humans unless in self-defence.