Officers arrested the ex-soldier early on Friday morning, just two days before France goes to the polls in the crucial second round of the French election.
According to RTL news, the suspect was already known to French intelligence services, and had a so-called "fiche S" (which are given to people considered a specific threat to security and includes those who have been radicalized).
Police were reportedly alerted to the man's arrival in the town after a "suspicious" car was seen parked near the Évreux-Fauville Air Base, a source told BFM TV channel.
The man, reportedly a 34-year-old who lives is from the Seine-Maritime department of Normandy, was arrested when he returned to the car in the early hours of the morning.
According to L'Express news site police found a cartridge of bullets in the car and a copy of the Koran.
A later report in L'Express quoting sources close to the investigation said several weapons had been found hidden in undergrowth near the base.
A pump action shot gun and two black powder revolvers as well as munitions were found.
Police have not confirmed the reports officially.
Counter terrorism prosecutors have been placed in charge of the investigation and officers are trying to determine the motives of the arrested individual.
It remains unclear at this point whether the man posed any kind of threat, but French police are taking no chances ahead of Sunday's crucial election between Emmanuel Macron and Marine Le Pen.
The arrest comes after police arrested four men and one woman and seized heavy weapons in a series of raids on Tuesday.
The raids were ordered by Paris prosecutors as part of a terror investigation, according to reports.
Police made another ten arrests in both Belgium and France last week, with various media reporting that the arrests were connected to the January 2015 terror attacks in Paris, where Amedy Coulibaly killed five people before he was shot dead by police.
France has been under a state of emergency since 130 people were killed in the November attacks in Paris in 2015.
There have been several other high profile terror attacks in France since then, most recently when a police officer was shot dead and two others were injured when a gunman opened fire on the Champs-Elysées in Paris two weeks ago.
It remains unclear at this point whether the man posed any kind of threat.