Hollande's affair with the actress made headlines worldwide in January, but photos of the pair on a private terrace of the palace which appeared last week were even more worrying for the embattled leader because they appeared to have been taken from within the Elysée itself.
The celebrity magazine Voici said the three pictures it published of Hollande, 60, and Gayet, 42, sitting together were taken secretly in October.
An investigation into the security breach has failed to identify the culprit, according to the Liberation daily. But police are convinced they were taken by a member of the president's personal staff on a mobile phone from within his private apartments.
The paper said that four of the five staff had been given their jobs in the palace by Hollande's predecessor, Nicolas Sarkozy, adding to suspicions that a mole was feeding embarrassing leaks to Sarkozy, who is widely presumed to be aiming for an electoral comeback.
"Five staff members assigned to the the president's private service are being reassigned," a source in the president's office said, partially confirming the paper's claims.
Paparazzi photos in January of Hollande in a scooter helmet visiting the actress in a Paris apartment set off a media firestorm from which Hollande has never recovered.
His partner at the time, journalist Valerie Trierweiler, published a bestselling tell-all memoir of their subsequent break-up, portraying the president as a cold-hearted egotist, who was contemptuous of the "toothless" poor.
"Thank You For This Moment" has now been translated into 12 languages and was published in English last week amid huge hype.
A poll published shortly after the book came out in France found only one percent of French people said they had "total confidence" in Hollande "to resolve the problems France is currently facing".