Quevilly - the first amateur club since Amiens in 2001 to reach the final - will play Lyon at the Stade de France on April 28th.
The third-tier minnows - who put Marseille out in the previous round - had looked dead and buried after Julien Feret put their Brittany rivals ahead after just eight minutes in Caen.
But the Normans never gave up with substitute Karim Herouat stepping up shortly after coming on to grab the equaliser after 64 minutes with Anthony Laup sealing an unlikely victory four minutes into injury time.
It was the second time the Normandy club have eliminated Rennes after also cutting down the Ligue 1 side in the last 16 in 2010.
"It was an incredible evening and the perfect scenario, scoring in the final minutes of the game," said Quevilley coach Regis Brouard.
"The first half was complicated, we were too timid, we lacked risk-taking.
"But after the break we managed to do some crazy things and push forward.
"We had very little left and on a final deep ball, despite the fatigue and emotion Laup managed to win his duel.
"It's the charm of the French Cup."
Rennes coach Frederic Antonetti was scathing about this team's performance.
"I can't defend the undefendable, I knew that we lacked character and that was proven," he said. "We were favourites but we didn't know how to approach the match psychologically.
"It's a big personal disappointment."
Lyon booked their berth on Tuesday by beating Gazelec Ajaccio 4-0 in Corsica.
Gazelec - the third division high-fliers who had previously knocked out Ligue 1 clubs Toulouse and Montpellier - were handicapped by the early sending-off of Anthony Colinet in a match played at the home of their top-flight neighbours AC Ajaccio.
However, Alexandre Lacazette gave OL the lead just before the hour, and skipper Lisandro Lopez doubled their advantage soon after.
Substitute Clement Grenier scored within seconds of coming on before Bafetimbi Gomis netted his 23rd goal of the season to wrap up the win.
Lyon can now look forward to a second trip to the Stade de France - they also have a League Cup final against Marseille to prepare for on Saturday.