The firebrand Copé took 50.03 percent of the vote, edging Fillon by only 98 votes to succeed former president Nicolas Sarkozy, the party's internal electoral commission announced.
The 48-year-old Copé, who pushed the more populist agenda during the campaign, promptly invited Fillon to work with him and stressed that his "hands were outstretched and his arms wide open."
Fillon himself stopped short of rejecting the result but mentioned "many irregularities" in the electoral process and said it was evidence of a deep rift in the party, six months after Sarkozy lost the presidency to Socialist François Hollande.
The public slanging match reached such a pitch that #UMP became the top trending hashtag on the French edition of Twitter, where the candidates were the targets of a deluge of mockery.
UMP party heavyweight Alain Juppé, Sarkozy's foreign minister, pleaded with the pair to put a stop to their supporters' "invectives" and warned that "the very existence of the UMP is in question".
"The movement has emerged divided and thus weakened by this excessive confrontation. Throughout the campaign, it has been less a question of the future of the UMP and more about the two candidates' obsession with 2017," when the next presidential election will be held, Juppe wrote on his blog.
Delighted by the debacle, the UMP's Socialist rivals mocked the party's struggles.
"This tragicomedy is a bad vaudeville act that does no honour to French democracy," said Bruno Le Roux, the head of the Socialists' parliamentary faction.
Copé is not even certain to be the party's next candidate as Sarkozy – whom polls say most UMP supporters want to have another tilt at the presidency – has not ruled out a return to politics.
Both Fillon, 58, and Copé are advocates of free market policies and economic reform . But they differ on social issues, with Copé sharing Sarkozy's tough-talking approach to immigration and Islam.
Aides to Copé, who accused Fillon supporters of "ballot stuffing" and "major fraud", had said before the commission announced the final results that he was 1,000 votes ahead of Fillon.
The electoral commission said around 175,000 votes were cast.
Copé will be taking over a party well-placed to capitalise on Hollande's slump in popularity and the economic gloom engulfing France, which Moody's stripped of its AAA status on Monday, becoming the second ratings agency to downgrade the country's top bond credit rating.
He will also face a difficult task in uniting the party after a bitter battle that delighted the UMP's rivals.
"It is obvious that whoever is elected president of the UMP will have no legitimacy whatsoever given that he will be in charge of a party broken in two," said Florian Philippot, deputy leader of the far-right National Front.
Fillon, who was prime minister for five years under Sarkozy, had gone into the vote as the marginal favourite, hoping to sell himself as a unity candidate capable of attracting centrist voters.
He accused his rival of opportunism, while seeking to portray himself as an experienced statesman – a stance that prompted Cope to dismiss him as the "Hollande of the right," in a reference to the president's perceived lack of charisma and reputation for dithering.
Copé has taken up where Sarkozy left off, unabashed in his bid to woo voters from the National Front, whose record strong score at this year's presidential election split the right-wing vote and torpedoed Sarkozy's re-election bid.
Copé last month published "A Manifesto for an Uninhibited Right", in which he lambasted a culture of "anti-white racism" amongst immigrant communities in impoverished urban areas.