The private conversation, held during the G20 summit in the French city of Cannes last week, was overheard by a number of journalists after it was inadvertently transmitted over a system used for translation, media website Arret sur Images reported.
"I can't stand him anymore, he's a liar," Sarkozy said in French during the talks.
"You may be sick of him, but me, I have to deal with him every day," Obama replied in comments that were translated into French.
A number of journalists contacted by AFP confirmed hearing the remarks.
Without providing further quotes, the website said Obama had also chastised Sarkozy for not having informed the United States of France's plans to vote in favour of Palestinian membership in UN cultural agency UNESCO.
Journalists were able to hear the conversation after they were given translation devices for a press conference but told they would receive headphones later, the website reported.
Plugging their own headphones into the devices, they realised they could hear the French translation of the conversation between the two leaders.
The website quoted a number of journalists saying a group decision was made not to report the conversation as it was considered private and off-the-record.
When asked about the incident, French foreign ministry spokesman Bernard Valero told journalists that it was "hype".
"I've heard about the hype.... I don't want to talk about it. It's hype," Valero said, telling journalists to ask Sarkozy's office to confirm the remarks.
"Ask the Elysee to confirm or deny" the reported quotes, Valero said.
"All this is getting us away from what's essential. All we want is to keep working so that things move forward because things aren't moving forward" on the Middle East, he said.
The story of the remarks was carried on the websites of most major Israeli dailies.
The premier's office had no immediate reaction to the report and the foreign ministry refused to comment.
Israel public radio correspondent Gidon Kutz, who covered the Cannes summit, said journalists who overheard the private conversation had agreed not to report the story due to "correctness and in order not to embarrass the presidential press service."
Speaking to parliament on Tuesday, French Foreign Minister Alain Juppe insisted that France and Sarkozy had a balanced position on the Middle East conflict, but he did not mention the undiplomatic incident in Cannes.
"We have a balanced position," Juppe said during a discussion of his ministry's budget for 2012.
"In his speech at the United Nations General Assembly the president said very clearly that if Israel's security was in question, we would be on Israel's side," Juppe said.
"But he also said that after so many decades it was no longer possible to accept that the Palestinian Authority is not little by little being recognised as a state.
"We were the only ones to put an alternative on the table to the current impasse with the passage at the General Assembly (of a resolution) that would allow Palestine to have observer status."
France confirmed last week that it would abstain from a Security Council vote on full Palestinian membership of the United Nations, calling instead for Palestine to be given non-member observer status.