In January, 12 members of the magazine's staff were massacred when two Islamist gunmen -- enraged by caricatures of the Prophet Muhammad -- stormed an editorial meeting in Paris.
This month, the publication is to be honored by the PEN authors association, but more than 150 writers have pulled out of the event in protest at cartoons they say are offensive to France's minority populations.
Speaking in Washington, Charlie Hebdo's film critic Jean-Baptiste Thoret -- who only avoided the attack because he had been late for work -- said the critics had misunderstood the reason for the PEN award.
"Maybe there is a little misunderstanding, I guess they imagine this award was given to Charlie Hebdo regarding its content," he said.
"There is a confusion. It's an award given to the principle of freedom of speech."
Those pulling out of the May 5 annual PEN American Center gala include celebrated writers such as Peter Carey, Michael Ondaatje, Francine Prose, Teju Cole, Rachel Kushner and Taiye Selasi.
Carey, a two-time Booker Prize winner, said the award went beyond the group's traditional role of protecting freedom of expression against
"A hideous crime was committed, but was it a freedom-of-speech issue for PEN America to be self-righteous about?" the New York Times quoted him as saying.
"All this is complicated by PEN's seeming blindness to the cultural arrogance of the French nation, which does not recognize its moral obligation to a large and disempowered segment of their population."
Charlie's editor-in-chief Gerard Biard dismissed this argument.
"It's always gratifying to have a contrarian opinion, to say 'I don't think like the masses. I'm above all that,'" he told AFP.
"That's their problem. If they think PEN is no longer about protecting free expression, why don't they leave it?"
Speaking at Freedom House, a democratic rights watchdog, Thoret pointed out that out of 500 Charlie Hebdo cover stories published before the attack -- only 11 dealt with Islam.
"That's the reality," he said. "We have never publish racist cartoons. Historically, we are an anti-racist magazine. It's in our DNA.
"Muhammad is an icon, a symbol. We at Charlie Hebdo are against icons -- like we are against soccer," he said.