The New York Times named the writers pulling out of the May 5th annual PEN American Center gala as Peter Carey, Michael Ondaatje, Francine Prose, Teju Cole, Rachel Kushner and Taiye Selasi.
Carey, a two-times Booker Prize winner, said the award stepped beyond the group's traditional role of protecting freedom of expression against government oppression, the Times reported.
"A hideous crime was committed, but was it a freedom-of-speech issue for PEN America to be self-righteous about?" the newspaper quoted him as saying in an email interview.
"All this is complicated by PEN's seeming blindness to the cultural arrogance of the French nation, which does not recognize its moral obligation to a large and disempowered segment of their population."
On January 7th, two Islamist brothers claiming to avenge the magazine's depiction of the Prophet Mohammed stormed the Charlie Hebdo offices in Paris, killing 12 people.
In a blog response, PEN said it did not believe Charlie Hebdo's intent was to "ostracize or insult Muslims, but rather to reject forcefully the efforts of a small minority of radical extremists to place broad categories of speech off limits."
"We will be sorry not to see those who have opted out of the gala, but we respect them for their convictions," it added.
Charlie Hebdo essayist Jean-Baptiste Thoret, who escaped the attack by arriving late to work, is to accept the PEN/Toni and James C. Goodale Freedom of Expression Courage Award on behalf of his colleagues at next week's gala.
British writer Salman Rushdie, who went into hiding after a 1989 fatwa called for his death over his book "The Satanic Verses," said his old friends, Carey and Ondaatje were "horribly wrong."
"If PEN as a free speech organization can't defend and celebrate people who have been murdered for drawing pictures, then frankly the organization is not worth the name," he told the Times.