INDIANAPOLIS -- Conservative commentator Rush Limbaugh crafts his popular radio show to get a reaction from both sides of the political spectrum, but Indianapolis Colts owner Jim Irsay said he doesn't want any of that in the NFL.

Limbaugh's effort to buy part of the St. Louis Rams NFL franchise has ruffled some feathers among those who consider Limbaugh's words too incendiary, 6News' Ericka Flye reported.

Comments that Limbaugh made in 2003 in reference to Philadelphia Eagles quarterback Donovan McNabb are the source of much of the controversy.

"I don't think he's been that good from the get-go. I think that we've had here a little social concern in the NFL," Limbaugh said. "I think the media has been very desirous that a black quarterback do well."

Some consider that comment to be racist, but Limbaugh said he was just pointing out what he perceived to be an issue with both the NFL and media coverage of the league.

"I couldn't even consider voting for him. When there are comments that have been made that are inappropriate, incendiary and insensitive … our words do damage, and it's something we don't need," Irsay said at an NFL owners meeting.

At a Make-A-Wish event Tuesday night hosted by Colts tight end Dallas Clark, coach Jim Caldwell and several players spoke out on the issue.

“I think what the league will do is they'll take those things into consideration," Caldwell said in reference to Limbaugh. "Everybody's entitled to their own political views. That's his business, and we're also entitled to ours as well."

"We really don't need that type of … drama in our game, and I think the owners are going to do the right thing," said Colts linebacker Gary Brackett.

Limbaugh said the 2003 comment and others are often taken out of context and even fabricated. He plans to press on with his bid to buy part of the Rams.