Andrew Bridgen, a member of the European Research Group, has described David Lammy as “not the sharpest tool in the box” after the Labour MP defended comparing the ERG to the Nazi party.
Mr Lammy defended his views on BBC1's The Andrew Marr Show saying: "I would say that that wasn't strong enough. In 1938 there were allies who hatched a plan for Hitler to annex part of Czechoslovakia, and Churchill said no, and he stood alone.
"We must not appease. We're in a situation now, and let me just be clear, I'm an ethnic minority.
"We have, in the ERG, in Jacob Rees-Mogg, someone who is happy to put onto his web pages the horrible, racist AfD [Alternative for Germany] party, a party that's Islamophobic and on the far right."
The AfD was founded in 2013 as a Eurosceptic party in Germany, and has since campaigned for tougher immigration laws. Mr Rees-Mogg had previously tweeted a clip from the party but denied supporting the AfD.
Mr Lammy added: "They're happy to use the phrase 'grand wizard'. KKK is what it evokes to me when I think of that phrase and the Deep South.”
Watch: David Lammy defending his comments about the ERG.
Mr Bridgen told talkRADIO’s Julia Hartley-Brewer that he was “insulted” by the Tottenham MP’s comments.
“I just can’t believe that David Lammy could come out with something that is as absolutely ludicrous as that,” he said.
“As far as I am concerned, I am not far right and I believe in democracy.
“It is insulting and I am embarrassed for him. The European Research Group is not a homogenous group.
“It is a group of MPs who believe in Brexit and delivering on the promises we made to the British people. It is a very diverse group.
He added: “I find it deeply insulting. David Lammy is a bit of a self-publicist and he is not the sharpest tool in the box.”
Conservative MP Dominic Grieve has described Mr Lammy's language as "completely wrong".
He told Julia: "His language is simply wrong. I may have disagreements with my Leave colleagues and I think that they are failing to see some aspects of national interest, which I wish they would see.
"They cannot be characterised in these terms at all."
Mr Lammy has been contacted for a comment.