A Tory MP has bemoaned members of his party being “tarred” by claims it has an “institutional problem” with Islamophobia during a parliamentary debate that saw Conservatives clash with opposition MPs over the issue.

Conservative James Daly was speaking during a Westminster Hall debate marking Islamophobia Awareness Month, where a number of Labour MPs hit out at the Conservatives’ record on the issue.

Labour MP Afzal Khan attacked what he described as “the Conservative Party’s Islamophobia crisis”, and said “what concerns me is that the Tory Party has an institutional problem and frankly does not care about Islamophobia”.

He said the Government refuses to accept the All-Party Parliamentary Group on British Muslims’ definition of Islamophobia, and criticised “the Prime Minister’s shocking comment about Muslim women and letterboxes”.

Mr Khan said the Singh review revealed “institutional failings within the Conservative Party in how it handled Islamophobia complaints”, adding “once again it did not even acknowledge or mention the term Islamophobia”.

He said: “The theme of this year’s Islamophobia Awareness Month is time for change. And it is time for change. It is time this Government changed its approach towards Islamophobia and tackles it head on.”

Mr Daly said he was “disappointed” the debate had turned into “the normal political attack”.

He referred to the Labour Party’s record on antisemitism, saying: “The sanctimony of an organisation that was investigated by the Human Rights Commission for prejudice and antisemitism, investigated, to lecture this party regarding prejudice is something.”

Mr Daly mentioned two people who work for him, one he said was his “best friend”, saying: “Those two people who have given years of service to my area and to the community in my area. And to be tarred with what has just been said. To actually suggest, that I do not, which the member for Manchester, Gorton (Afzal Khan) knows is not correct, I fight every day in my constituency to ensure that my Muslim constituents have the best possible representation.”

Mr Khan responded: “The issue isn’t that you’ve got two employees who are Muslim, or you have friends who are Muslims. The issue is we have a Government who is failing to tackle this problem, you’re a member of that Government and that’s what you need to do as well.”

Mr Daly said Mr Khan “made a generalised statement trying to slur every Conservative MP with a prejudice that is not correct”.

Equalities minister Kemi Badenoch said on “such a sensitive subject” the “highly partisan nature” would “if watched by our Muslim constituents, won’t look like people trying to tackle anti-Muslim hatred, but actually look like they as a community are being used as a political football for political goals”.

Labour MP Peter Dowd, who was chairing the debate, made a number of warnings about the way it was proceeding, saying at one point: “I do not want shouting across the chamber. It’s a very passionate subject, but I will ask members to stop it.”

Liberal Democrat MP for Bath Wera Hobhouse said she was “a little disturbed” by the “aggression” of some Conservatives in the initial part of the debate.

She said it “behoves” people like herself who have not faced discrimination based on their skin colour or religion “to listen carefully to those who have the lived experience, and not to call it politics, and recognise it as the hurt that has been caused”.

Ms Badenoch described the debate as “feisty” and closed the debate by saying she would not be “intimidated or bullied”.

The minister said: “I’m not afraid of using the phrase Islamophobia. We are not going to have a semantic argument. But there are good reasons why we refer to anti-Muslim hatred.”

And she added: “We must not allow those who seek to divide our diverse and multi-faith society to succeed. And we are united here today in our determination to protect people and end discrimination.”

In response to Ms Hobhouse, she said: “I will tell her of my many lived experiences of racism at the hands of Liberal Democrats who made disgusting and vile comments which I’m sure she would be happy to apologise for. We should be able to have a debate here without making those partisan sort of attacks.”

Ending her speech, the minister said: “This is an issue that I’m prepared to work with all members of the House on, but what I will not do is to be intimidated or bullied.”