Tory chairman Brandon Lewis has asked Boris Johnson to apologise for his remarks about people wearing burkas.
The ex-foreign secretary was accused of Islamophobia after writing in his newspaper column that Muslim women wearing them “look like letter boxes”.
He rejected a ban on full-face veils but said they looked “ridiculous”.
He was criticised by Labour MPs, some Tories and Muslim groups, who said the party was not doing enough to tackle prejudice.
Earlier, Foreign Office Minister Alistair Burt said there was a “degree of offence” in Mr Johnson’s comments.
On Twitter, Mr Lewis said he agreed with Mr Burt and called on the MP and former London mayor to apologise.
Speaking on BBC Radio 4’s The World at One, Lord Sheikh, who founded the Conservative Muslim Forum, suggested Mr Johnson was “using Muslims as a springboard” for his ambition to lead the Tory Party.
“It is a joke but in very, very bad taste,” he said, adding that the former foreign secretary had a “weird sense of humour”.
Former Tory chairwoman Lady Warsi called for an independent inquiry into Islamophobia in the Conservative Party, telling Channel 4 News: “Muslim women should not be a useful political battleground for Old Etonians.”
But Conservative backbench MP Andrew Bridgen said Mr Johnson had raised an important subject in a “light-hearted way”.
The reaction “says a lot about internal Conservative Party politics” he told the BBC.
Another Conservative MP, Nadine Dorries, said the government should apologise and that “Boris didn’t go far enough”.
