Gisela Stuart, the former chair of Vote Leave, warned Labour MPs that voting for Theresa May’s Chequers deal was “dangerous” for the party and that “even established parties could be wiped out”.
She said that losing Labour seats in Scotland made the party vulnerable.
Ms Stuart, who is now the chair of Change Britain, Vote Leave's successor organisation, also warned that the Conservative Party was under threat too if they betrayed the Brexit vote, but that she “cared a lot more about Labour”.
Speaking at a Labour Leave event on “How Labour can maintain the support of Leave voters”, she said: “Nobody has changed their mind. I don’t know one person who has said they have changed how they would vote.
“Look at the People’s Vote March, I haven’t spoken to one person from that march who voted to leave the European Union. If they were there, I haven’t spoken to them.
“Everyone, who was at that march voted Remain.
“Refusing to leave the European Union is dangerous for the Labour Party. It is dangerous for the Conservative Party too but I obviously care a lot more about Labour.
“The 2015 election saw Labour lose all of its seats but one.
“An established party has been wiped out before.”
The General Election in 2015 saw the Labour Party lose 40 of its 41 seats in Scotland to the SNP, and the last election in 2017 only saw the party take back six other seats.
London is a 'different planet'
Labour MP Kate Hoey added to the warning, saying that betraying Brexit voters was not only bad for Labour but also "very dangerous for democracy".
"The People's Vote is ridiculous," she said.
"I would really be disappointed if the leader whipped for a People's Vote."
"Not pushing for leaving is dangerous for the Labour Party. It is very dangerous for democracy."
The MP for Vauxhall added that she was "confident" support for Brexit was there, despite there only being "a few" pro-Brexit Labour MPs in Parliament.
"I am confident that support for Brexit is there.
"There may only be a few of us in Parliament that will openly speak out but I know there are some Labour MPs from constituencies in the Midlands and North East who have sympathy.
"They may not have spoken out but they know how important Brexit is for their constituencies."
She added: "London is like being on a different planet."
Labour is in 'great danger'
John Mills, a Labour donor and founder of JML, joined the other two at the event and reiterated their warnings, saying that Labour needed the support of pro-Brexit constituencies.
“Labour’s future is under “great danger”, he said.
- Read more: Theresa May’s appeal to traditional Labour voters was ‘smart politics’, says Conservative MP Grant Shapps
“The Labour Party is not going to win the next General Election with large majorities in London.
“The party needs support in the north east, central Scotland and Wales.”