During a fractious Question Time in Parliament today Prime Minister John Key broke the rules by talking while Speaker David Carter was standing.
Mr Key was attempting to defend his comments yesterday when he linked charities to foreign trusts in the wake of the Panama Papers scandal.
"I was shocked actually to find Greenpeace International on the database," Mr Key said yesterday in the House.
Greenpeace and Amnesty International have called on him to apologise and correct the record, but he has refused.
Greenpeace executive director Dr Russel Norman said the Prime Minister was wrong.
"He's got the wrong database and the wrong year," he told TVNZ's Breakfast today.
"He's basically besmirching the good names of these charities to help himself out of a political hotspot."
"He should apologise. What he did was wrong.
"In the process he threw mud at three charities which have nothing to do with it."
The last time a prime minister was ejected from parliament was in 2005 when Helen Clark interjected as National's Nick Smith was asking a question.
All questions must be heard by the House in silence, so Speaker Margaret Wilson was forced to ask Miss Clark to leave.
However, it is a very rare occurrence for a prime minister to be kicked out.