Jami-Lee Ross has admitted to two affairs, one with an MP and another with a former staffer.
The revelations came during an interview on Newstalk ZB today.
Mr Ross apologised to his wife for his behaviour. He says a "scab has been picked" on personal issues in Parliament.
"I apologise for the hurt I've caused."
During the interview a recording of a conversation between Mr Ross and National Party leader Simon Bridges was played.
The tape related to what Mr Ross claims were allegations of inappropriate behaviour towards women.
On the tape, Mr Ross says, "I disagree with you, I deserve to see it".
A reply to that was muffled.
Jami-Lee Ross says he will not be resigning from Parliament.
He said it was more important for him to stay expose what he believed to be any flaws.
Mr Ross stood down from the National Party on Tuesday, moments before the party's caucus expelled him. Mr Ross says he quit because he alleges Mr Bridges is a "corrupt" politician who committed electoral fraud. Mr Bridges denies these allegations.
Read more: An overview of the developments to date.