The National Party is pledging to make it illegal for people not to alert authorities to instances of child abuse they're aware of.
However, the Justice Minister says a change in law could impact innocent people.
National's policy announcement today comes after a four-year-old boy was found at a property in Flaxmere, Hastings last month with a severe brain injury and bruises all over his body.
Simon Bridges said his party's proposed new law would carry a maximum penalty of three years in prison.
"The law would require someone to give police information unless they had a reasonable excuse not to.
"I know something needs to be done about the appalling rates of child abuse in this country," Mr Bridges said.
Children's Commissioner Andrew Becroft yesterday called for the Government to review the right to silence in cases of serious child abuse.
"Children's lives can depend on adults speaking up for them," he wrote on Twitter.
"We did it for fraud, let’s do it for our kids."
Justice Minister Andrew Little said changing the law would need a "widespread, broad, rigorous public debate".
"If somebody knows something about what happened to that little boy ... they have a moral duty to step up and speak.
"You're not going to get that in a law, the impact of which can affect a whole bunch of other people in a negative way, particularly people who are innocent."
He has asked for advice on changing caution around the right to silence and that he was "quite attracted" to the British law which gives a warning to people.
Today, Prime Minister Jacinda Ardern said the law "does need another look".
"I don't think anyone is comfortable in this case where no one is being held responsible for the harm to a vulnerable child."
Yesterday, Ms Ardern said she shared the "discomfort" with the Children's Commissioner.
ACT pledged on Monday to change the law, with deputy leader Beth Houlbrooke saying that if a person failed to answer reasonable questions from police around child abuse, they could be prosecuted.
"Police must be given proper tools to investigate child abuse," she said. "Our children deserve better."