The Junior All Whites have been eliminated from football's under-20 World Cup in Poland, losing to Colombia from the penalty spot in the round of 16.
Entering the match after pool stage victories over Norway and Honduras, New Zealand looked to cause yet another upset and book a quarter-final spot for the first time in their history.
However, Des Buckingham's young side were rocked early as Colombia opened the scoring after 10 minutes - defender Andres Reyes nodding home from close range after a well worked free kick from the left.
Despite falling behind early on and striker Callum McCowatt missing a chance to equalise late in the first half, New Zealand weren't disheartened.
The team's persistence would be rewarded before the break as winger Elijah Just scored thanks to a brilliant cross from left-back Liberato Cacace, with the two sides going into halftime locked at 1-1.
Just could have put New Zealand ahead in the second half, played through one-on-one with Colombian goalkeeper Kevin Mier, Reyes clearing the ball off the line.
Both sides had their chances to score a winner towards the end of normal time, with Sarpreet Singh trying to chip Mier - seeing his effort unsuccessful before extra time was needed.
Striker Max Mata had a chance to put the Kiwis ahead in the second period of extra time, his point-blank header saved by Mier after Singh's curling free kick into the Colombian penalty area.
Substitute Matt Conroy won a free kick in the dying moments of extra time, but captain Joe Bell's low effort was intercepted and led to a Colombian break before goalkeeper Michael Woud got his side out of danger with a fantastic save to his right.
Unable to be separated after 120 minutes, the match would head to the penalty spot to determine the winner - with Colombia taking first.
Left-back Brayan Vera attempted the first kick of the shootout, only for Woud to make another save, this time to his left. Then Sarpreet Singh stepped up to give New Zealand a 1-0 lead.
Johan Carbonero was the next to try his luck from the spot - Woud again the hero with his second save. Captain Joe Bell couldn't double New Zealand's lead, though, with Mier this successful with a save.
Woud had his third save in a row after that, deflecting Andres Perea's effort - only to have been judged to have come off his goal line. Perea was successful with his retake and levelled the scores at 1-1.
Max Mata stepped up to try and restore New Zealand's lead, firing his shot to the keeper's left and into the back of the net. Luis Sandoval scored Colombia's next attempt, with the scores again locked at 2-2 in the shootout.
Gianni Stensness then fired his effort over the bar, giving Colombia the chance to take the lead. Ivan Angulo coolly scored, leaving Callum McCowatt needing to score to keep New Zealand in the contest at 3-3.
The young striker obliged and sent the shootout into sudden death. Andres Balanta again gave Colombia the advantage, with Liberato Cacace keeping his cool from the spot.
Colombia's Carlos Cuesta's success meant Conroy had to score. However, Mier proved to be the hero to send Colombia through to the quarter finals - winning the shootout 5-4.
New Zealand will head home after a valiant campaign, equalling their best-ever result at an under-20 World Cup.