Cabinet minister David Benson-Pope, who stood down on Monday, is continuing to deny allegations he assaulted students when he was a teacher in the 1980s.
The Associate Education Minister stood down from cabinet while an inquiry gets underway into allegations that he physically abused pupils.
Former pupil Phil Weaver says he was renowned for talking and one of the punishments he was handed included having a tennis ball put in his mouth.
"Obviously with a tennis ball in your mouth you're very quiet. I took it out and he put it back in and then taped my hands to the desk so I couldn't take it out myself," Weaver told One News.
Another student claims Benson-Pope caned him so hard he ended up bleeding.
The minister admits using corporal punishment, but has vehemently denied bullying or assaulting students.
When asked about the claims by One News on Tuesday morning he again refuted them saying he stood by his comments.
"I'm keen to get to the bottom of. No further comment," Benson-Pope said as he was heading back to his Dunedin electorate.
Benson-Pope's former boss has defended his practice as a teacher saying he was hard working, enthusiastic and fun.
Anne Pentecost was the Head of Languages at Bayfield High School when the Benson-Pope worked at the school.
She says she never heard any complaints against his code of discipline and she finds the allegations and the subsequent inquiry bizarre.
National MP Judith Collins, who was one of those to raise the allegations in parliament last week says there is more to come.
"There are even more allegations that are coming out. More came to me in the weekend about more recent events and I think even more serious," Collins says.
The Prime Minister has called for an inquiry into the
allegations and a decision about what form it will take will be
made by the end of the week.