Columbia, SC (WLTX) - The South Carolina House of Representatives has voted to take away the salary of the state's public safety agency director, saying he has mismanaged the department he oversees.

The House voted 76-20 to delete the salary of South Carolina Department of Public Safety (SCDP) Director Leroy Smith from the budget. Lawmakers do not have the authority to fire Smith--only the governor can remove or appoint the position--but they can take away funding in an effort to convince the governor that a change is needed.

The vote came after a spirited debate in the chamber, led by a small group of lawmakers that said they had deep problems with how Smith has run the agency that he's led for nearly six years.

SCDPS includes several agencies, including the South Carolina Highway Patrol, the Bureau of Protective Services, and the State Transport Police.

Lawmakers who were critical of Smith said the agency is plagued by low morale and retention problems.

"We have a serious problem at DPS and that problem is at the top," said Rep. Eddie Tallon.

Tallon said he's gotten calls from troopers who said they're afraid to work.

Other lawmakers cited statistics which they said show fatal collisions have gone up, even as citations by the agency have dropped. Rep. Todd Rutherford noted that this has gone on, even though the agency's budget has increased by 4 percent.

"People are dying and he is doing nothing about it," Rutherford said.

Others, though, spoke in Smith's defense. Rep. Justin Bamberg questioned the numbers, and why the House was debating the issues, asserting that Gov. Henry McMaster should make the call on whether Smith should continue to serve.

After the vote, McMaster told News19 that he supports Director Smith, and that he's aware of House members' concerns and takes them seriously.

SCDPD Director Smith responded in a written statement released Tuesday afternoon. "I was disappointed to hear how some members of the House of Representatives voted on the budget amendment today," Smith said. I, along with other dedicated employees at the South Carolina Department of Public Safety, work very hard to make South Carolina a safe place for its residents and visitors."