Документация:
Создаем:
New-Service -Name "MyService" -BinaryPathName "C:\Path\to\myservice.exe"
Удаляем:
Remove-Service -Name "TestService"
Удаляем по DisplayName:
Get-Service -DisplayName "Test Service" | Remove-Service