If (Test-Path $FilePath){
Remove-Item $FilePath
}
This snippet uses the Test-Path to ensure the file exists. If it does, it will then delete it using Remove-Item
PS C:\> $FilePath = "C:\temp\temp01.csv"
>> If (Test-Path $FilePath){
>> Remove-Item $FilePath
>> }