You can only remove unassigned filters. If you try to remove a filter that is in use, an error will appear providing the location where it's in use.
To delete a filter:
Go to Configuration > Filters.
Expand the Filters tree and select a filter.
Click Delete.