You only pay the seller success fee if your object is successfully sold.

If the reserve price is not reached during the auction, the object remains unsold. In that case, you will not be charged a seller success fee.

You can choose to offer the object for auction again. If you do, you may want to consider setting a lower reserve price to increase the chance of selling it.