When you are no longer actively scouting a prospect, you can archive them to remove them from your Prospect lists:

After archiving a prospect, they will be moved to the Archived list. From here you can choose to restore or permanently delete the prospect from your account.