Why is email being kept both in POPFile and in my email program?

Email will show up in POPFile's history (http://127.0.0.1:8080/history) and in your email client (e.g. Outlook). The history page in POPFile is only for reclassifying email and any email that is there for two days (by default) or more will automatically be deleted.

See also What does POPFile do to the email it receives?