INTRODUCTION

It has been close to 2 months since the last POPFile release and boy have I
and others been busy.  POPFile v0.18.0 is a *major* change from v0.17.9 and
you'd do well to read the section entitled ESSENTIAL READING before you
install it. This version is backwards compatible with previous versions (even
autofixing some errors that the old versions created) but there are some
differences.

This is also the first release in which two additional developers (additional
to me that is) really contributed a lot of code to POPFile.  They are Stanley
Krute and Sam Schinke.  Stan and Sam worked on many of the changes to POPFile
(which you can read all about in the CVS logs) including massive changes to
the mail parser, whole new skinning system, HTML 4.01 compliance, tons of
testing, lots of user interface changes.  Guys, *you* made a big difference to
POPFile, and I want to say a big "Thank you" in public.

This version of POPFile supports unlimited numbers of connections, multiple
natural languages, has a much improved UI, has many improvements to help catch
spam, has a new Quarantine feature that lets POPFile automatically hide all
the mail in a specific bucket (each mail is replaced with a new mail stating
that POPFile quarantined a message with an option to see the original), and
incorporates features and bug fixes submitted by users.

ESSENTIAL READING

1. BACK UP YOUR OLD INSTALLATION: POPFile makes this really easy, just copy
   the entire POPFile directory somewhere.  You can then safely install
   POPFile v0.18.0 on top of your current installation; I just think a back up
   is a sensible precaution.

2. YOUR HISTORY WILL CLEAR: I have changed the format of the History files
   used in this version which means that the old History files will not be
   read by POPFile and will in fact be deleted to save disk space.  If you
   need to do any reclassifications prior to installing v0.18.0 do them!
   POPFile does NOT clear buckets, statistics or anything else on install of
   this version, just the History files.  To refresh the History view click
   the History tab at any time; POPFile will check disk for new messages.

3. IF YOU HAD BROKEN MAGNETS: I have added automatic update of broken magnets.
   If you had a magnet like [foo] which POPFile mistakenly changed to \[foo\]
   it should get magically fixed and start working.

4. ACCURACY MIGHT DROP FOR A SHORT WHILE: because of some changes made in the
   mail parser it is possible that you might see accuracy drop initially and
   you may find yourself reclassifying a few messages that used to work.  This
   is unfortunate but necessary to make POPFile even more accurate than before
   and v0.18.0 incorporates changes that make POPFile's classification
   accuracy better; however old corpuses might need a little retraining.

DOWNLOADING

You can obtain the latest release of POPFile by visiting

    http://sourceforge.net/project/showfiles.php?group_id=63137

UPGRADING

Just install POPFile on top of the currently installed version.  But did you
read the ESSENTIAL READING above first?

FAQ

zonk3r has spent a great deal of time on a POPFile FAQ.  Please check it out
as it covers many questions that you might have:

    http://sourceforge.net/docman/display_doc.php?docid=14421&group_id=63137

THE GORY DETAILS

 1. Total rearchitecture of the POPFile code into a collection of loadable
    modules to allow for future expansion of POPFile with different proxy
    services and different UIs.  The major modules are now:

    - popfile.pl: controlling process that loads the modules that implement
      POPFile's main functionality;

    - Proxy::POP3 the original POP3 proxying code from POPFile separated into
      its own module.  In future we should expect to be able to have other
      Proxy:: modules such as Proxy::IMAP.

    - UI::HTML contains the current HTML user interface.  I hope that we'll
      add other UI modules such as UI::XMLRPC or UI::SOAP in future.

    - Classifier::Bayes is the module that does the Naive Bayes
      classification. (Note this module also includes a method called
      classify_and_modify that performs the classification and modification of
      email messages that may be of use to other module writers)

    POPFile now supports two process types: the top level process round robins
    between the main modules listed above providing concurrent access to the
    UI and POP3 proxy.  The POP3 proxy itself (see below) forks to provide
    multiple simultaneous connections. (That last part was originally
    requested by sschinke)

 2. Introduction of a scheme for localization of the user interface using
    message catalogs stored in the languages/ subfolder. Currently shipping
    with languages: English, French, German, Danish, Dutch, Brazilian, and
    (for fun and for testing) H4X0R. Translators can take the English.msg file
    and translate the strings within to create their own language for the user
    interface. POPFile detects available languages on start up and they are
    available from the Configuration page. (Originally requested by
    pfaelzerchen)

 3. Rework of much of the user interface to make pages shorter and clearer.
    This is mostly limited to look and feel and no items have changed tab in
    the UI.

 4. Total rewrite of the history caching code for clarity with a change from
    multiple arrays to a single hash containing all the cached information.

 5. Automatic update checking now defaults to off and there's a large
    explanation of what it does in the user interface to allay the fears of
    many people concerning the possibility that programs like POPFile are
    spying on them. (Requested by quite a few people)

 6. When creating the user interface password the password is now hidden when
    typing it. (Requested by kraelen, pfaelzerchen)

 7. The Proxy::POP3 now forks after a new connection is accepted from a valid
    client and calls a special child() method that handles the connection
    between a client and mail server via POPFile for the life of the client
    connection.  This is the beginning of a version of POPFile that supports a
    very large number of clients. (Requested by many people)

 8. A new command line parameter -test_language can be set to 1 for UI
    translators.  The UI will show the language identifiers used for each
    element rather than the strings associated with the identifier.

 9. Rework of the History page to take into account ideas and suggestions from
    many people: Search now operates on From and Subject, From and Subject
    have popup windows to show the full text of each, you can sort by ID,
    From, Subject or Classification, you can filter by magnet classified mail,
    navigation is at the top and the bottom, state is remembered so that you
    can sort/filter and remove pages and reclassify and stay in the same
    place, many small improvements.

10. Major work on the MailParse.pm for dealing with improved mail parsing and
    the handling of spammers tricks (like S P A C E D out words etc.).  Switch
    to using the Base64 and QuotedPrint modules from CPAN for improved mail
    cracking. (Work done mainly by sschinke)

11. Automated test suite now part of POPFile. (Suggested by aziegler)

12. Rework of all the HTML and CSS so that everything can be skinned and so
    that the complete UI is HTML 4.01 compliant. (Work done mainly by
    stanley_krute; there was a request from nobody for this)

13. You can now filter the History page by magnet filtered mails to look at
    them or delete them. (Requested by jarmstro, tcdk, msc_buff, fitzsimons,
    dyoungmcwicom)

14. The UI password no longer appears in URLs in plain text (requested by
    gribnif)

15. You can do multiple classifications at a time by selecting multiple
    messages and changing the drop down and hitting any Reclassify button.
    (Requested by dave823; implemented by sschinke)

16. You can now sort the History page by message ID, From, Subject and
    Classification (bucket). (Requested by jerometremblay, tomvoss, biobear)

17. When you remove a page the History no longer goes back to the start but
    stays at the same place in the History if it can. (Requested by ajweiner)

18. After you reclassify a message the History page stays at the same position
    (at the reclassified message). (Requested by harleythebrave, transit)

19. The installer now picks up old settings from a current POPFile install to
    fill in the options on the final dialog. (Requested by harleythebrave)

20. Added two new options -logdir and -msgdir to specify the location of the
    log directory (default to ./) and the messages (where the history is
    stored; defaults to messages/). (Requested by nobody)

21. Search in History now works on From as well as Subject. (Requested by
    transit)

22. The navigation from page to page in the History is now at the top and at
    the bottom of the page. (Requested by transit, nobody, nobody, nobody)

23. There are new [Remove] buttons on each message in the History to let you
    remove messages one by one. (Requested by nobody)

24. The X-POPFile-Link: repositions the History page at the message selected
    when you click the link. (Requested by zonk3r)

25. There is a new filter for unclassified messages. (Requested by nobody)

26. New -toptoo command line option enables classification on the POP3 TOP
    command and is only recommended for people on fast connections. (Requested
    by jberkes)

27. When viewing a colorized message there is a blank line between the header
    and the body of the message. (Requested by whiteg)

28. Message IDs (and History filenames) are now globally unique per POPFile
    install. (Requested by helphand)

29. You can now control the inclusion of the X-Text-Classification and X-
    POPFile-Link headers. (Requested by nobody)

30. Refreshing a page does not repeat the action any more. (Reported by
    nobody)

31. Fixed a bug where magnets with certain characters in them (e.g. [ or <)
    did not work and could not be deleted. (Reported by nobody, nobody,
    nobody, alanmuk, boneill, bana-sea-)

32. Fixed a bug where you could make POPFile exit with a divide by zero error
    if you started it with no corpus and then tried to Lookup a word's
    probability. (Reported by sunshine, gribnif)

33. Fixed a bug where the Shutdown link on the POPFile Control Center did not
    work if the UI password had been set. (Reported by actoews, nobody)

34. Fixed an embarassing bug where any POP server that contained the word
    'stat' would not work with POPFile. (Reported by nobody, markabcxyz)

35. Fixed a bug where To magnets did not work correctly in the presence of
    other To style email headers. (Reported by dyoungmcwicom)

36. Fixed a bug where a specific email was causing the History page to lock up
    when you tried to view the colorized version. (Reported by ajweiner)

37. Fixed a bug where having pages in POPFile automatically refresh every ten
    minutes could cause POPFile to repeat an action (such as Reclassification)
    every 10 minutes. (Reported by mayfoev)

38. Fixed a bug where parts of a > attribute could appear in the From or
    Subject lines on the History (Reported and fixed by sschinke)

39. Fixed a bug where some CLS and MSG files were not getting deleted from the
    History when they were no longer needed. (Reported by helphand)

TRANSLATIONS

Thanks to the following for helping to make POPFile a global piece of
software:

    Brazilian       Adriano RG
    Danish          Nicki Brøchner Nielsen
    Dutch           Jilles Oldenbeuving
    French          Antony Boucher
    German          Matthias Deege
    Norweigian      Jo Inge Fjellstad
    Russian         Alexander Saltanov

Please translate the UI and the manual into your language.

DONATIONS

Thank you to everyone who as clicked the Feed Me! button and donated their
hard earned cash to me in support of POPFile.  Thank you also to the people
who have contributed patches, feature requests and bug reports.

Keep the ideas and bug reports coming.

John.