INTRODUCTION

v0.20.0 is a major update to POPFile with the focus being on performance.
In addition POPFile makes another leap forward in support for non-English
languages with many new UI translations (including our first right to 
left language) and full support for parsing of Japanese and Korean.

v0.20.0 is intended to end-of-life v0.18.x and v0.19.x.  All future 
development work will be occurring around the v0.20.0 line of code and no
bugs will be fixed against previous versions.  A lot of work has gone
into making v0.20.0 the version of POPFile to have.

To improve POPFile's performance the following changes have been made:

1. The corpus (where the word lists are stored) has been changed from a
   flat text file to a <nowiki>BerkeleyDB</nowiki> database.  When you run POPFile v0.20.0
   for the first time you will see your existing corpus get automatically
   upgraded to the <nowiki>BerkeleyDB</nowiki>.

   (See the 'license' file for details of <nowiki>BerkeleyDB</nowiki>'s license)

   The use of the database both speeds up POPFile (especially the
   reclassification process which has slowed down in v0.19.x) and reduces
   its memory requirements.

   The time to load the corpus has now gone from minutes/seconds to close
   to zero.

2. The history data is cached between sessions.  If you regularly start and
   stop POPFile (e.g. you start and stop your PC every day) then you'll
   notice another load-time speedup: the history data is being cached 
   between POPFile sessions to make loading faster.

3. The history is progressively updated.  As messages are downloaded from
   a server POPFile used to store all the messages for insertion into the
   history the next time the history was viewed.  Now as messages are 
   downloaded they are inserted in to the history progressively.

4. When viewing a colored message you'll notice a big speedup because there
   were previously two scans through the message (one for classification
   and one for coloring) this is now reduced to one.

5. When downloading messages we previously saved the message to disk and 
   then reloaded it for classification.  Now the mail parser has been 
   modified so that the text of the message is streamed into it as it is
   read from the mail server and classification happens in line without the
   need to reread from disk.

6. On Windows the default configuration for the proxy is to no longer fork()
   the server.  This means that downloading mail starts very quickly, but
   has the downside that only one email account can be checked at a time
   and the UI cannot be used during download.  This new option is controlled
   by a configuration parameter (-pop3_force_fork) and through the UI.  On
   non-Windows platforms POPFile will fork() each new connection.

To improve POPFile's stability:

There's been a huge effort to write a complete test suite for POPFile.  
Currently we have tests that cover 99% of POPFile's code (i.e. almost every 
line of code is exercised by a test) and the plan going forward is to try 
to keep it that way.

The test suite exercises the UI as if it were a user clicking buttons and
submitting forms, it includes a complete POP3 server and client so that the
proxy functionality can be tested and contains hundreds of tests for mail
parsing.  

Every module has an equivalent <nowiki>TestFOO.tst</nowiki> that tests it, if you are interested
in running the test suite get the tests/ directory from CVS and run 'gmake
test'.

To improve POPFile's accuracy:

A number of bugs have been fixed that sometimes caused POPFile internally to
get the right classification and then insert the wrong headers.  The mail
parser has been updated with the latest spammers' tricks and new pseudowords
and we've done an experiement with 'unsure' classifications and decided to
ship with code that will mark a message as 'unclassified' if it isn't 100
times more certain it's in bucket A than bucket B.  This should reduce the
false positive rate a little at the expense of POPFile saying it's not sure.

ESSENTIAL READING IF YOU ARE UPGRADING TO v0.20.0

1. BACK UP YOUR OLD INSTALLATION: POPFile makes this really easy, just copy
   the entire POPFile directory somewhere.  You can then safely install
   POPFile v0.20.0 on top of your current installation; I just think a back up
   is a sensible precaution.

2. IF YOU ARE RUNNING WINDOWS: Please read the section below I AM RUNNING
   WINDOWS AND NEED TO CHECK MULTIPLE EMAIL ACCOUNTS SIMULTANEOUSLY

3. ON WINDOWS POPFILE IS NOW AN EXE.  Windows users will now be able to see
   POPFile running in the Task Manager with an executable called popfileXX.exe
   where the XX is one of f, if, b, ib depending on configuration.  POPFile
   is started by running popfile.exe which chooses the appropriate popfileXX.exe

   This might cause your firewall to ask about giving popfileXX.exe permissions, 
   in addition if you had allowed Perl permissions in your firewall they are
   NO LONGER needed.

4. The installer will cause POPFile to run in the foreground if the database
   upgrade is required so that the upgrade process is evident to the user. 
   Once upgraded you can switch to background my going to the Configuration
   tab and changing "Run POPFile in a console window?" to No.

I AM RUNNING WINDOWS AND NEED TO CHECK MULTIPLE EMAIL ACCOUNTS SIMULTANEOUSLY

Because the time taken to start a new process on Windows is long under Perl
there is an optimization for Windows that is present by default: when a new
connection is made between your email program and POPFile, POPFile handles it
in the 'parent' process.  This means that the connect happens fast and mail
starts downloading very quickly, but is means that you can only downloaded
messages from one server at a time (up to 6 other connections will be queued
up and dealt with in the order they arrive) and the UI is unavaiable while
downloading email.

You can turn this behavior off (and get simultaneous UI/email access and as
many email connections as you like) by going to the Configuration panel in
the UI and making sure that "Allow concurrent POP3 connections:" is set to
Yes, or by specifying -pop3_force_fork 1 on the command line.

I AM USING THE CROSS PLATFORM VERSION

POPFile requires a number of Perl modules that are available from CPAN.  New
in v0.20.0 are the need for the following:

 <nowiki>BerkeleyDB</nowiki>
 Text::Kakasi (if you want Japanese language support)
 Encode       (if you want Japanese language support)

I LIKE TO LIVE DANGEROUSLY

In a future version POPFile will add official support for message classification
through the SMTP and NNTP (Usenet news) protocols.  There are currently proxy
modules for these protocols that work with v0.20.0, but they have not been 
fully tested.  If you are interested in getting them get them here:

http://cvs.sourceforge.net/viewcvs.py/*checkout*/popfile/engine/Proxy/SMTP.pm?rev=1.22
http://cvs.sourceforge.net/viewcvs.py/*checkout*/popfile/engine/Proxy/NNTP.pm?rev=1.21

and place them in POPFile's Proxy/ directory.  

DOWNLOADING

You can obtain the latest releases of POPFile by visiting

    http://sourceforge.net/project/showfiles.php?group_id=63137

UPGRADING

Just install POPFile on top of the currently installed version.  But did you
read the ESSENTIAL READING above first.

INTERNATIONALIZATION

POPFile's support for non-English languages has grown and the UI is now
localized into 26 languages:

        Bulgarian
        Chinese (simplified)
        Chinese (traditional)        
NEW     Czech
        Danish
        Dutch
        English
        English (UK)
        Finnish
        French
        German
NEW     Greek
NEW     Hebrew
        Hungarian
NEW     Italian
NEW     Japanese
        Korean
        Norwegian
NEW     Polish
NEW     Portugese (Iberian)
        Portugese (Brazilian)
        Russian
        Slovak
        Spanish
        Swedish
        Ukrainian
  
Also added support for understanding Japanese and Korean and doing word
splitting correctly.

DONATIONS

Thank you to everyone who has clicked the Donate! button and donated their
hard earned cash to me in support of POPFile.  Thank you also to the people
who have contributed patches, feature requests, bug reports and translations.

    http://sourceforge.net/forum/forum.php?forum_id=213876

CONCLUSION
        
Keep the ideas and bug reports coming.  If you are interested in knowing 
more about what's planned for future POPFile versions (or just learning
about POPFile's history) visit the POPFile Roadmap:

http://sourceforge.net/docman/display_doc.php?docid=17906&group_id=63137

John.