INTRODUCTION For the first time in POPFile history we are releasing two versions at the same time. Please take a moment to decide which version is most appropriate for you. v0.18.2 is a minor bug fix update to the last v0.18.1 made in February and contains no new features. It is intended for conservative users who are happy with POPFile as it stands and would just like bug fixes and nothing else. v0.19.0 is a major update to v0.18.1 and include everything that is fixed in v0.18.2 plus much more. Full details are below. A special thank you goes out to Sam Schinke who has contributed a great deal to this release particularly with his work fixing outstanding bugs and porting them from v0.19.0 and v0.18.2. Nice work Sam. Brian Smith also needs your applause for major updates to the Windows installer making it easier than ever to install and configure POPFile on Windows. I recommend v0.19.0 because of the many improvements, but v0.18.2 is the STABLE release for those who want rock solid reliability and want others to do the bug hunting on the new v0.19.x line :-) (Of course if you are really conservative you might like to download and use v0.18.1 which has been in use by many people since February 27). ESSENTIAL READING IF YOU ARE UPGRADING TO v0.19.0 1. BACK UP YOUR OLD INSTALLATION: POPFile makes this really easy, just copy the entire POPFile directory somewhere. You can then safely install POPFile v0.19.0 on top of your current installation; I just think a back up is a sensible precaution. 2. ACCURACY MIGHT DROP FOR A SHORT WHILE: because of some changes made in the mail parser it is possible that you might see accuracy drop initially and you may find yourself reclassifying a few messages that used to work. This is unfortunate but necessary to make POPFile even more accurate than before and v0.19.0 incorporates changes that make POPFile's classification accuracy better; however old corpuses might need a little retraining. DOWNLOADING You can obtain the latest releases of POPFile by visiting http://sourceforge.net/project/showfiles.php?group_id=63137 UPGRADING Just install POPFile on top of the currently installed version. But did you read the ESSENTIAL READING above first if you are upgrading from a pre v0.19.0 version? FAQ zonk3r has spent a great deal of time on a POPFile FAQ. Please check it out as it covers many questions that you might have: http://sourceforge.net/docman/display_doc.php?docid=14421&group_id=63137 THE GORY DETAILS FOR v0.18.2 1. Fixed bug where -toptoo (now -pop3_toptoo) would cause duplicates in the history. Reports: 701390 () (fixed by sschinke) 2. Fixed a bug where -toptoo resulted in message overwriting and the wrong message classification. Reports: 705448, 694002 () (fixed by sschinke) 3. Fix warning on unclassified messages Reports: 697278 (jdeifik) 4. Remove lines in headers consisting of only whitespace (Eudora has trouble with them). Patch: esniper Reports: 701981 (esniper) 5. Fix colorized display of quoted-printable HTML Reports: 699098 (mfichtner) 6. Recognize protocol-less href's Reports: biljir 7. Re-work of header handling to handle multi-line headers. This will make our handling of MIME messages much more robust. Reports: 695565 (biljir), 702215 (gdvieira), 702316 (spf) 8. Fixed bug preventing decoding when headers are in an unexpected order Reports: 729551 (fibrizo) 9. Allow "+" to be submitted encoded in a form Reports: 719989 (thedonga) 10. Fixed unrequested History deletion, history doesn't go back far enough Reports: 703364 (besonen), 708387 (beej) (and others) 11. Fixed XPL problems with filtered history view Reports: 697046 (helphand), 692673 (nobody) 12. Restored main history form to POST Reports: 690451 (nobody) 13. Fixed reporting of remaining child pid's Reports: none 14. Make hostname decoding case-insensitive Reports: none 15. Fix odd return value from classify_file for some messages: "unclassified" Reports: none THE GORY DETAILS FOR v0.19.0 All of the above plus... (sschinke had a hand in lots of this too) 1. Large overhaul of the magnets system so that we add Cc magnets, make magnets editable from the Magnets page, and a new feature called QuickMagnets which allows you to create magnets from received emails by selecting parts of the From, To, Cc and Subject lines. Requests: 676341 (reason1000) 2. When an email is clicked on to see the colored version a new page appears with complete coloring (including for the pseudowords which now have popups on the colored areas), a new full dump of the probabilities used in the calculation of the message classification. Requests: lost track 3. On the Buckets page there are new statistics counting the number of errors made by bucket. For each bucket we have the number of false positives (i.e. mails that went in the bucket that should not have) and the number of false negatives (i.e. mails that should have been in that bucket but were not). Requests: 692600 (nobody), In addition when clicking on an individual bucket to look at the words in it a new page appears with a clickable index for speed and words are sorted by frequency so that you can see the most important words in each bucket. Requests: 691386 (philiplaw) From the bucket page it is possible to remove all the words in a bucket so that you can start retraining a bucket without having to delete and reinsert it. Requests: 675983 (nobody) 4. The Configuration page is now pluggable so that as new modules are written their elements appear in the configuration page automatically. Requests: none There is a link to the current log file so that you can access it from within the browser. Requests: 5. The Security page is now pluggable so that as new modules are written their elements appear in the configuration page automatically. Requests: none 6. The Advanced tab no longer uses a hard coded set of ignored words, they are now stored in a file and can be edited through the UI. Requests: lost track 7. POPFile's internal structure has undergone a large change to make use of Perl's object oriented features and there is now a common base class for all POPFile modules (called POPFile::Module) and as a result many of the POPFile modules have been simplified greatly. An offshoot of this is that there are a number of experimental modules that you can obtain only if you get the cross platform version: SMTP.pm - SMTP proxy that does SMTP mail classification NNTP.pm - NNTP proxy that does NNTP Usenet news post classification XMLRPC.pm - Full access to POPFile's API (see Classifier::Bayes) through XML-RPC The XMLRPC available methods are (documentation in Classifier::Bayes): classify_file classify_and_modify get_buckets get_bucket_word_count get_bucket_word_list get_word_count get_bucket_unique_count get_bucket_color set_bucket_color get_bucket_parameter set_bucket_parameter get_html_colored_message create_bucket delete_bucket rename_bucket add_message_to_bucket remove_message_from_bucket get_buckets_with_magnets get_magnet_types_in_bucket clear_bucket clear_magnets get_magnets create_magnet get_magnet_types delete_magnet get_stop_word_list add_stopword remove_stopword Requests: none 8. Numbers are now legal in bucket names. Requests: 720681 (deaper) 9. The characters used around the classification in subject modification can now be reconfigured with options -bayes_subject_mod_left and -bayes_subject_mod_right and they default to [ and ] Requests: 722837 (stanleyspanner) 10. All command line parameters have changed name. The old names all work correctly and are upgraded automatically, Old Parameter New Parameter corpus bayes_corpus unclassified_probability bayes_unclassified_probability piddir config_piddir debug GLOBAL_debug ecount GLOBAL_ecount mcount GLOBAL_mcount msgdir GLOBAL_msgdir subject GLOBAL_subject imeout GLOBAL_timeout xpl GLOBAL_xpl xtc GLOBAL_xtc download_count GLOBAL_download_count logdir logger_logdir localpop pop3_local port pop3_port sport pop3_secure_port server pop3_secure_server separator pop3_separator toptoo pop3_toptoo archive html_archive archive_classes html_archive_classes archive_dir html_archive_dir history_days html_history_days language html_language last_reset html_last_reset last_update_check html_last_update_check localui html_local page_size html_page_size password html_password send_stats html_send_stats skin html_skin test_language html_test_language update_check html_update_check Requests: none 11. There is now a Reclassify button at the top of the History page as well as at the bottom. Requests: 727834 (garowetz), 687138 (johnmccurdy) 12. If you do a search that results in no results there is now a search box for you to search again. Requests: 685327 (tomvoss) 13. The Windows installer now guides you in the creation of an initial set of buckets. Requests: 691348 (pkarlin) 14. Each time the history cache is reloaded we insert a marker line in the UI, these markers delimit the times the user reloaded the history with new messages and provide a useful marker for using POPFile throughout the day. Everything before the last marker is newly received. Requests: none 15. POPFile now has 19 UI translations available in the following languages: Bulgarian Chinese (simplified) Chinese (traditional) Danish Dutch English English (UK) Finnish French German Hungarian Korean Norwegian Portugese (Brazilian) Russian Slovak Spanish Swedish Ukrainian (http://zope.net.ua/POPFile) Requests: none 16. To help in the fight against spam a number of new pseudowords have been added to track spams that load images from across the web and web bugs, the complete list of pseudowords is now: encoding:<mime encoding type> header:<email header> html:numericentity html:td html:imgremotesrc html:imgwidth<pixels> html:imgheight<pixels> html:fontsize<size> html:encodedurl html:comment html:authorization trick:spaceout trick:dottedwords trick:invisibleink Requests: none 17. Numerous modifications by Sam and I to the mail parser, MIME handling, HTML parser, colorizer, Base 64 and Quoted Printable decoder to improve accuracy. Requests: lost track 18. Windows users will discover that POPFile adds an icon to the system tray with a popup menu (right click) that offers to take you to the POPFile UI or to shutdown POPFile. A double click on the icon navigates to the POPFile UI. Requests: lost track 19. Sorting of columns in the History can be set to ascending or descending by clicking on the column header. There's a new indicator that tells you which column is sorted and in what order. Requests: none 20. Fixed security problem with the X-POPFile-Link functionality and passwords. Requests: none DONATIONS Thank you to everyone who has clicked the Donate! button and donated their hard earned cash to me in support of POPFile. Thank you also to the people who have contributed patches, feature requests, bug reports and translations. http://sourceforge.net/forum/forum.php?forum_id=213876 CONCLUSION Keep the ideas and bug reports coming. John.