INTRODUCTION
It has been close to 2 months since the last POPFile release and boy have I
and others been busy. POPFile v0.18.0 is a *major* change from v0.17.9 and
you'd do well to read the section entitled ESSENTIAL READING before you
install it. This version is backwards compatible with previous versions (even
autofixing some errors that the old versions created) but there are some
differences.
This is also the first release in which two additional developers (additional
to me that is) really contributed a lot of code to POPFile. They are Stanley
Krute and Sam Schinke. Stan and Sam worked on many of the changes to POPFile
(which you can read all about in the CVS logs) including massive changes to
the mail parser, whole new skinning system, HTML 4.01 compliance, tons of
testing, lots of user interface changes. Guys, *you* made a big difference to
POPFile, and I want to say a big "Thank you" in public.
This version of POPFile supports unlimited numbers of connections, multiple
natural languages, has a much improved UI, has many improvements to help catch
spam, has a new Quarantine feature that lets POPFile automatically hide all
the mail in a specific bucket (each mail is replaced with a new mail stating
that POPFile quarantined a message with an option to see the original), and
incorporates features and bug fixes submitted by users.
ESSENTIAL READING
1. BACK UP YOUR OLD INSTALLATION: POPFile makes this really easy, just copy
the entire POPFile directory somewhere. You can then safely install
POPFile v0.18.0 on top of your current installation; I just think a back up
is a sensible precaution.
2. YOUR HISTORY WILL CLEAR: I have changed the format of the History files
used in this version which means that the old History files will not be
read by POPFile and will in fact be deleted to save disk space. If you
need to do any reclassifications prior to installing v0.18.0 do them!
POPFile does NOT clear buckets, statistics or anything else on install of
this version, just the History files. To refresh the History view click
the History tab at any time; POPFile will check disk for new messages.
3. IF YOU HAD BROKEN MAGNETS: I have added automatic update of broken magnets.
If you had a magnet like [foo] which POPFile mistakenly changed to \[foo\]
it should get magically fixed and start working.
4. ACCURACY MIGHT DROP FOR A SHORT WHILE: because of some changes made in the
mail parser it is possible that you might see accuracy drop initially and
you may find yourself reclassifying a few messages that used to work. This
is unfortunate but necessary to make POPFile even more accurate than before
and v0.18.0 incorporates changes that make POPFile's classification
accuracy better; however old corpuses might need a little retraining.
DOWNLOADING
You can obtain the latest release of POPFile by visiting
http://sourceforge.net/project/showfiles.php?group_id=63137
UPGRADING
Just install POPFile on top of the currently installed version. But did you
read the ESSENTIAL READING above first?
FAQ
zonk3r has spent a great deal of time on a POPFile FAQ. Please check it out
as it covers many questions that you might have:
http://sourceforge.net/docman/display_doc.php?docid=14421&group_id=63137
THE GORY DETAILS
1. Total rearchitecture of the POPFile code into a collection of loadable
modules to allow for future expansion of POPFile with different proxy
services and different UIs. The major modules are now:
- popfile.pl: controlling process that loads the modules that implement
POPFile's main functionality;
- Proxy::POP3 the original POP3 proxying code from POPFile separated into
its own module. In future we should expect to be able to have other
Proxy:: modules such as Proxy::IMAP.
- UI::HTML contains the current HTML user interface. I hope that we'll
add other UI modules such as UI::XMLRPC or UI::SOAP in future.
- Classifier::Bayes is the module that does the Naive Bayes
classification. (Note this module also includes a method called
classify_and_modify that performs the classification and modification of
email messages that may be of use to other module writers)
POPFile now supports two process types: the top level process round robins
between the main modules listed above providing concurrent access to the
UI and POP3 proxy. The POP3 proxy itself (see below) forks to provide
multiple simultaneous connections. (That last part was originally
requested by sschinke)
2. Introduction of a scheme for localization of the user interface using
message catalogs stored in the languages/ subfolder. Currently shipping
with languages: English, French, German, Danish, Dutch, Brazilian, and
(for fun and for testing) H4X0R. Translators can take the English.msg file
and translate the strings within to create their own language for the user
interface. POPFile detects available languages on start up and they are
available from the Configuration page. (Originally requested by
pfaelzerchen)
3. Rework of much of the user interface to make pages shorter and clearer.
This is mostly limited to look and feel and no items have changed tab in
the UI.
4. Total rewrite of the history caching code for clarity with a change from
multiple arrays to a single hash containing all the cached information.
5. Automatic update checking now defaults to off and there's a large
explanation of what it does in the user interface to allay the fears of
many people concerning the possibility that programs like POPFile are
spying on them. (Requested by quite a few people)
6. When creating the user interface password the password is now hidden when
typing it. (Requested by kraelen, pfaelzerchen)
7. The Proxy::POP3 now forks after a new connection is accepted from a valid
client and calls a special child() method that handles the connection
between a client and mail server via POPFile for the life of the client
connection. This is the beginning of a version of POPFile that supports a
very large number of clients. (Requested by many people)
8. A new command line parameter -test_language can be set to 1 for UI
translators. The UI will show the language identifiers used for each
element rather than the strings associated with the identifier.
9. Rework of the History page to take into account ideas and suggestions from
many people: Search now operates on From and Subject, From and Subject
have popup windows to show the full text of each, you can sort by ID,
From, Subject or Classification, you can filter by magnet classified mail,
navigation is at the top and the bottom, state is remembered so that you
can sort/filter and remove pages and reclassify and stay in the same
place, many small improvements.
10. Major work on the MailParse.pm for dealing with improved mail parsing and
the handling of spammers tricks (like S P A C E D out words etc.). Switch
to using the Base64 and QuotedPrint modules from CPAN for improved mail
cracking. (Work done mainly by sschinke)
11. Automated test suite now part of POPFile. (Suggested by aziegler)
12. Rework of all the HTML and CSS so that everything can be skinned and so
that the complete UI is HTML 4.01 compliant. (Work done mainly by
stanley_krute; there was a request from nobody for this)
13. You can now filter the History page by magnet filtered mails to look at
them or delete them. (Requested by jarmstro, tcdk, msc_buff, fitzsimons,
dyoungmcwicom)
14. The UI password no longer appears in URLs in plain text (requested by
gribnif)
15. You can do multiple classifications at a time by selecting multiple
messages and changing the drop down and hitting any Reclassify button.
(Requested by dave823; implemented by sschinke)
16. You can now sort the History page by message ID, From, Subject and
Classification (bucket). (Requested by jerometremblay, tomvoss, biobear)
17. When you remove a page the History no longer goes back to the start but
stays at the same place in the History if it can. (Requested by ajweiner)
18. After you reclassify a message the History page stays at the same position
(at the reclassified message). (Requested by harleythebrave, transit)
19. The installer now picks up old settings from a current POPFile install to
fill in the options on the final dialog. (Requested by harleythebrave)
20. Added two new options -logdir and -msgdir to specify the location of the
log directory (default to ./) and the messages (where the history is
stored; defaults to messages/). (Requested by nobody)
21. Search in History now works on From as well as Subject. (Requested by
transit)
22. The navigation from page to page in the History is now at the top and at
the bottom of the page. (Requested by transit, nobody, nobody, nobody)
23. There are new [Remove] buttons on each message in the History to let you
remove messages one by one. (Requested by nobody)
24. The X-POPFile-Link: repositions the History page at the message selected
when you click the link. (Requested by zonk3r)
25. There is a new filter for unclassified messages. (Requested by nobody)
26. New -toptoo command line option enables classification on the POP3 TOP
command and is only recommended for people on fast connections. (Requested
by jberkes)
27. When viewing a colorized message there is a blank line between the header
and the body of the message. (Requested by whiteg)
28. Message IDs (and History filenames) are now globally unique per POPFile
install. (Requested by helphand)
29. You can now control the inclusion of the X-Text-Classification and X-
POPFile-Link headers. (Requested by nobody)
30. Refreshing a page does not repeat the action any more. (Reported by
nobody)
31. Fixed a bug where magnets with certain characters in them (e.g. [ or <)
did not work and could not be deleted. (Reported by nobody, nobody,
nobody, alanmuk, boneill, bana-sea-)
32. Fixed a bug where you could make POPFile exit with a divide by zero error
if you started it with no corpus and then tried to Lookup a word's
probability. (Reported by sunshine, gribnif)
33. Fixed a bug where the Shutdown link on the POPFile Control Center did not
work if the UI password had been set. (Reported by actoews, nobody)
34. Fixed an embarassing bug where any POP server that contained the word
'stat' would not work with POPFile. (Reported by nobody, markabcxyz)
35. Fixed a bug where To magnets did not work correctly in the presence of
other To style email headers. (Reported by dyoungmcwicom)
36. Fixed a bug where a specific email was causing the History page to lock up
when you tried to view the colorized version. (Reported by ajweiner)
37. Fixed a bug where having pages in POPFile automatically refresh every ten
minutes could cause POPFile to repeat an action (such as Reclassification)
every 10 minutes. (Reported by mayfoev)
38. Fixed a bug where parts of a > attribute could appear in the From or
Subject lines on the History (Reported and fixed by sschinke)
39. Fixed a bug where some CLS and MSG files were not getting deleted from the
History when they were no longer needed. (Reported by helphand)
TRANSLATIONS
Thanks to the following for helping to make POPFile a global piece of
software:
Brazilian Adriano RG
Danish Nicki Brøchner Nielsen
Dutch Jilles Oldenbeuving
French Antony Boucher
German Matthias Deege
Norweigian Jo Inge Fjellstad
Russian Alexander Saltanov
Please translate the UI and the manual into your language.
DONATIONS
Thank you to everyone who as clicked the Feed Me! button and donated their
hard earned cash to me in support of POPFile. Thank you also to the people
who have contributed patches, feature requests and bug reports.
Keep the ideas and bug reports coming.
John.
Should you find anything in the documentation that is incomplete, unclear, outdated or just plain wrong, please let us know and leave a note in the Documentation Forum.