POPFile is written in Perl. In addition to installing the POPFile program the Windows installer installs a minimal Perl system which provides everything needed to run POPFile. The installer also ensures that new installations create at least two buckets and it can reconfigure email accounts for some email clients to make them use POPFile. Several small utilities are also installed to help diagnose problems and check the SQLite database.

These instructions describe how to build the POPFile Windows installer and the NSIS-based POPFile utilities on a Windows system.

These notes were written after the POPFile 1.1.1 release and assume an up-to-date ActivePerl installation will be used therefore the minimal Perl may have newer versions of some modules used for the 1.1.1 release. If you wish to build an installer that installs an identical minimal Perl to that used for the 1.1.1 release the safest way to do this is by “reverse engineering” the minimal Perl from the 1.1.1 release.

Explain how to “reverse engineer” the minimal Perl from the 1.1.1 release.

• Install third-party software:
  • ActivePerl
  • Perl Dev Kit (PDK) (optional)
  • NSIS Compiler
  • Additional NSIS plugins
• Obtain the source files from POPFile's Subversion repository
• Install the Kakasi package (used to analyze Japanese text)
• Install the SQLite command-line utilities
• Create the missing files (two change note files, anything else?)
• Build the Perl-based programs (optional)
• Build the POPFile Windows installer

The Windows version of POPFile uses a minimal Perl based upon ActiveState's ActivePerl. Although Perl 5.10.x has been available for some time, POPFile still uses Perl 5.8.x. When POPFile 1.1.1 was released the most recent ActivePerl 5.8.x release was ActivePerl 5.8.9 Build 826.

The Windows version of POPFile is a 32-bit program so the 32-bit (x86) Windows version of ActivePerl 5.8.x is required (the Windows Installer (MSI) version is recommended instead of the AS Package (ZIP) version). ActivePerl 5.8.9 Build 827 was released on 2 February 2010.

The source files used to build the POPFile installer assume that ActivePerl is installed in the default location (C:\Perl) but they can be easily reconfigured if ActivePerl is installed in a different location. ¹⁾

Use PPM to apply all of the available updates to the ActivePerl installation. PPM may install additional packages not mentioned in the list of updates in order to meet the dependency requirements of the packages which are being updated.

At the time of writing (February 2010) the PPM in ActivePerl 5.8.9 build 827 identified 20 packages to be updated.

By default PPM is configured to use the ActiveState repository. POPFile requires some additional Perl modules which are not included in this repository therefore PPM has to be configured to use three additional repositories from the suggestions list included in PPM:

• bribes (bribes :: Bribes de Perl)
• tcool (tcool :: Kenichi Ishigaki's repository)
• uwinnipeg (uwinnipeg :: University of Winnipeg)

If a particular Perl module is available from more than one repository the POPFile Project has always preferred to use the ActiveState repository as the source.

By default PPM does not display the Repo column. It is recommended that the Repo column is selected in the View Columns list (under the View menu in PPM) as this makes it easy to see the source repository for each module listed.

POPFile requires some additional Perl packages which are not part of the default ActivePerl installation. Use PPM to install the following extra Packages:

The BerkeleyDB v0.22 and DBD-SQLite2 v0.33 packages are available from ActiveState as zip files which can be installed using PPM's command-line mode.

• PPM refused to install DBD-SQLite2 until I removed the “<REQUIRE NAME=“DBI” VERSION=“1.21” />” line from its PPD file

The BerkeleyDB package is used when upgrading an old POPFile 0.20.x installation. An old version of this package is required in order to be able to upgrade databases created by POPFile 0.20.x (otherwise the database upgrade loses all of the old data from the BerkeleyDB database files!).

By default POPFile databases currently use the SQLite 3.x format. The DBD-SQLite2 package is used when upgrading an old SQLite 2.x format database (as used by POPFile 0.21.0 to 1.0.1).

If POPFile is to be used to make connections to mail servers that require SSL connections (e.g. Gmail) then two additional packages are required (Net_SSLeay.pm and IO-Socket-SSL). However these packages are not included in the installer, they are always downloaded, if required, during installation.

The Net_SSLeay.pm package is very old and generates conflict warnings when PPM tries to install or update other packages; need to switch to the newer Net-SSLeay package (note the hyphen instead of an underscore in this package's name!).

The Net-SSLeay contains the OpenSSL DLLs (unlike the old Net_SSLeay.pm package) so there is no longer any need to download these DLLs separately.

As mentioned earlier, these notes were written after POPFile 1.1.1 was released. Some Perl packages have been updated since POPFile 1.1.1 was released. This table summarises the differences between the minimal Perl shipped with the 1.1.1 release and that produced by following these notes:

The Windows version of POPFile ships with six Perl-based programs:

• popfile.exe (the main POPFile executable, starts one of the following four programs)
  • popfileb.exe (runs POPFile in the background)
  • popfilef.exe (runs POPFile in a “DOS-box” console window)
  • popfileib.exe (as popfileb.exe plus a system tray icon)
  • popfileif.exe (as popfilef.exe plus a system tray icon)
• popfile-service.exe (runs POPFile as a service)

These programs are all built using ActiveState's Perl Dev Kit (PDK). The Windows version of POPFile is a 32-bit program so the 32-bit (x86) Windows version of the PDK is required. Unlike ActivePerl, the PDK requires a licence. In January 2010 the cost of a single-user PDK licence was US $295. A time-limited free trial version of the PDK is also available.

Note that POPFile's Subversion repository includes built versions of these six programs which is why the PDK is described as “optional” here.

PDK 8.2.0 build 291995 was released Febraury 2010

The Windows installer for POPFile and several of the Windows utilities shipped with POPFile are built using NSIS, the Nullsoft Scriptable Install System. NSIS was SourceForge “Project of the Month” for January 2006 (POPFile was “Project of the Month” for May 2003).

The source files for the installer and the NSIS-based utilities all mention the version of the NSIS compiler which should be used (e.g. lines 38 to 59 of the app-vcheck.nsi file). If a different version of the compiler is used then a suitable warning message will appear in the compiler log.

At the time of writing POPFile uses version 2.45 of the NSIS compiler which can be downloaded from SourceForge using this link:

http://prdownloads.sourceforge.net/nsis/nsis-2.45-setup.exe?download

After NSIS is installed the compiler can be run using the GUI front-end, the command-line version or via the context menu for NSIS script files.

The POPFile Makefile uses the command-line version of the compiler (makensis.exe) and it is easier to run this version if you update the PATH to include the folder containing this file.

The POPFile installer and some of the NSIS-based POPFile utilities require some additional NSIS plugins. Each plugin is usually made available as a zip file containing the plugin DLL, the source code for the DLL, some documentation, one or more example scripts and occasionally some macros and other support functions in a NSIS header (*.nsh) file.

Version information is rarely included in these DLLs. Some plugins are rarely updated, others get frequent updates; this means that some plugins come in zip files which no longer match the current folder structure using by the NSIS compiler package.

To make it easier to build the installer and NSIS-based utilities for POPFile these extra plugins (and their associated documentation etc) have been repackaged as a single zip file ( 1.1.1-repackaged-plugins.zip) which can simply be unzipped into the main NSIS compiler folder (normally C:\Program Files\NSIS). A complete set of the original plugin zip files is also available ( 1.1.1-original-plugins.zip) as some of the plugins are not the most recent versions and are therefore harder to find.

Need to upgrade to GetVersion v1.1 (the current release) because v0.9 cannot detect Windows 7 (it returns an empty string)

Do any other plugins need to be updated before the next release?

The source code for POPFile, the POPFile installer and the NSIS-based POPFile utilities is stored in the POPFile project's Subversion (SVN) repository. This repository holds more than just the files used for the current release; it also holds the files used for all previous releases and the latest developments towards future releases. For further details see How to obtain the source code.

These notes will describe how to build a Windows installer for the current 1.x development version of POPFile.

Although the location of the source files used to build the installer is not hard-coded, the relative position of the source files is important. It is recommended that a new directory is created to hold the source files obtained from the Subversion repository and some of the additional utility software required to build the installer.

These notes will use “C:\Build\POPFile” as the build directory but a different location can be used if you prefer.

The first step is to retrieve (or “checkout”) the necessary source files from the project's Subversion repository. The easiest way to do this is by using a Subversion client, as mentioned in the How to obtain the source code page.

Use a Subversion client to checkout the current POPFile 1.x development program source files:

Use a Subversion client to checkout the current POPFile 1.x development installer and other source files:

Note that both of these checkout operations will actually checkout more files than are needed; this approach was chosen to avoid the need for multiple checkout commands.

POPFile uses a naïve Bayes algorithm to classify email. In other words, POPFile uses statistics to track which words are likely to appear in which messages. Japanese words are not separated by spaces (the English equivalent would look like Japanesewordsarenotseparatedbyspaces). This makes it harder for POPFile to classify messages. Therefore POPFile uses a special parser to split Japanese text into words to allow the text to be analysed properly. The 1.0.0 (or later) release offers a choice of three parsers (Kakasi, MeCab and internal). Prior releases of POPFile only supported the Kakasi parser.

The Windows installer includes all of the files needed for the Kakasi and internal parsers. There are two parts to the Kakasi parser: the Perl interface (Text::Kakasi) and the Kakasi package (some binary files and Japanese dictionary files). PPM can be used to install the Perl interface for Kakasi but the Windows version of the Kakasi package has to be downloaded separately.

Further information about the Kakasi package is available from the KAKASI (Kanji Kana Simple Inverter) site.

The current 32-bit Windows version of the Kakasi package is v2.3.4.

Create a directory called “kakasi_package” in the “C:\Build\POPFile\windows” directory.

The Kakasi package is distributed as a ZIP file which contains several folders (bin, doc, include, lib and share) all of which are under a top level folder called kakasi. Unzip the Kakasi package into the new C:\Build\POPFile\windows\kakasi_package directory.

The C:\Build\POPFile\windows\kakasi_package directory should now contain a directory called kakasi which contains the files and sub-directories for the Kakasi package.

POPFile uses an SQLite database to hold essential information, such as the corpus used to classify messages. The SQLite library includes a simple command-line utility that allows the user to manually enter and execute SQL commands against an SQLite database. This utility can also be used to check the integrity of the database and defragment it.

POPFile currently uses SQLite 3.x format databases which are not compatible with the SQLite 2.x format used by some earlier versions of POPFile. In addition to the SQLite 3.x format command-line utility (sqlite3.exe) the installer also installs the SQLite 2.x format utility (sqlite.exe). Both utilities are available from the official SQLite website.

The installer usually includes SQLite command-line utilities which match the library versions used by the DBD::SQLite and DBD::SQLite2 modules it installs.

Notes:

1. DBD::SQLite v1.25 is the version used for the POPFile 1.1.1 release
2. DBD::SQLite v1.29 is the version used if the instructions on this page are used to assemble the minimal Perl
3. DBD::SQLite2 v0.33 is the most recent SQLite 2.x format module in the ActiveState repository and v2.8.17 is the most recent SQLite 2.x format command-line utility.

Extract the “sqlite.exe” and “sqlite3.exe” files from the appropriate zip files and store them in the “C:\Build\POPFile\windows” directory.

The release notes are not stored in the project's Subversion repository. The release notes are plain text files and should be stored in the C:\Build\POPFile\engine directory.

Use a text editor to create a plain text file called v1.1.1.change (for the English release notes) and another called v1.1.1.change.nihongo (for the Japanese release notes; if you don't know Japanese just use a copy of the v1.1.1.change file).

If the files checked out from Subversion are marked with a release number other than 1.1.1 then change the filenames of these two files to match (e.g. v1.1.2.change and v1.1.2.change.nihongo)

Either build the six popfile*.exe files (may need to edit the engine\vars.mak file) or use the versions available from Subversion (or a suitable official release?)

Add instructions here (use RC=“-SVN” option?)