These instructions describe how to use a Windows system to build the POPFile Windows installer and the NSIS-based POPFile utilities for a new POPFile release. These notes record the steps taken to create the POPFile 1.1.2 release from scratch (i.e. on a system not previously used to build the installer).

• Install third-party software:
  • ActivePerl
  • Perl Dev Kit (PDK) (optional)
  • NSIS Compiler
  • Additional NSIS plugins

• Obtain the source files from POPFile's Subversion repository

• Obtain the Kakasi package (used to analyze Japanese text)

• Obtain the relevant SQLite command-line utilities

• Install Make and 7-Zip (anything else?)

• Create the missing files (two change note files, anything else?)

• Build the Perl-based programs (optional)

• Build the POPFile Windows installer

The Windows version of POPFile uses a minimal Perl based upon ActiveState's ActivePerl. Although Perl 5.10.x and 5.12.x have been released, POPFile still uses Perl 5.8.x. When work started on the POPFile 1.1.2 release the most recent ActivePerl 5.8.x release was ActivePerl 5.8.9 Build 829.

The Windows version of POPFile is a 32-bit program so the Windows Installer (MSI) file for the Windows (x86) version of ActivePerl 5.8.x is required. When I used the MSI installation package I disabled the Perl ISAPI, PerlEx and PerlScript options.

ActiveState only provide free download links for the current builds of the Windows, Linux and Mac OS X versions of ActivePerl 5.8.x. ¹⁾ This means there may be some differences between the minimal Perl used in the official release and the minimal Perl produced using the free release of ActivePerl. If you wish to build an installer that installs an identical minimal Perl to that used for a particular POPFile release the safest way to do this is by “reverse engineering” the minimal Perl from that release.

Explain how to “reverse engineer” the minimal Perl from a POPFile release.

The source files used to build the POPFile installer assume that ActivePerl is installed in the default location (C:\Perl) but they can be easily reconfigured if ActivePerl is installed in a different location. ²⁾

By default the Perl Package Manager (PPM) does not display the Repo column. It is recommended that the Repo column is selected in the View Columns list (under the View menu in PPM) as this makes it easy to see the source repository for each module listed.

By default PPM is configured to use the ActiveState repository. POPFile requires some additional Perl modules which are not included in this repository therefore PPM has to be configured to use some additional repositories. Add the following repositories from the drop-down list provided by PPM:

If a particular Perl module is available from more than one repository the POPFile Project has always preferred to use the ActiveState repository as the source. However there is one important exception to this rule (SOAP-Lite).

Use the Perl Package Manager (PPM) to apply all of the available updates to the ActivePerl installation. PPM may install additional packages not mentioned in the list of updates in order to meet the dependency requirements of the packages which are being updated.

At the time of writing (April/May 2011) the PPM in a fresh installation of ActivePerl 5.8.9 build 829 identified 50 packages to be updated (and a further 5 were identified as prerequisites). However by the time you read these notes more updates will have been made available.

POPFile requires some additional Perl packages which are not part of the default ActivePerl installation. Use PPM to install the following extra Packages:

POPFile has used a SQLite 3.x format database by default since the 1.1.0 release in November 2008. To allow POPFile to upgrade databases used by older versions of POPFile, two more packages are required:

The BerkeleyDB package is used when upgrading a POPFile 0.20.x installation. An old version of this package is required in order to be able to upgrade databases created by POPFile 0.20.x (otherwise the database upgrade loses all of the old data from the BerkeleyDB database files!).

The DBD-SQLite2 package is used when upgrading an old SQLite 2.x format database (as used by POPFile 0.21.0 to 1.0.1).

Since both of these packages are old versions it seems that an ActiveState Business Edition license has to be purchased in order to download the binary versions from ActiveState. However the package source files are still available from CPAN ( BerkeleyDB-0.22.tar.gz and DBD-SQLite2-0.33.tar.gz).

Give download links for the BerkeleyDB-0.22-modified.zip and DBD-SQLite2-0.33.zip files plus installation instructions for these files.

If POPFile is to be used to make connections to mail servers that require SSL connections (e.g. Gmail) then two additional packages are required:

Note the '-' in Net-SSLeay's name; do not install the obsolete package called Net_SSLeay.pm.

Unlike the old Net_SSLeay.pm package which required the separate download of two OpenSSL DLLs, the Net-SSLeay package includes the necessary OpenSSL code in its single DLL.

The Windows version of POPFile ships with six Perl-based programs:

• popfile.exe (the main POPFile executable, starts one of the following four programs)
  • popfileb.exe (runs POPFile in the background)
  • popfilef.exe (runs POPFile in a “DOS-box” console window)
  • popfileib.exe (as popfileb.exe plus a system tray icon)
  • popfileif.exe (as popfilef.exe plus a system tray icon)
• popfile-service.exe (runs POPFile as a service)

These programs are all built using ActiveState's Perl Dev Kit (PDK). The Windows version of POPFile is a 32-bit program so the 32-bit (x86) Windows version of the PDK is required. Unlike ActivePerl, the PDK requires a licence. In January 2010 the cost of a single-user PDK licence was US $295. A time-limited free trial version of the PDK is also available.

Note that POPFile's Subversion repository includes built versions of these six Perl-based programs which is why the PDK is described as “optional” here.

PDK 9.0.1 build 293382 was released on 9 August 2010.

The Windows installer for POPFile and several of the Windows utilities shipped with POPFile are built using NSIS, the Nullsoft Scriptable Install System. NSIS was SourceForge “Project of the Month” for January 2006 (POPFile was “Project of the Month” for May 2003).

The source files for the installer and the NSIS-based utilities all mention the version of the NSIS compiler which should be used (e.g. lines 38 to 59 of the app-vcheck.nsi file). If a different version of the compiler is used then a suitable warning message will appear in the compiler log.

At the time of writing POPFile uses version 2.45 of the NSIS compiler which can be downloaded from SourceForge using this link:

http://prdownloads.sourceforge.net/nsis/nsis-2.45-setup.exe?download

After NSIS is installed the compiler can be run using the GUI front-end, the command-line version or via the context menu for NSIS script files.

The POPFile Makefile uses the command-line version of the compiler (makensis.exe) and it is easier to run this version if you update the PATH to include the folder containing this file.

The POPFile installer and some of the NSIS-based POPFile utilities require some additional NSIS plugins. Each plugin is usually made available as a zip file containing the plugin DLL, the source code for the DLL, some documentation, one or more example scripts and occasionally some macros and other support functions in a NSIS header (*.nsh) file.

Version information is rarely included in these DLLs. Some plugins are rarely updated, others get frequent updates; this means that some plugins come in zip files which no longer match the current folder structure using by the NSIS compiler package.

To make it easier to build the installer and NSIS-based utilities for POPFile these extra plugins (and their associated documentation etc) have been repackaged as a single zip file ( 1.1.1-repackaged-plugins.zip) which can simply be unzipped into the main NSIS compiler folder (normally C:\Program Files\NSIS). A complete set of the original plugin zip files is also available ( 1.1.1-original-plugins.zip) as some of the plugins are not the most recent versions and are therefore harder to find.

Need to upgrade to GetVersion v1.1 (the current release) because v0.9 cannot detect Windows 7 (it returns an empty string)

Do any other plugins need to be updated before the next release?

The source code for POPFile, the POPFile installer and the NSIS-based POPFile utilities is stored in the POPFile project's Subversion (SVN) repository. This repository holds more than just the files used for the current release; it also holds the files used for all previous releases and the latest developments towards future releases. For further details see How to obtain the source code.

These notes will describe how to build a Windows installer for the current 1.x development version of POPFile.

Although the location of the source files used to build the installer is not hard-coded, the relative position of the source files is important. It is recommended that a new directory is created to hold the source files obtained from the Subversion repository and some of the additional utility software required to build the installer.

These notes will use “C:\Build\POPFile” as the build directory but a different location can be used if you prefer.

The first step is to retrieve (or “checkout”) the necessary source files from the project's Subversion repository. The easiest way to do this is by using a Subversion client, as mentioned in the How to obtain the source code page.

Use a Subversion client to checkout the current POPFile 1.x development program source files:

Use a Subversion client to checkout the current POPFile 1.x development installer and other source files:

Note that both of these checkout operations will actually checkout more files than are needed; this approach was chosen to avoid the need for multiple checkout commands.

POPFile uses a naïve Bayes algorithm to classify email. In other words, POPFile uses statistics to track which words are likely to appear in which messages. Japanese words are not separated by spaces (the English equivalent would look like Japanesewordsarenotseparatedbyspaces). This makes it harder for POPFile to classify messages. Therefore POPFile uses a special parser to split Japanese text into words to allow the text to be analysed properly. The 1.0.0 (or later) release offers a choice of three parsers (Kakasi, MeCab and internal). Prior releases of POPFile only supported the Kakasi parser.

The Windows installer includes all of the files needed for the Kakasi and internal parsers. There are two parts to the Kakasi parser: the Perl interface (Text::Kakasi) and the Kakasi package (some binary files and Japanese dictionary files). PPM can be used to install the Perl interface for Kakasi but the Windows version of the Kakasi package has to be downloaded separately.

Further information about the Kakasi package is available from the KAKASI (Kanji Kana Simple Inverter) site.

The current 32-bit Windows version of the Kakasi package is v2.3.4.

Create a directory called “kakasi_package” in the “C:\Build\POPFile\windows” directory.

The Kakasi package is distributed as a ZIP file which contains several folders (bin, doc, include, lib and share) all of which are under a top level folder called kakasi. Unzip the Kakasi package into the new C:\Build\POPFile\windows\kakasi_package directory.

The C:\Build\POPFile\windows\kakasi_package directory should now contain a directory called kakasi which contains the files and sub-directories for the Kakasi package.

POPFile uses an SQLite database to hold essential information, such as the corpus used to classify messages. The SQLite library includes a simple command-line utility that allows the user to manually enter and execute SQL commands against an SQLite database. This utility can also be used to check the integrity of the database and defragment it.

POPFile currently uses SQLite 3.x format databases which are not compatible with the SQLite 2.x format used by some earlier versions of POPFile. In addition to the SQLite 3.x format command-line utility (sqlite3.exe) the installer also installs the SQLite 2.x format utility (sqlite.exe). Both utilities are available from the official SQLite website.

The installer usually includes SQLite command-line utilities which match the library versions used by the DBD::SQLite and DBD::SQLite2 modules it installs.

Notes:

1. DBD::SQLite v1.25 is the version used for the POPFile 1.1.1 release

1. DBD::SQLite v1.29 is the version used if the instructions on this page are used to assemble the minimal Perl

1. DBD::SQLite2 v0.33 is the most recent SQLite 2.x format module in the ActiveState repository and v2.8.17 is the most recent SQLite 2.x format command-line utility.

Extract the “sqlite.exe” and “sqlite3.exe” files from the appropriate zip files and store them in the “C:\Build\POPFile\windows” directory.

The release notes are not stored in the project's Subversion repository. The release notes are plain text files and should be stored in the C:\Build\POPFile\engine directory.

Use a text editor to create a plain text file called v1.1.1.change (for the English release notes) and another called v1.1.1.change.nihongo (for the Japanese release notes; if you don't know Japanese just use a copy of the v1.1.1.change file).

If the files checked out from Subversion are marked with a release number other than 1.1.1 then change the filenames of these two files to match (e.g. v1.1.2.change and v1.1.2.change.nihongo)

Either build the six popfile*.exe files (may need to edit the engine\vars.mak file) or use the versions available from Subversion (or a suitable official release?)

Add instructions here (use RC=“-SVN” option?)

make winexe
make xplat
make windows

make winexe
make xplat RC="-RC1"
make windows RC="-RC1"