Failed to Open Stopwords file

[Larry Betz]

I just realized my Popfile has not been working fo a couple days. I restarted PF and then checked the log. Here is what I found:

2015/10/4 12:00:44 4824: POPFile stopped 2015/10/4 12:00:44 4824: --------------- 2015/10/4 12:01:28 1576: ----------------------- 2015/10/4 12:01:28 1576: POPFile v1.1.3 starting 2015/10/4 12:01:28 1576: bayes: 811: Attempting to connect to dbi:SQLite:dbname=C:\PROGRA~2\POPFile/popfile.db (1) 2015/10/4 12:01:28 1576: bayes: 912: Using SQLite library version 3.7.2 2015/10/4 12:01:30 1576: wordmangle: 88: Failed to open stopwords file 2015/10/4 12:02:12 1576: IMAP-Client: 177: Connected to imap.1and1.com:993 timeout 60 2015/10/4 12:02:31 1576: IMAP-Client: 177: Connected to imap.1and1.com:993 timeout 60 2015/10/4 12:09:42 1576: IMAP-Client: 177: Connected to imap.1and1.com:993 timeout 60 2015/10/4 12:10:23 1576: IMAP-Client: 177: Connected to imap.1and1.com:993 timeout 60

Suggestions?

Thanks, Larry

[Larry Betz]

Here is my PFIdiag file:

---

POPFile PFI Diagnostic Utility v0.3.0 (full mode)

---

String data report format (not used for numeric data)

string not found : >< empty string found : < > string with 'xyz' value found : < xyz >

---

Current UserName = Administrator (Admin)

Operating System = Windows (64-bit) IsNT return code = 1 Internet Explorer = 9.1

---

Location used to store temporary files

---

$TEMP folder path = < C:\Users\ADMINI~1\AppData\Local\Temp\1 >

---

Start Menu Locations

---

AU: $SMPROGRAMS = < C:\ProgramData\Microsoft\Windows\Start Menu\Programs > AU: $SMSTARTUP = < C:\ProgramData\Microsoft\Windows\Start Menu\Programs\Startup >

Search results for the "AU: $SMSTARTUP" folder:

*.lnk files found = 0 POPFile shortcuts = 0

CU: $SMPROGRAMS = < C:\Users\Administrator\AppData\Roaming\Microsoft\Windows\Start Menu\Programs > CU: $SMSTARTUP = < C:\Users\Administrator\AppData\Roaming\Microsoft\Windows\Start Menu\Programs\Startup >

Search results for the "CU: $SMSTARTUP" folder:

Shortcut name = < Run POPFile.lnk > Shortcut start in = < C:\Program Files (x86)\POPFile > Shortcut target = < C:\Program Files (x86)\POPFile\runpopfile.exe > Shortcut argument = < /startup > Target status = found

*.lnk files found = 5 POPFile shortcuts = 1

---

Obsolete/testbed Registry Entries

---

[1] Pre-0.21 Data:

Pre-0.21 POPFile = >< Pre-0.21 Testbed = ><

[2] 0.21 Test Installer Data:

HKLM: RootDir_LFN = >< HKLM: RootDir_SFN = ><

HKCU: RootDir_LFN = >< HKCU: RootDir_SFN = >< HKCU: UserDir_LFN = >< HKCU: UserDir_SFN = ><

[3] Current PFI Testbed Data:

MRI PFI Testbed = >< MRI PFI Testdata = ><

---

POPFile Registry Data

---

NTFS SFN Disabled = < 2 >

HKLM: MRI Version = < 1.1.3 >

HKLM: NewParser = ><

HKLM: InstallPath = < C:\Program Files (x86)\POPFile > HKLM: RootDir_LFN = < C:\Program Files (x86)\POPFile > HKLM: RootDir_SFN = < C:\PROGRA~2\POPFile >

HKLM: *.exe count = 6 (this is OK)

HKCU: Data Owner = < Administrator > HKCU: MRI Version = < 1.1.3 > HKCU: RootDir_LFN = < C:\Program Files (x86)\POPFile > HKCU: RootDir_SFN = < C:\PROGRA~2\POPFile >

HKCU: UserDir_LFN = < C:\Program Files (x86)\POPFile > HKCU: UserDir_SFN = < C:\PROGRA~2\POPFile >

HKCU: popfile.pl = found HKCU: popfile.cfg = found

HKCU: *.exe count = 6 (this is OK)

---

POPFile Corpus/Database Backup Data

---

HKCU: backup locn = < C:\Program Files (x86)\POPFile\backup >

backup.ini file = found Flat-file folder = not found Flat / BDB folder = not found SQLite DB backup = found

---

POPFile Logger Settings

---

Logger output = < 1 > (To File) Logger format = < default > Logger level = < 0 > Logger directory = < ./ > (C:\Program Files (x86)\POPFile)

---

POPFile Environment Variables

---

'POPFILE_ROOT' = < C:\PROGRA~2\POPFile > 'POPFILE_USER' = < C:\PROGRA~2\POPFile >

Env: popfile.pl = found Env: popfile.cfg = found

ROOT: *.exe count = 6 (this is OK)

'ITAIJIDICTPATH' = >< (this is OK) 'KANWADICTPATH' = >< (this is OK)

'MECABRC' = >< (this is OK)

---

POPFile Configuration Settings (subset)

---

POPFile UI port = < 8082 > POP3 Listen port = < 110 > Concurrent POP3 = < 0 >

---

(report created 04-Oct-2015 @ 12:35:16)

---

[Brian Smith]

2015/10/4 12:01:30 1576: wordmangle: 88: Failed to open stopwords file

The stopwords file is a simple text file containing a list of words that POPFile will ignore if it finds them during classification of a message. These words are likely to be used in any sort of message, such that they are not likely to be useful in classification.

POPFile is shipped with a default 'stopwords' file (containing words like this, and, that, etc) but it will still work if this file is empty or missing ~ so this entry in the log is really just a warning.

According to the PFI Diagnostic Utility's report your installation expects to find the following 'stopwords' file:

C:\Program Files (x86)\POPFile\stopwords

If your installation was using a 'stopwords' file which has now been deleted then this may have an adverse effect upon classification accuracy. The installer creates a backup copy of the initial 'stopwords' file so if you want to restore this backup all you have to do is make a copy of the 'stopwords.default' file and call the copy 'stopwords'. The 'stopwords.default' file should be in the "C:\Program Files (x86)\POPFile" folder.

If you did not deliberately remove this file then it is possible that whatever removed it also removed some other more important files from your POPFile installation.

The Windows version of POPFile 1.1.3 is no longer compatible with some SSL servers (e.g. Microsoft's *.live.com and outlook.com accounts). This is because POPFile 1.1.3 does not use an up-to-date version of the OpenSSL library. An SSL updater has been produced as a temporary fix until POPFile 1.1.4 is released.

I was going to suggest you run this updater (using the download link on the home page) but it looks like your POPFile installation was able to connect to the IMAP server using SSL:

2015/10/4 12:02:12 1576: IMAP-Client: 177: Connected to imap.1and1.com:993 timeout 60

However by default POPFile's log file does not show much detail so it is possible that something went wrong with the connection to the server. If you set the logger_level to 1 then the log will contain more detail about what POPFile is doing. Do not set the logger_level to 2 for now as this will generate huge log files ~ logger_level 1 should provide some more information about what is going wrong.

The online manual has a page explaining how to change the logger_level and another page showing examples of log file entries.

[Larry Betz]

I found Stopwords.default file. Does this look correct... The contents is:

gone smtp status plaintext applet edu oct it's embed helvetica param map cdt tue height you strike our del going received esmtp ltd width not person nov thead head marquee she message pdt com fri are return yet his from blink samp kbd mail note sub has frame spot jul may alt cite center nbsp subject dir address the basefont doing caption being frameset xmp form mailto date went www big sup jun path listing align org will link serif var cellspacing could isindex goes input and inc script pre that meta anotherbigword title nobr sat select span dfn encoding mon blockquote gmt strong est jan for cgi did apr been have base math had bgcolor fig any author having feb dec html sep this valign off with thu net range would can color but font was menu abbrev table sun tbody ask https wed tfoot localhost charset lang body wbr textarea http col spacer iframe img acronym src helo him colgroup div done advanced out pst aug your small tab yes noframes its mar ins multicol etc also code does area banner her were all edt cst textflow overlay bgsound sans border mbox htm header:From header:To header:Subject

[Brian Smith]

Yes, the file looks OK.

The original stopwords file used for the POPFile 1.1.3 release can be seen at http://getpopfile.org/browser/tags/v1_1_3/engine/stopwords and there is a download link at the bottom of that page.

The Windows installer always creates a 'stopwords.default' backup copy of this file (to make it easy to restore it).

[Larry Betz]

I re-ran the installer over top of my 1.1.3. I did not receive any errors. But there is a new item in the log:

html: 345: HTTP error 500 returned

Here is my log file right after it restarted:

2015/10/4 20:46:24 7828: POPFile stopped 2015/10/4 20:46:24 7828: --------------- 2015/10/4 20:48:42 4864: ----------------------- 2015/10/4 20:48:42 4864: POPFile v1.1.3 starting 2015/10/4 20:48:42 4864: bayes: 811: Attempting to connect to dbi:SQLite:dbname=C:\PROGRA~2\POPFile/popfile.db (1) 2015/10/4 20:48:42 4864: bayes: 912: Using SQLite library version 3.7.2 2015/10/4 20:48:44 4864: wordmangle: 88: Failed to open stopwords file 2015/10/4 20:48:44 4864: html: 345: HTTP error 500 returned 2015/10/4 20:49:25 4864: IMAP-Client: 171: Connected to imap.1and1.com:993 timeout 60

[Brian Smith]

A WikiFormatting link is provided above the ticket comment box

[Brian Smith]

It would be easier to read your log file extracts if you used {{{ and }}} as described in the Preformatted Text section of the WikiFormatting page which is mentioned above the comment box (see attached screenshot)

html: 345: HTTP error 500 returned

That might be harmless ~ I think I've seen that a few times, especially when installing from scratch. I'm currently rebuilding my development system and don't have access to all of my development history so I cannot confirm this.

2015/10/4 20:48:44 4864: wordmangle: 88: Failed to open stopwords file

Does the "C:\Program Files (x86)\POPFile\stopwords"" file exist?

If the installer is upgrading an existing installation and finds that the file is missing it will ask you if you want to leave things as they are (i.e. no stopwords file will be installed) or use the default stopwords file.

In addition to setting the logger_level to 1 you could try shutting down POPFile and using the Message Capture utility to start POPFile ~ this will display some messages from POPFile that are normally hidden. The installer creates a Start Menu shortcut for the utility:

Start -- All Programs -- POPFile -- Support - Message Capture utility