Opened 11 years ago

Last modified 11 years ago

#173 new defect

Popfile exits unexpectedly

Reported by: yorkshirejumbo Owned by: Manni Heumann
Priority: normal Milestone:
Component: IMAP Version: 1.1.3
Severity: normal Keywords:
Cc:

Description

I've just spent a few days checking my PC for disk errors due to a couple of BSOD on Monday. I found and corrected a few problems, caused by a loose wire to a disk after replacing another disk.

When I started the PC back into Vista this morning, I had >250 messages awaiting classification by Popfile IMAP. After processing over half the messages, Popfile exited. It did this a couple more times before I ran the checkdb utility and then ran it with the console. It ran with minimal CPU for a couple of minutes before going to high CPU on one processor for several seconds and then exiting. Here is the output:

------------------------------------------------------------
POPFile Message Capture Utility v0.2.0
------------------------------------------------------------
POPFILE_ROOT   = C:\PROGRA~1\POPFile
POPFILE_USER   = C:\Users\Charlie\AppData\Roaming\POPFile
Using 'popfileif.exe' to run POPFile
------------------------------------------------------------
(report started 07-Dec-2011 @ 10:08:48)
------------------------------------------------------------

POPFile Engine loading

    Loading... 
         {core: windows}
         {core: config history logger mq} 
         {classifier: bayes wordmangle} 
         {interface: html} 
         {proxy: nntp pop3 smtp} 
         {services: imap} 

POPFile Engine v1.1.3 starting

    Initializing... 
         {core: config history logger mq windows} 
         {classifier: bayes wordmangle} 
         {interface: html} 
         {proxy: nntp pop3 smtp} 
         {services: imap} 

    Starting...     
         {core: config history logger mq windows} 
         {classifier: bayes wordmangle} 
         {interface: html} 
         {proxy: pop3} 
         {services: imap} 

POPFile Engine v1.1.3 running
Out of memory!
Callback called exit at C:\PROGRA~1\POPFile/POPFile/Module.pm line 767.


POPFile Engine v1.1.3 stopping

    Stopping... 
         {classifier: bayes wordmangle} 
         {core: config history logger mq windows} 
         {interface: html} 
         {proxy: pop3} 
         {services: imap} 

POPFile Engine v1.1.3 terminated

------------------------------------------------------------
Status code: 1
------------------------------------------------------------
(report finished 07-Dec-2011 @ 10:14:42)
------------------------------------------------------------

The only thing slightly off is "wordmangle", but searching through the help for this didn't produce much help. Any help gratefully accepted...

Change History (4)

comment:1 by yorkshirejumbo, 11 years ago

Bit more info. I ran the full PFI utility, which didn't show anything obvious. I changed the logger level to 2 and ran it again. The process manager said the process's peak working set (memory) went up to >820,000k just before stopping. The last few entries in the log are:

2011/12/7 10:35:12 5304: bayes: 659: Read slurp data es is limited to 10 device=[0d][0a]s.[0d][0a]=09=09=09</div>[0d][0a]=09=09</td>[0d][0a]=09</tr>[0d][0a]</table>[0d][0a]<!-- END NOTE SECTION --><!--  /67435 -->[0d][0a]<!-- 67435 - linesep -->[0d][0a]<!-- Sta
2011/12/7 10:35:12 5304: bayes: 659: Read slurp data rt sep note table -->[0d][0a]<table width=3D"698" border=3D"0" cellspacing=3D"0" cellpadding=3D"0">[0d][0a]=09<tr>[0d][0a]=09=09<td height=3D"12"><img src=3D"http://images.apple.
2011/12/7 10:35:12 5304: bayes: 659: Read slurp data com/cbx/eu/11/12/34=[0d][0a]67/v1/uk/i/spacer.gif" width=3D"1" height=3D"12" style=3D"display:block; ma=[0d][0a]rgin:0;"></td>[0d][0a]=09</tr>[0d][0a]</table>[0d][0a]<!-- End sep note table 
2011/12/7 10:35:12 5304: bayes: 659: Read slurp data --><!-- /67435 - linesep -->[0d][0a]<!--  67429 -->[0d][0a]<!-- START NOTE SECTION -->[0d][0a]<table width=3D"698" border=3D"0" cellspacing=3D"0" cellpadding=3D"0" align=[0d][0a]=3D"ce
2011/12/7 10:35:12 5304: bayes: 659: Read slurp data nter" id=3D"aapl-footer">[0d][0a]=09<tr>[0d][0a]=09=09<td valign=3D"top" align=3D"left" style=3D"padding:0 20px 0 20px;">[0d][0a]=09=09=09<div style=3D"font-family: Geneva, Verda
2011/12/7 10:35:12 5304: bayes: 659: Read slurp data na, Arial, Helvetica, sans=[0d][0a]-serif;  font-size:9px; line-height:1.34em; color:#999999;">[0d][0a]=09=09=09=09Some products and promotions are not available in all cou
2011/12/7 10:35:12 5304: bayes: 659: Read slurp data ntries=[0d][0a].[0d][0a]=09=09=09=09 <!--  67430 -->[0d][0a][0d][0a]=09=09=09=09=09=09=09=09Product specifications are subject to change.[0d][0a]=09=09=09=09=09=09<!--  /67430 -->[0d][0a][0d][0a]=09=09=
2011/12/7 10:36:31 5304: Perl fatal error : Callback called exit at C:\PROGRA~1\POPFile/POPFile/Module.pm line 767.[0a]
2011/12/7 10:36:32 5304: POPFile stopped
2011/12/7 10:36:32 5304: ---------------
2011/12/7 10:36:32 5304: mq: 380: post RELSE (0iRB13l2O760ecCp3VcO)
2011/12/7 10:36:32 5304: mq: 387: queuing post RELSE (0iRB13l2O760ecCp3VcO)
2011/12/7 10:36:32 5304: mq: 389: RELSE queue length now 0
2011/12/7 10:36:32 5304: imap: 461: Trying to disconnect all connections.
2011/12/7 10:36:32 5304: IMAP-Client: 579: Logging out
2011/12/7 10:36:32 5304: IMAP-Client: 334: << A00018 LOGOUT[0d][0a]
2011/12/7 10:36:32 5304: IMAP-Client: 659: Read slurp data * BYE LOGOUT Requested[0d][0a]A00018 OK 73 good day (Success)[0d][0a]
2011/12/7 10:36:32 5304: IMAP-Client: 439: >> * BYE LOGOUT Requested[0d][0a]
2011/12/7 10:36:32 5304: IMAP-Client: 427: >> A00018 OK 73 good day (Success)[0d][0a]
2011/12/7 10:36:32 5304: IMAP-Client: 287: IMAP-Client is exiting

comment:2 by yorkshirejumbo, 11 years ago

Trying to find if there was a particular message which caused the problem, I've found that Popfile has classified a load of messages as spam when they should have been moved to other folders in gmail or left in the inbox.

comment:3 by yorkshirejumbo, 11 years ago

Bit more research - I've noticed that the stored messages suddenly started increasing in size after processing 170 messages. Before then, virtually all messages were <100kb. The final messages stored were >400Mb with huge amounts of duplicated text, as though the message read process had got itself into a loop.

comment:4 by yorkshirejumbo, 11 years ago

Tried deleting the user info, then recreating it, then repopulating with old popfile.db and .cfg, but still crashed. Then I replaced popfile.db with a popfile.db.backup that had been created before the PC crash, again without success. I've now reinstalled popfile completely and it's gone through fine. Now all I have to do is recreate my many years' worth of corpus...

Note: See TracTickets for help on using tickets.