Empty page due to the use of unitialized value in substitution and pattern match.

[zephryus]

Recently, I upgraded my POPfile version from v1.1.0 to v1.1.1

But now, I notice the following lines are dumped to the invoking console when I hit [filter] button or delete the page button sometimes and NOT always.

Use of uninitialized value in substitution (s///) at /extra/ishikawa/download/POPFILE-DIR//UI/HTML.pm line 396.
Use of uninitialized value in substitution (s///) at /extra/ishikawa/download/POPFILE-DIR//UI/HTTP.pm line 251.
Use of uninitialized value in pattern match (m//) at /extra/ishikawa/download/POPFILE-DIR//UI/HTTP.pm line 253.

These are shown in triples whenever the mentioned problem appears.

HTML.pm

  395	    if ( $command =~ /POST/i ) {
  396	        $content =~ s/[\r\n]//g;
  397	        $self->parse_form_( $content );

HTTP.pm

  250	
  251	    $arguments =~ s/&/&/g;
  252	
  253	    while ( $arguments =~ m/\G(.*?)=(.*?)(&|\r|\n|$)/g ) {
  254	        my $arg = $1;
  255	

And when such messages are dumped, the web GUI page shown anew is blank. If I reload the page (the browser shows the warning about resubmitting the request), then the new page is correctly shown with the listing of other messages if any, or an empty list, i.e., no matching message is there and so the list is empty.

I would think that the $argument in HTTP.pm and $component in HTML.pm are already blank before the evaluation is done on each of the Perl source file lines. And maybe my perl setting is set to halt(?) when a strict checking raises an error or something. But I am not sure what the current perl setting is. The PC runs old Fedora distribution, but I upgraded the necessary Perl modules using CPAN and installed the POPfile script and such by downloading this from getpopfile.org download page.

TIA

[Brian Smith]

Thank you for the bug report.

I use a modified version of POPFile on my Windows system and sometimes see a similar problem. I had thought the problem was caused by my modifications.

The problem does not happen very often (once every few months). I have not found a way to reproduce the problem. This makes it difficult to investigate the problem.

Which browser do you use to display the message history page?

On my Windows system I have found that some POPFile UI problems only appear when I use the Opera browser.

[zephryus]

So it is not a problem seen by me only. What a relief :-)

I use Firefox browser from a different computer.

The POPFile runs on a different linux PC and I use Firefox browser on a linux guest inside a VirtualBox instance on a different PC (that runs XP).

Oh, by the way, I upgraded the perl installation on the PC where POPFile runs to see if it solves the problem. But I got the same error after I upgraded perl and so I suspect that the problem is in the POPFile itself.

TIA

[naoki iimura]

I've updated UI/HTTP.pm to solve the problem: [3667] I hope the change works.

Naoki

[zephryus]

Thank you for the update.

I will check and see if the modification eliminates the problem. (But I have to download the special "branch" distribution, correct?)

TIA

[Brian Smith]

Naoki only changed one file so all you need to do is add this file to your existing POPFile 1.1.1 installation.

Shutdown POPFile then replace the existing UI/HTTP.pm file with the new version.

If you make a backup copy of the original file (e.g. UI/HTTP.pm.bak) then it will be easy to go back to the original file if the new version does not work.

There are several ways to get the new file:

(1) You can download the latest popfile_0_22_2.tgz tarball and extract the new UI/HTTP.pm file

(2) You can use a Subversion client to download the new UI/HTTP.pm file from the b0_22_2 branch of the Subversion repository. See How to obtain the source code for further details, including the repository URLs

(3) You can use your browser to download the UI/HTTP.pm file from our source code browser using the "Original Format" link at the bottom of the page found at

http://getpopfile.org/browser/branches/b0_22_2/engine/UI/HTTP.pm

Option (3) is probably the easiest way to get a single file from our Subversion repository.

[zephryus]

Thank you for the pointer. I will change and report my findings. (This particular POPfile is installed on an office PC, and we are already on weekends in Japan.)

[zephryus]

Dear naoki and brian,

Thank you for the patch and the pointer to apply the patch.

Well, a little progress.

Here is what I found out.

I downloaded the modified HTTP.pm and replaced it, and restarted POPfile.

During my testing, one type of error is now reported occasionally like the following (now at one line of code) instead of at three locations.

POPFile Engine v1.1.1 running
Use of uninitialized value $content in substitution (s///) at /download/POPFILE-DIR//UI/HTML.pm line 396.
Use of uninitialized value $content in substitution (s///) at /download/POPFILE-DIR//UI/HTML.pm line 396.

If this error is reported after deleting a page-full of article from history, I get a blank page, and if I reload the page, I am now back to a non-empty LIST of the remaining articles (in a particular bucket).

Previously, if I recall correctly, most of the time, the blank screen goes back to the EMPTY list of the articles in a particular bucket after deleting a page full of articles. (That is I noticed the problem occurred when the remaining list after a deletion of articles is empty. But come to think of it, my memory may not be that great.)

Now, looking at HTML.pm, I suspect $content is also not-set(?) after a screenful of articles is deleted, and just before the next screenful was going to be displayed? I am not sure.

HTML.pm:

  394
  395	    if ( $command =~ /POST/i ) {
  396		$content =~ s/[\r\n]//g;
  397		$self->parse_form_( $content );
  398	    }
  399

this is in subroutine url_handler:

It is called thusly:

/download/POPFILE-DIR/UI/HTML.pm:239:    $self->{url_handler_} = \&url_handler__;
/download/POPFILE-DIR/UI/HTML.pm:347:# url_handler__ - Handle a URL request
/download/POPFILE-DIR/UI/HTML.pm:360:sub url_handler__
/download/POPFILE-DIR/UI/HTTP.pm:229:    return $self->{url_handler_}( $self, $client, $url, $command, $content );

Does the call to handle_url contain correct number of arguments? I bet.

>
>                        $code = $self->handle_url( $client, $2, $1, $content );
>                        $self->log_( 2,                                # PROFILE BLOCK START

Well, I am not sure what will fix the problem.

But if I added a few lines of code in HTML.pm like this, the problem is not seen at least for the day. (I have deleted all the spam articles from history during testing. So I can only check more tomorrow.)

sub url_handler__
{
    my ( $self, $client, $url, $command, $content ) = @_;

    if ( !defined $content ) {    # These are additions. 
	$content = '';            #
    }	                          # 

This seems like a bandage fix.

I don't follow the logic well, but what if $content_length is 0 when we hit the call of handle_url in HTTP.pm here. Can $content be undefined or in an undesirable state???

                    if ( $content_length > 0 ) {
                        $content = $self->slurp_buffer_( $client,  # PROFILE BLOCK START
                            $content_length );                     # PROFILE BLOCK STOP
                        $self->log_( 2, $content );
                    }

                    if ( $request =~ /^(GET|POST) (.*) HTTP\/1\./i ) {
                        $code = $self->handle_url( $client, $2, $1, $content );
                        $self->log_( 2,                                # PROFILE BLOCK START
                            "HTTP handle_url returned code $code\n" ); # PROFILE BLOCK STOP
                    } else {

But reading the code, I don't see how it can be unset. Hmm...

Some kind of weird timing problem?

FYI, I am using a POPfile instance running on a linux PC, and then my TB3 (mail client) is running inside a linux guest inside Oracle Virtual Box under Windows XP on a different PC.

As a last resort, I can run wireshark or something to capture the packet to see what happens when the error is observed.

TIA

[zephryus]

OK, I am not a perl hacker, but I found out that slurp_buffer that returns the data to $content CAN return "undef" value, it seems. See the definition in POPFile/Module.pm

sub slurp_buffer_
{
    my ( $self, $handle, $length ) = @_;

    while ( $self->slurp_data_size__( $handle ) < $length ) {
        my $c;
        if ( $self->can_read__( $handle, 0.01 ) && ( sysread( $handle, $c, $length ) > 0 ) ) {
            $slurp_data__{"$handle"}{data} .= $c;
        } else {
            last;
        }
    }

    my $result = '';

    if ( $self->slurp_data_size__( $handle ) < $length ) {
        $result = $slurp_data__{"$handle"}{data};
        $slurp_data__{"$handle"}{data} = '';
    } else {
        $result = substr( $slurp_data__{"$handle"}{data}, 0, $length );
        $slurp_data__{"$handle"}{data} =                       # PROFILE BLOCK START
            substr( $slurp_data__{"$handle"}{data}, $length ); # PROFILE BLOCK STOP
    }

    return ($result ne '')?$result:undef;
}

So for a reason unfathomable to me, the value of $result remains

''

, its initial value (assigned in the middle), at the end of the execution, then slurp_buffer decides to return "undef", which presumably causes the "Uniitialized" warning for the use of $content (!).

A suspicious behavior.

I will leave the rest to people in the know.

[Brian Smith]

Thanks for the updates. Sorry to hear that you are still having trouble.

I installed the new HTTP.pm file on my Windows system on Saturday 11 September and have not seen any console error or warning messages since then (it is now Wednesday 15 September).

I clear my HISTORY page several times a day. Perhaps the problem only shows up when there is more than one page of HISTORY. I will stop clearing my HISTORY to see if this makes any difference.

Brian

[Brian Smith]

With over 400 messages in my HISTORY no console messages appeared as I tried using the UI to view, sort and delete messages. I used Single Message View and deleted messages using the checkboxes, remove page, remove all, search filters, bucket filters, inverted searches, and inverted bucket filters.

These tests were run on my Windows system using Naoki's patch and Perl 5.8.9 Build 826

Brian

[naoki iimura]

It seems that the web browser sends invalid data (the header has non-zero Content-Length but POPFile does not receive any content).

I've written another patch for HTTP.pm to return 400 (Bad Request) error if web browsers send such invalid requests. [3677]

You can get the new HTTP.pm from here: http://getpopfile.org/browser/branches/b0_22_2/engine/UI/HTTP.pm?rev=3677

I hope the new code solves your problem.

Naoki

[zephryus]

Thank you for looking into this old problem. I have been using Firefox and am surprised to hear that it may send invalid data occasionally. But over the years, firefox suffered from many different bugs, and so come to think of it, it may be the cause of the problem.

Since Japan, where I live, is in the middle of long holiday week (people call it Golden Week), I will test the patch this Friday or Monday.

Strangely, I have not seen this bug for the last few months. (To be honest, I have come to rely on POPfile and most of the times, POPfile happily classifies incoming mails correctly. Only when a new correspondent starts with a new party with a different set of topics, I need to jump in and correct the notion of category POPfile has about certain e-mails. So the frequency of using web interface to POPfile has dwindled than before.)

This reduced frequency of the observation of the bug may be due to

• less frequent use of POPfile GUI (like I explained above),
• updates of firefox,
• updates of Debian GNU/Linux under which firefox runs,
• updates of VirtualBox in which the guest Debian GNU/Linux instance runs.

(Hmm, so many improvements or changes on the client side.)

POPfile runs on a different PC that runs old version of Fedora, but the versions of POPfile, Perl, OS have remained the same since my first report.

I will report back once the patch is applied and test is done.

TIA

[zephryus]

This has been fixed in the long time between my last report and today.

Sorry to fail to update the bug entry! I just thought of submitting another bug report today, and while looking for a similar bug report, I stumbled upon my old bug entry.

Thank you again for making this great package available to wide community.