Help → PF + GMail POP3 & IMAP + Thunderbird: help!

PF + GMail POP3 & IMAP + Thunderbird: help!

I once installed, configured, and used PopFile? to great (99.97%) effect, but then I drifted or was forced away. I'm trying to return to using it again, having figured out finally how to bypass GMail's god-awful spam filtering. However, I can't navigate all the complexities involved now, with authentication and everything else. I spent two evenings trying to figure it out, and failed. I'm begging for help.

I would like to configure PopFile? to work through Thunderbird. I have one primary and several secondary GMail accounts, and one with AT&T. I understand that I may only be able to filter one of them with PopFile?, and if that is the case I would choose the one primary GMail account. I have both POP3 and IMAP service enabled for it; I enabled IMAP to allow me to manage the spam folder without having to mess with the Webmail interface. I had little need for IMAP otherwise at the time, but I've read that PopFile? can do some neat tricks with IMAP, so I may have other reasons to keep it enabled now? If possible, then, I might want to configure PopFile? with both protocols on that same GMail account; if not, then I'll stick with just POP3. I also installed the SMTP and NNTP interfaces, though I'm on the fence whether I need them.

I also have Proxomitron as an HTTP filter. Because of its presence and use of port 8080, I (belatedly) configured PopFile? to use port 9090 instead, and set up an HTTP bypass for "localhost" in Proxomitron, so that I can access the PopFile? Web interface. I also have AVG Free, though I have its e-mail scanner disabled at the moment; I will probably leave it so unless I return to using Outlook 2002 instead of Thunderbird. (I've been considering doing that because of persistent annoying bugs in Thunderbird that aren't getting fixed.)

If someone can help me navigate the required setup in PopFile? and Thunderbird (and GMail?) to make this possible, I would be appreciative.

  • Message #661

    Hi.

    It sounds like you will have to make up your mind about how you are going to access all those accounts and how POPFile should enter the equation before you start configuring things.

    Do you want to use POP3 for your primary account or do you want to leave the messages on the server and use IMAP instead?

    Are you planning to access all of those accounts separately or are you going to set up forwarders so that everything ends up in only one INBOX?

    I can tell you right away that you are not going to need the nntp and smpt modules. Also, making POPFile access 1 account through both IMAP and POP3 is not going to work.

    • Message #665

      Let's say that I just want to set it up for use with Thunderbird and GMail with POP3 access. How do I configure PopFile? and how do I configure Thunderbird? I couldn't navigate the seemingly conflicting settings in PopFile? and instructions for Thunderbird for that scenario, given that GMail now requires authentication and doesn't use the traditional POP3 port. What instructions I could find were just not clear and helpful to do that, and there are too many configuration options to make it practical to figure it out the hard way. It's no longer intuitive the way it once was.

      • Message #669

        There isn't any special configuration needed for POPFile. Just set up your buckets.

        To get to the non-default port, you will have to change your Thunderbird account settings. The user/login name is the field to change. This is described here.

        • Message #672

          I had already done that, but it wasn't apparent at the time that the port value in the Thunderbird account setting needed to be changed to 110. THIS IS NOT SPECIFIED in the PopFile? instructions for Thunderbird. Naturally it was completely failing to communicate. I have changed the port value from 995 to 110, but there's still a problem: now it attempts to connect to 127.0.0.1 but times-out. PopFile? is active because I can access the UI.

          Is this happening because this is still not taking the SSL into account? I'm not clear how to do that. I installed the SSL module, but I'm not clear how this all fits together and the documentation is not helping.

          I'm telling you, the documentation is poorly written in places for certain purposes. Take the section on AVG, for instance: it fails to specifically address the actual settings in the current AVG interface. A user who is not an expert - which apparently I am still not - is unable to translate the instructions into actual changes in AVG. I know just enough to recognize that the documentation for that isn't doing the job, but not enough to rewrite it myself.

          • Message #673

            I figured out finally why it was timing out: Thunderbird was CONNECTING to PopFile? but not COMMUNICATING with it, because it was trying to use SSL to do it and PopFile? wasn't listening. I had to change the Security Settings from SSL back to Never, and then add :SSL *back* into the username string. I had done this much earlier, but it failed to work because it wasn't clear to me at the time that I also needed to change that security setting. Of course at the time I had also not changed the PORT in the account settings, because the documentation didn't mention doing so at all and my comprehension of the process was poor.

            Are you beginning to see my point yet? The documentation is written from the narrow perspective of SOMEONE WHO ALREADY UNDERSTANDS the whole process, how the various parts of the process fit together. Understanding how it all fits is critical to making the correct changes in a mail client and PopFile? (and other pieces like AVG and even Proxomitron). For someone who doesn't grasp that process, the documentation can be next to useless. I'm more knowledgeable than a user off the street, but I was very rusty and the documentation did not make it easy for me to relearn what I needed to know about the process. In my case, I have more pieces in the process than most: PopFile?, Proxomitron, AVG (which is STILL disabled because I can't make sense of what to do), GMail, SSL, and Thunderbird (and IMAP as well).

            The documentation needs to be rewritten from the perspective of someone who DOESN'T understand the process and technologies involved.

            • Message #674

              The documentation needs to be rewritten from the perspective of someone who DOESN'T understand the process and technologies involved.

              Well maybe you are right (although I think you are exaggerating a bit here), but wouldn't the POPFile developers be the least suitable people for that job?

              • Message #675

                I suppose I should just be happy that POPFILE FINALLY FILTERED! the mail notification about your reply just now!

                I don't think I was exaggerating at all. Everything is "in there" - with the exceptions I noted - but the bits and pieces are scattered and not integrated. The effect of that is not unique to me: I happened to skim a post about SSL just a few minutes ago, and the eventual resolution occurred because the person realized the answer was actually in the documentation but not where he could find it or expected to find it (where it was useful to his limited understanding).

                I don't see why the PopFile? devs can't just take a step back and "forget" what they know. It's a simple(?) matter of thinking through the whole process from scratch, as it applies specifically to various scenarios like mine, and then writing a set of start-to-finish instructions *for each specific scenario*. In other words, integrate the pieces into sets of master instructions, so that the user doesn't have to scratch his (bald) head and try to find them all buried in the docs and FAQ and wherever. At the very top of it all would be descriptions of each one of those scenarios, with links to the specific instructions to deal with each scenario.

                My first visit to the documentation should have, ideally, presented me with a page listing all the possible scenarios that could involve PopFile?; I should have been able to click on the one that most closely described my setup and then been taken to complete instructions how to do it, all on one page (or referenced from it). Yes, of course this may possibly involve a lot of duplication (unless a lot of linking is used, and that may not be good enough), but that is what is required to actually help the user, which is after all the whole point of the docs, right? It might be that the reason PopFile? has never achieved widespread adoption by the general public precisely because the documentation and process is intimidating to people who don't understand it; if someone can barely manage to set up Thunderbird to get their e-mail from a GMail account, how much harder will it be for them currently to add PopFile? to that process? This stuff is HARD for the average user to grasp.

                Maybe I could help with that later, but I'm not knowledgeable enough to do it yet. I need a major chunk of what the devs already know before I could do it effectively. I might not have personal time to learn it now.

          • Message #681

            I'm telling you, the documentation is poorly written in places for certain purposes.

            We know the documentation is not perfect; this is why the footer of every page says "Should you find anything in the documentation that is incomplete, unclear, outdated or just plain wrong, please let us know and leave a note in the Documentation Forum."

            Take the section on AVG, for instance: it fails to specifically address the actual settings in the current AVG interface.

            It would help if you supplied links to the specific pages you are complaining about. If you can supply links please do so in the Documentation Forum.

            If you are referring to the Using the AVG Personal Email Scanner (AVG EMS) with POPFile page it was written for use with the Free Edition of AVG Anti-Virus v7. However that page includes a link to the AVG 8 FAQ page where AVG provide instructions for providing SSL support.

            There is a very simple reason why that page does not say much about AVG 8 - I'm still using AVG 7.5 and nobody has bothered to supply the information needed to update the page for use with AVG 8.

            If you have several Gmail accounts then a more flexible solution may be to use the AVG Personal Email Scanner instead of POPFile's SSL mode. This would make it easier for you to use an email client which can check more than one email account at a time.

            Brian

            • Message #682

              It's all been rendered moot: I can't disable GMail's spam filtering as I expected that I could. The filter trick was so obviously logical that it never occurred to me that it wouldn't work.

              It didn't. GMail is still filtering mail, and that renders PopFile? useless, since the whole point was to do more accurate local filtering and put an end to all the false positives caused by GMail.

              I'm back to doing without PopFile? again.

              • Message #683

                In that case, you might want to give POPFile's IMAP module a try. The IMAP module can check several folders for incoming messages. In your case that would be the Inbox and the folder where gmail sorts your "spam".

                Of course, that would also mean that you'd have to set your email client to use IMAP as well; an alternative to that would be the exclusive use of the web interface.

                • Message #684

                  I already have both IMAP and POP3 enabled in two of my GMail accounts, and both of those have corresponding accounts in Thunderbird. I did that so I could view their Spam folders without having to waste time with the Webmail interface. That's just a kludge, of course, because it doesn't stop the false positives in the first place, only lets me catch them better after the fact!

                  Given that, could I actually use PopFile? somehow in that scenario to shift the mail that GMail drops into that Spam folder back out of it? If I set up PopFile? to work with the IMAP account of my primary address, for instance, could I then use it to somehow classify the stuff that GMail puts in the Spam folder and then use a TB rule to forcibly move it back to the IMAP Inbox?

                  If I can do that, then that would still accomplish the goal all around, because if I can move it back to the IMAP Inbox, when the POP3 account checks that address for mail it will be able to retrieve it.

                  Is that feasible?

                  Damn Google for putting me through this headache!

                  • Message #685

                    It's even simpler than that.

                    You tell POPFile to have an eye on your gmail spam folder and it will take everything it thinks is not spam and put it somewhere else. No client rules needed.

                    • Message #686

                      Seriously?! Thank goodness I didn't uninstall PopFile? yet, then! Well, I installed the IMAP module already, so I'll see if I can disable the POP3 module and enable the IMAP one.

                      Can I effectively train a corpus that will work in that situation? I ask because it won't have access to the rest of my "good" mail for comparison, only what shows up in the Spam folder. Is that right, that PopFile?'s activity will be specific to one folder only? It might take a REALLY long time to train it properly.

                      • Message #689

                        As Manni stated in Message #683, you need to have POPFile watch two of Gmail's folders. It would watch the Inbox to identify what your good mail is, and Gmail's "Spam" folder to identify what it should call spam.

                        There is no need to disable the POP3 module, you can still use POPFile with other POP3 accounts you have. Change the account settings for the one Gmail account you want IMAP access to in Thunderbird, and set that account up in POPFile.

                        • Message #690

                          I didn't post an update here, but I'm past that particular point. PopFile? is doing a nominal job of filtering my IMAP account now. I've still got a LOT to figure out about all of it, both IMAP itself and how Thunderbird and PopFile? work with it. I have the POP3 module disabled for now; I may find that I don't need it.

                          I have some things happening that I didn't anticipate. For instance, at the moment GMail has stopped depositing spam into the Spam folder and has begun dropping it into Trash instead! I don't know why or how I could have caused that. I've been compensating by dragging them from Trash back to Spam, at which point they get classified (I think).

                          I'm not sure about the classification beyond what it tells me in the UI, because no Bucket nor any other header is being added to any messages, even though it's configured to add them.

                          I'm also having problems with deleted messages not staying deleted and reappearing, as I think some other people have. When they don't stay deleted, they get sent off down the POP3 pipeline whenever I trigger the POP3 account in TB to check for new mail. It seems I can't "delete" them using TB and can't drag them to the TB-created Trash folder; instead I have to drag messages to the [GMail]\Trash folder, which at least seem to hide them from TB's view.

                          • Message #691

                            It seems you have made some good progress since the last time I was able to check the messages here.

                            I guess the deleted messages can be avoided when you set POPFile to expunge deleted messages. You can find that option on the configuration tab.

                            The IMAP module is not able to modify email headers. We never found a good way to convey that on the Buckets tab, though. Sorry for the confusion.

                            I'm a bit confused about your last paragraph: Why are you still using POP3 in your client to access your account?

                            • Message #692

                              You're right, I wasn't clear that it didn't modify headers. I thought I might have read that it didn't, but I couldn't recall with certainty. I was a little afraid of enabling the "expunge" feature without knowing EXACTLY what its effect would be. I need to read more about it. I'm not entirely certain that the effect(s) I'm seeing are due to PopFile?... some of it could be Thunderbird behavior. I may have set up the IMAP account poorly: I had it configured to check ALL folders for new mail, which I came to realize was probably a Bad Thing.

                              As far as my stubborn use of POP3, the reason is simple: archiving. Should I trust Google? And what of my pre-Google mail? I save most of all my e-mail; I have mail history extending back to the early Nineties (though the earliest is now trapped in a corrupt PST file). I had hoped I could use IMAP and still be able to archive everything locally, but it doesn't seen to be possible in spite of what Thunderbird's settings imply. So... I use both.

                              95.04%, that's where I'm at.

                              • Message #693

                                Fair enough.

                                About expunging deleted messages: When you delete something from an IMAP mail box/folder, the message is simply marked as deleted. Physically, the only when you tell the server to expunge. Since most email clients also keep their own trash can, the feature is a bit redundant.

                                • Message #696

                                  Having the dual Trash folders, one subscribed from GMail and the other forced by Thunderbird, was just a small part of my confusion. I discovered the hard way that anything I dropped in Thunderbird's trash can or deleted from the context menu would magically reappear in the inbox. Is that caused by PopFile? or Thunderbird or GMail? I have no idea.

                                  The whole IMAP affair is quite hard to grasp. There are so many settings, so many interactions going on that I can't anticipate the effect of anything. Doing anything seems to step on something else.

                                  I just had three pieces of spam come through POP3 even though PopFile? correctly classified them. Two of them weren't even new mail but old; the newest one didn't even get moved to the designated IMAP folder. The other two were ones that had shown up in the IMAP Trash folder instead of Spam and I had dragged them there manually to classify them. I don't even know where the third one showed up in IMAP before it got "expunged" to POP3.

                                  The more I think about how little I know about what's happening and why, the more it freaks me out. I'm starting to wonder if those few occasional false positives caused by GMail are the more practical scenario; atleast I understood what was happening. It was my OCD about it that got me into this.

                                  • Message #697

                                    Okay, I'm done with this: for the third time I just caught PopFile? claiming that I had reclassified spam into another bucket, when I did no such thing. I'm having to undo reclassifications that I never even made.

                                    This really is too complex to be useful, and since GMail won't relinquish its stranglehold on spam so that I can use the predictable POP3 module, I think it's time to throw in the towel.

                                    This is what I get for relying on the permanence of a GMail address and shunning my ISP account.