Open Discussion → email base64 encoded not decoded

email base64 encoded not decoded

Dear all,
I noticed this forum has been subject of spam posts recently and POPFile has an hard life to classify the notification emails I receive; they are base64 encoded and their content is not decoded, so the only clue POPFile has is the subject. Is there a plan to handle also this kind of email encode?

Paolo

Hereafter follows the email content as it appears in POPFile:

This is a multi-part message in MIME format.

[email protected]
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64

ZGF2Y2VmYWkgaGFzIGp1c3QgcG9zdGVkIGEgbmV3IG1lc3NhZ2Ugb24gdGhlICJIZWxwIiBmb3J1
bSBvbiBnZXRwb3BmaWxlLm9yZzoKCgpJIGhhdmUganVzdCB0cmllZCB0byBpbnN0YWxsIERCRDo6
U1FMaXRlMi4gVGhpcyBmYWlsZWQuIEF0IHRoZSBlbmQgSSBnb3QgdGhlIGZvbGxvd2luZzoNCg0K
VGVzdCBTdW1tYXJ5IFJlcG9ydA0KLS0tLS0tLS0tLS0tLS0tLS0tLQ0KdC9hay1kYmQudCAgICAg
ICAgICAgIChXc3RhdDogMTEgVGVzdHM6IDExIEZhaWxlZDogMCkNCiAgTm9uLXplcm8gd2FpdCBz
dGF0dXM6IDExDQogIFBhcnNlIGVycm9yczogQmFkIHBsYW4uICBZb3UgcGxhbm5lZCA0OSB0ZXN0
cyBidXQgcmFuIDExLg0KRmlsZXM9MjQsIFRlc3RzPTMxMCwgIDEgd2FsbGNsb2NrIHNlY3MgKCAw
LjEyIHVzciAgMC4wMyBzeXMgKyAgMC43MSBjdXNyICAwLjE2IGNzeXMgPSAgMS4wMiBDUFUpDQpS
ZXN1bHQ6IEZBSUwNCkZhaWxlZCAxLzI0IHRlc3QgcHJvZ3JhbXMuIDAvMzEwIHN1YnRlc3RzIGZh
aWxlZC4NCm1ha2U6ICoqKiBbdGVzdF9keW5hbWljXSBFcnJvciAyNTUNCiAgTVNFUkdFQU5UL0RC
RC1TUUxpdGUyLTAuMzMudGFyLmd6DQogIC91c3IvYmluL21ha2UgdGVzdCAtLSBOT1QgT0sNCi8v
aGludC8vIHRvIHNlZSB0aGUgY3Bhbi10ZXN0ZXJzIHJlc3VsdHMgZm9yIGluc3RhbGxpbmcgdGhp
cyBtb2R1bGUsIHRyeToNCiAgcmVwb3J0cyBNU0VSR0VBTlQvREJELVNRTGl0ZTItMC4zMy50YXIu
Z3oNClJ1bm5pbmcgbWFrZSBpbnN0YWxsDQogIG1ha2UgdGVzdCBoYWQgcmV0dXJuZWQgYmFkIHN0
YXR1cywgd29uJ3QgaW5zdGFsbCB3aXRob3V0IGZvcmNlDQpGYWlsZWQgZHVyaW5nIHRoaXMgY29t
bWFuZDoNCiBNU0VSR0VBTlQvREJELVNRTGl0ZTItMC4zMy50YXIuZ3ogICAgICAgICAgICA6IG1h
a2VfdGVzdCBOTw0KDQpJIGtub3cgdGhhdCB0aGlzIGlzIHBvc3NpYmx5IG5vdCB0aGUgZm9ydW0g
Zm9yIFBlcmwgcHJvYmxlbXMgYnV0IGFueSBoZWxwIHdpbGwgYmUgYXBwcmVjaWF0ZWQuDQoNClRo
YW5rcwoKLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t
LS0tLS0tLS0tLS0tCk1lc3NhZ2UgVVJMOiBodHRwOi8vZ2V0cG9wZmlsZS5vcmcvZGlzY3Vzc2lv
bi8xLzM1My8xNDUxIzE0NTEKUE9QRmlsZSAtIEF1dG9tYXRpYyBFbWFpbCBDbGFzc2lmaWNhdGlv
biBodHRwOi8vZ2V0cG9wZmlsZS5vcmcvCg==

[email protected]
Content-Type: text/plain; charset=us-ascii; name="footer.txt"
Content-Disposition: inline
Content-Transfer-Encoding: 8bit

  • Message #1453

    Hi,

    POPFile supports Base64 encoding.
    The cause is irregular MIME boundary ("[email protected]").

    In RFC 2046, the boundary is defined as:

         boundary := 0*69<bchars> bcharsnospace
    
         bchars := bcharsnospace / " "
    
         bcharsnospace := DIGIT / ALPHA / "'" / "(" / ")" /
                          "+" / "_" / "," / "-" / "." /
                          "/" / ":" / "=" / "?"
    

    According to RFC 2046, "@" should not be used as boundary character.

    Manni, can we change the Trac configuration not to use "@" as boundary character? Or should we allow to use "@" in POPFile?

    Naoki

    • Message #1456

      I just found a configuration setting that sets the default encoding for each outgoing mail. I changed it to "none" from "base64". Please consider this a test message.

      Manni

      • Message #1457

        Looks pretty good to me. No?

        Manni

        • Message #1458

          Of course, we also need to test UTF-8 characters: ÄÖÜßäöü.

          POPFile は自動メール振り分けツールです。一旦、設定とトレーニングがきちんとされれば、あなたのパソコンのバックグラウンドで動き続け、到着したメールを読み取って、好きなように振り分けてくれます。ゴミメールを選り分けるといった簡単な仕事から、十数個のフォルダにメールを振り分けるといった複雑な仕事までこなすことができます。いわば、あなたの受信メールボックスを管理してくれるあなた専用のアシスタントです。

          • Message #1460

            Here's one more test message. Sorry.

            • Message #1461

              Seems like the culprit wasn't trac after all, but our mailing list software minimalist. No wonder I couldn't find the relevant sources.

              • Message #1462

                And here is my final test message for today:

                Of course, we also need to test UTF-8 characters: ÄÖÜßäöü.

                POPFile は自動メール振り分けツールです。一旦、設定とトレーニングがきちんとされれば、あなたのパソコンのバックグラウンドで動き続け、到着したメールを読み取って、好きなように振り分けてくれます。ゴミメールを選り分けるといった簡単な仕事から、十数個のフォルダにメールを振り分けるといった複雑な仕事までこなすことができます。いわば、あなたの受信メールボックスを管理してくれるあなた専用のアシスタントです。

    • Message #1459

      Alright. We are now using quoted printable, the notification mails look ok, and the forum doesn't crash when you enter 8-bit characters.

      Manni

      • Message #1465

        Dear,
        sorry for my late reply, I had not access to my computer these days. Now the email content is correctly decoded. I understand this is the result of a change in the notification mail; however I think POPFile should be updated to decode the content even with "@" used as a boundary character, to avoid this trick can be used by spammers to fool this wonderful Classifying Program.
        One more point: I noticed the UTF-8 characters are not treated correctly, am I missing some POPFile configuration stuff? This is how your last test email email is seen by POPFile:

        --START--
        manni has just posted a new message on the "Open Discussion" forum on getpopfile.org:

        Of course, we also need to test UTF-8 characters: ÄÖÜßäöü.

        POPFile ã¯è‡ªå‹•ãƒ¡ãƒ¼ãƒ«æŒ¯ã‚Šåˆ†ã‘ãƒ„ãƒ¼ãƒ«ã§ã™ã€‚ä¸€æ—¦ã€è¨ å®šã¨ãƒˆãƒ¬ãƒ¼ãƒ‹ãƒ³ã‚°ãŒãã¡ã‚“ã¨ã•ã‚Œã‚Œã°ã€ã‚ãª
        ãŸã®ãƒ‘ã‚½ã‚³ãƒ³ã®ãƒãƒƒã‚¯ã‚°ãƒ©ã‚¦ãƒ³ãƒ‰ã§å‹•ãç¶šã‘ã€åˆ°ç€ã—ãŸãƒ¡ãƒ¼ãƒ«ã‚’èª ã¿å–ã£ã¦ã€å¥½ããªã‚ˆã†
        に振り分けてくれます。ゴミメールを選り分けるといった簡単な仕事から、十数個のフォ
        ルダにメールを振り分けるといった複雑な仕事までこなすことができます。いわば、あな
        たの受信メールボックスを管理してくれるあなた専用のアシスタントです。

        ---END---

        Ciao

        Paolo

        • Message #1467

          Hi,

          One more point: I noticed the UTF-8 characters are not treated correctly, am I missing some POPFile configuration stuff? This is how your last test email email is seen by POPFile:

          Currently POPFile supports only one character encoding for one language except for Japanese. POPFile does not convert character encoding and it treats characters in their original encoding.
          I hope future version of POPFile (maybe v2.0?) will support multiple encoding (include UTF-8).

          Naoki

      • Message #1466

        Looks good for me. Thanks!

        Naoki