OTRS 5: »No valid 'utf-8' string« at MailAccountFetch

[mejo]

We've a rather annoying issue after upgrading to OTRS 5. Probably the issue is completely unrelated to the upgrade though.

The MailAccountFetch cronjob sends an error message »No valid 'utf-8' string« every time it runs. The weird thing is that the error message seems to be about the same message/string each time, and the error is prefixed by another instance of itself each time. As a result, the message grow to 25k of »No valid 'utf-8' strings in the meantime, see below.

It's obvious that the string in question is malformed and has wrong encoding.

What is unclear to me is why MailAccountFetch keeps processing the message/string again and again each time it runs. I'm unable to find the message in the inbox, at least the POP3 server displays an empty Inbox to me.

Does OTRS have an internal incoming queue where the message got stock? I didn't find any, neither in the database nor on the filesystem.

What else can I do to delete this message? Currently we get spammed by mails with the errors every time MailAccountFetch is run, which is every five minutes in our case.

Here's the (shortened) error message:

No valid 'utf-8' string: 'No valid 'utf-8' string: 'No valid 'utf-8' string: 'No valid
'utf-8' string: 'No valid 'utf-8' string: 'No valid 'utf-8' string: 'No valid 'utf-8'
[...]
string: 'No valid 'utf-8' string: 'No valid 'utf-8' string: 'No valid 'utf-8' string: 'No
valid 'utf-8' string: 'No valid 'utf-8' string: 'No valid 'utf-8' string: 'Anruf von
rB�ro - *****'!
'!
[...]
'!
ERROR: OTRS-otrs.Console.pl-Maint::PostMaster::MailAccountFetch-30 Perl: 5.10.1 OS: linux
Time: Wed Nov 11 15:40:14 2015

Message: Need Search, UserLogin, PostMasterSearch, CustomerIDRaw or CustomerID!

Traceback (11548):
Module: Kernel::System::CustomerUser::DB::CustomerSearch Line: 189
Module: Kernel::System::CustomerUser::CustomerSearch Line: 176
Module: Kernel::System::PostMaster::NewTicket::Run Line: 155
Module: Kernel::System::PostMaster::Run Line: 372
Module: Kernel::System::MailAccount::POP3::Fetch Line: 215
Module: Kernel::System::MailAccount::MailAccountFetch Line: 440
Module: Kernel::System::Console::Command::Maint::PostMaster::MailAccountFetch::Run
Line: 94
Module: (eval) Line: 441
Module: Kernel::System::Console::BaseCommand::Execute Line: 435
Module: (eval) Line: 175
Module: Kernel::System::Daemon::DaemonModules::SchedulerTaskWorker::Cron::Run Line: 159
Module: Kernel::System::Daemon::DaemonModules::SchedulerTaskWorker::Run Line: 259
Module: (eval) Line: 316
Module: main::Start Line: 316
Module: bin/otrs.Daemon.pl Line: 138

'!
[...]
'!
No valid 'utf-8' string: 'Anruf von B�ro - *****'!
'!
[...]
'!
No valid 'utf-8' string: 'Anruf von B�ro - *****'!
'!
[...]
'!
No valid 'utf-8' string: 'No valid 'utf-8' string: 'No valid 'utf-8' string: 'No valid
[...]
'utf-8' string: 'No valid 'utf-8' string: 'No valid 'utf-8' string: 'No valid 'utf-8'
string: 'No valid 'utf-8' string: 'No valid 'utf-8' string: 'Anruf von B�ro -
*****'!
'!
[...]
'!
ERROR: OTRS-otrs.Console.pl-Maint::PostMaster::MailAccountFetch-30 Perl: 5.10.1 OS: linux
Time: Wed Nov 11 15:40:14 2015

Message: Need Search, UserLogin, PostMasterSearch, CustomerIDRaw or CustomerID!

Traceback (11548):
Module: Kernel::System::CustomerUser::DB::CustomerSearch Line: 189
Module: Kernel::System::CustomerUser::CustomerSearch Line: 176
Module: Kernel::System::PostMaster::NewTicket::Run Line: 155
Module: Kernel::System::PostMaster::Run Line: 372
Module: Kernel::System::MailAccount::POP3::Fetch Line: 215
Module: Kernel::System::MailAccount::MailAccountFetch Line: 440
Module: Kernel::System::Console::Command::Maint::PostMaster::MailAccountFetch::Run
Line: 94
Module: (eval) Line: 441
Module: Kernel::System::Console::BaseCommand::Execute Line: 435
Module: (eval) Line: 175
Module: Kernel::System::Daemon::DaemonModules::SchedulerTaskWorker::Cron::Run Line: 159
Module: Kernel::System::Daemon::DaemonModules::SchedulerTaskWorker::Run Line: 259
Module: (eval) Line: 316
Module: main::Start Line: 316
Module: bin/otrs.Daemon.pl Line: 138

'!
[...]
'!
No valid 'utf-8' string: 'Anruf von B�ro - *****'!
'!
[...]
'!
No valid 'utf-8' string: 'Anruf von B�ro - *****'!
'!
[...]
'!

Thanks in advance for your help

[mejo]

Ok, after thinking again, probably the following happens:

initially we got a mail with a malformed string. MailAccountFetch fetched that mail and threw an error. That error resulted in a new mail to our inbox, which in turn got fetched by the OTRS MailAccountFetch cronjob. But the error mail contained the original malformed string, so processing the error mail produced a new error message. And now this happens again and again.

I removed all occurences of the error message mails in the inbox now and hope that the problem is gone

[mejo]

Indeed it was as suspected: The OTRS error messages regarding "no valid 'utf-8' string" created a loop of error messages that were sent to the inbox and fetched by OTRS, which in turn produced a new error message.

Unfortunately, this again happened as a rather old and buggy telephone system keeps sending mails with wrongly encoded strings to the OTRS inbox.

I consider this as a bug in OTRS. It's good to warn about malformed strings, but it's clearly a bug if these messages in turn produce new warnings and result in a recursive loop.

Do you have any suggestion about how to stop this vicious circle?

Probably the correct solution would be that OTRS ignores charset errors in warning/error messages that were produced by itself.