Long time between fetching emails and opening the tickets

[rsaffi]

Hi all,

My company currently runs OTRS 3.1.11 on Linux and we've been facing a rather annoying issue over the last two months.
System is configured to fetch e-mails every two minutes from 11 different accounts and open tickets in different queues, accordingly.
Problem is that for some specific queues it is taking too long between downloading the messages via IMAPS and opening the ticket.

I've done tons of digging on this: I've checked network, database, application, changed some parameters hoping they could help me debug this, but no luck. Operating system where the application is installed is normal. Database is normal, system loads are normal (front-end and database servers).

I logged in the webmail for those accounts and noticed that the test messages I sent arrive there instantly and as soon as OTRS fetches them (I know when by looking at the log file) they do "disappear" from the inbox. But then the otrs.PostmasterMailbox.pl finishes running and for some queues (two in particular) the tickets don't show up on the system and the sender doesn't received the e-mail confirming the opening of the ticket.
I tried running a "show full processlist" on mysql to see if there were any procedures running, but there's nothing there. I tried running a few SELECTs to look for my e-mails on the `ticket` table, but they don't show up until the ticket effectively shows up on the system. Some tickets take hours (!!) to open, even though they were downloaded via IMAPS instantaneously. See the time difference between time the email was sent and when it showed up on the system:

mysql> select id,tn,title,create_time from ticket where queue_id = 51 and customer_id = 'username@domain.com' and ticket_state_id = 1 order by create_time;

Code: Select all

+--------+------------------+----------------------------+---------------------+
| id     | tn               | title                      | create_time         |
+--------+------------------+----------------------------+---------------------+
| 275555 | 2015121410002386 | TEST 1 - "20151214_140727" | 2015-12-14 16:22:22 |
| 275556 | 2015121410002395 | TEST 2 - "20151214_140727" | 2015-12-14 16:34:18 |
| 275557 | 2015121410002402 | TEST 3 - "20151214_140827" | 2015-12-14 16:36:19 |
| 275558 | 2015121410002411 | TEST 4 - "20151214_140927" | 2015-12-14 16:44:23 |
| 275559 | 2015121410002421 | TEST 5 - "20151214_141027" | 2015-12-14 16:46:20 |
| 275561 | 2015121410002448 | TEST 6 - "20151214_141127" | 2015-12-14 17:06:21 |
| 275562 | 2015121410002457 | TEST 7 - "20151214_141227" | 2015-12-14 17:14:25 |
+--------+------------------+----------------------------+---------------------+
7 rows in set (0.01 sec)

As you can see on the title, the emails were sent with a timestamp (they were sent from a script I made to send them one minute after the other). I sent 10 test emails. Three of them didn't even open yet. They are more than 3 hours late already!

Does anyone have any idea what's going on? Some tables are really really big, like so:

-rw-rw---- 1 mysql mysql 27G Dec 14 17:36 article_attachment.MYD
-rw-rw---- 1 mysql mysql 34G Dec 14 17:36 article_plain.MYD

What am I missing? Where else do I need to look? What exactly can I do to pinpoint the culprit for this huge delay?
Any insight is highly appreciated!

Regards,

Ricardo

[jojo]

IMAP/POP3 is not the best for mass import of mail. I suggest to use the MTA on the linux server instead and forward the mails.

Neverless, 3.1. is out of support including bugfixes and security updates for several years. Please update to the latest version of OTRS and try again as there are several fixes and enhancements also in the mailhandling

[rsaffi]

IMAP/POP3 is not the best for mass import of mail. I suggest to use the MTA on the linux server instead and forward the mails.

Neverless, 3.1. is out of support including bugfixes and security updates for several years. Please update to the latest version of OTRS and try again as there are several fixes and enhancements also in the mailhandling

Thanks for the input, jojo. I have already pointed out the importance of upgrading the system and it'll be done right after the hollidays.
Meanwhile, I wonder if you could help me throw some light into the matter and suggest something else I could check.

What bugs me the most is that the email is downloaded from the account's inbox but it still takes a huge ammount of time for it to show up on the system. Does OTRS use any temporary table to store the messages that have been downloaded but haven't yet appeared as tickets on the system?

Regards,

Ricardo