CoreCacheCleanup Fail

HelpdeskMan · Post by **HelpdeskMan** » 30 Aug 2024, 10:11

Hello All!

I have recently migrated an OTRS 4 install on Ubuntu 16.04 to Znuny 6.5 Lts on Ubuntu 22.04.

The move went fairly smoothly, and the ticketing system is running as expected. However, I noticed when looking at the Maint::Daemon::Summary the CoreCacheCleanup was failing.

Code: Select all

Gathering summary for all daemons...

  System configuration sync:
    Daemon is active.

  Tasks to be executed in future:
    There are currently no tasks to be executed in future.

  Recurrent cron tasks:
  +---------------------------------+---------------------+-------------+----------------+---------------------+
  | Name                            | Last Execution      | Last Status | Last Duration  | Next Execution      |
  +---------------------------------+---------------------+-------------+----------------+---------------------+
  | ArticleSearchIndexRebuild       | 2024-08-30 08:23:00 | Success     | < 1 Second     | 2024-08-30 08:25:00 |
  | CalendarTicketCreate            | 2024-08-30 08:22:00 | Success     | 1.0 Second(s)  | 2024-08-30 08:26:00 |
  | CalendarTicketCreateCleanup     | 2024-08-30 03:00:00 | Success     | 1.0 Second(s)  | 2024-08-31 03:00:00 |
  | CommunicationLogDelete          | 2024-08-30 04:00:00 | Success     | 1.0 Second(s)  | 2024-08-31 04:00:00 |
  | ConfigurationDeploymentCleanup  | 2024-08-25 01:40:00 | Success     | 2.0 Second(s)  | 2024-09-01 01:40:00 |
  | CoreCacheCleanup                | 2024-08-25 01:20:00 | Fail        | 34.0 Second(s) | 2024-09-01 01:20:00 |
  | EscalationCheck                 | 2024-08-30 08:20:00 | Success     | 1.0 Second(s)  | 2024-08-30 08:25:00 |
  | GenerateDashboardStats          | 2024-08-30 08:05:00 | Success     | 3.0 Second(s)  | 2024-08-30 09:05:00 |
  | GenericInterfaceDebugLogCleanup | 2024-08-30 04:02:00 | Success     | < 1 Second     | 2024-08-31 04:02:00 |
  | ITSMChangesCheck                | 2024-08-30 08:20:00 | Success     | 1.0 Second(s)  | 2024-08-30 08:30:00 |
  | LoaderCacheDelete               | 2024-08-25 01:30:00 | Success     | < 1 Second     | 2024-09-01 01:30:00 |
  | MailAccountFetch                | 2024-08-30 08:20:00 | Success     | 2.0 Second(s)  | 2024-08-30 08:30:00 |
  | MailQueueSend                   | 2024-08-30 08:23:00 | Success     | < 1 Second     | 2024-08-30 08:25:00 |
  | RebuildEscalationIndex          | 2024-08-30 08:20:00 | Success     | < 1 Second     | 2024-08-30 08:28:00 |
  | RenewCustomerSMIMECertificates  | 2024-08-30 03:02:00 | Success     | < 1 Second     | 2024-08-31 03:02:00 |
  | SessionDeleteExpired            | 2024-08-30 07:55:00 | Success     | < 1 Second     | 2024-08-30 09:55:00 |
  | SpoolMailsReprocess             | 2024-08-30 01:10:00 | Success     | < 1 Second     | 2024-08-31 01:10:00 |
  | SupportDataCollectAsynchronous  | 2024-08-30 08:01:00 | Success     | < 1 Second     | 2024-08-30 09:01:00 |
  | TicketAcceleratorRebuild        | 2024-08-30 02:01:00 | Success     | < 1 Second     | 2024-08-31 02:01:00 |
  | TicketDraftDeleteExpired        | 2024-08-30 07:55:00 | Success     | < 1 Second     | 2024-08-30 08:55:00 |
  | TicketNumberCounterCleanup      | 2024-08-30 08:20:00 | Success     | < 1 Second     | 2024-08-30 08:30:00 |
  | TicketPendingCheck              | 2024-08-30 07:45:00 | Success     | < 1 Second     | 2024-08-30 09:45:00 |
  | TicketUnlockTimeout             | 2024-08-30 07:35:00 | Success     | < 1 Second     | 2024-08-30 08:35:00 |
  | WebUploadCacheCleanup           | 2024-08-30 07:46:00 | Success     | < 1 Second     | 2024-08-30 08:46:00 |
  | ReindexSMIMECertificates        | 2024-08-30 03:02:00 | Success     | < 1 Second     | 2024-08-31 03:02:00 |
  +---------------------------------+---------------------+-------------+----------------+---------------------+

  Recurrent generic agent tasks:
  +---------------------------------------------------+---------------------+-------------+---------------+---------------------+
  | Name                                              | Last Execution      | Last Status | Last Duration | Next Execution      |
  +---------------------------------------------------+---------------------+-------------+---------------+---------------------+
  | Archive tickets                                   | 2024-08-30 02:00:00 | Success     | 5.2 Minute(s) | 2024-08-31 02:00:00 |
  | Close stock tickets                               | 2024-08-30 01:00:00 | Success     | < 1 Second    | 2024-08-31 01:00:00 |
  | Delete tickets from the Junk queue after 2 months | 2024-08-30 01:00:00 | Success     | < 1 Second    | 2024-08-31 01:00:00 |
  +---------------------------------------------------+---------------------+-------------+---------------+---------------------+

  Unhandled Worker Tasks:
    There are currently no tasks waiting to be executed.

  Handled Worker Tasks:
    There are currently no tasks being executed.

I have tried to implement solutions found on this forum, unfortunately the error persists.

I was seeing an error before, unsure if related though "Error: Active indexing process already running! Skipping..." which I fixed by running:

Code: Select all

su -c "/opt/otrs/bin/otrs.Console.pl Maint::Ticket::FulltextIndexRebuildWorker --children 1 --limit 1 --force-pid" -s /bin/bash otrs

Now get "no indexing needed" - Done.

Code: Select all

su -c "/opt/otrs/bin/otrs.Console.pl Maint::Config::Rebuild --cleanup" -s /bin/bash otrs

Done with no errors

Code: Select all

su -c "/opt/otrs/bin/otrs.Console.pl Admin::Config::ListInvalid" -s /bin/bash otrs

All settings valid

Code: Select all

su -c "/opt/otrs/bin/otrs.Console.pl Maint::Loader::CacheCleanup" -s /bin/bash otrs

Done with no errors

Code: Select all

/opt/otrs/bin/otrs.SetPermissions.pl

All permissions updated

In the System Log (index.pl?Action=AdminLog) and apache error log there is the following error which repeats fairly regularly.

Code: Select all

ERROR: OTRS-CGI-10 Perl: 5.34.0 OS: linux Time: Fri Aug 30 07:35:40 2024

 Message: Need module!

 RemoteAddress: 10.0.30.55
 RequestURI: /otrs/index.pl?Action=AgentDashboard

 Traceback (678182):
   Module: Kernel::System::Main::Require Line: 77
   Module: Kernel::Modules::AgentDashboardCommon::_Element Line: 910
   Module: Kernel::Modules::AgentDashboardCommon::Run Line: 629
   Module: Kernel::System::Web::InterfaceAgent::Run Line: 1144
   Module: ModPerl::ROOT::ModPerl::Registry::opt_otrs_bin_cgi_2dbin_index_2epl::handler Line: 39
   Module: (eval) (v1.99) Line: 207
   Module: ModPerl::RegistryCooker::run (v1.99) Line: 207
   Module: ModPerl::RegistryCooker::default_handler (v1.99) Line: 173
   Module: ModPerl::Registry::handler (v1.99) Line: 32

While writing this post I looked into that a little more with help from viewtopic.php?t=42499 and installed Znuny-Search and Znuny-SearchFrontend. Which highlighted I didn't have Proc::Find or Search::Elasticsearch installed for Perl (cpanm Proc::Find, cpanm Search::Elasticsearch). Unfortunately, this didn't resolve the issue!

Any direction would be greatly appreciated.

HelpdeskMan · Post by **HelpdeskMan** » 30 Aug 2024, 16:44

CheckModules output - still working on the issue with the "Need Module" error (hoping its in some way linked to the CoreCacheCleanup)

Code: Select all

    o Apache::DBI......................ok (v1.12)
  o Apache2::Reload..................ok (v0.13)
  o Archive::Tar.....................ok (v2.38)
  o Archive::Zip.....................ok (v1.68)
  o Authen::SASL.....................ok (v2.16)
  o CPAN::Audit......................ok (v0.15)
  o Crypt::Eksblowfish::Bcrypt.......ok (v0.009)
  o Crypt::JWT.......................ok (v0.035)
  o Crypt::OpenSSL::X509.............ok (v2.0.1)
  o CSS::Minifier::XS................ok (v0.13)
  o Data::UUID.......................ok (v1.0602)
  o Date::Format.....................ok (v2.24)
  o DateTime.........................ok (v1.55)
    o DateTime::TimeZone.............ok (v2.51)
  o DBI..............................ok (v1.643)
  o DBD::mysql.......................ok (v4.050)
  o DBD::ODBC........................Not installed! To install, you can use: 'apt-get install -y libdbd-odbc-perl'. (optional - Required to connect to a MS-SQL database.)
  o DBD::Oracle......................Not installed! (optional - Required to connect to a Oracle database.)
  o DBD::Pg..........................Not installed! To install, you can use: 'apt-get install -y libdbd-pg-perl'. (optional - Required to connect to a PostgreSQL database.)
  o Digest::SHA......................ok (v6.02)
  o Encode::HanExtra.................ok (v0.23)
  o Excel::Writer::XLSX..............ok (v0.95)
  o Hash::Merge......................ok (v0.302)
  o iCal::Parser.....................ok (v1.21)
  o IO::Socket::SSL..................ok (v2.074)
  o JavaScript::Minifier::XS.........ok (v0.15)
  o Jq...............................ok (v0.01)
  o JSON::XS.........................ok (v4.03)
  o List::Util::XS...................ok (v1.55)
  o LWP::UserAgent...................ok (v6.53)
  o Mail::IMAPClient.................ok (v3.43)
    o IO::Socket::SSL................ok (v2.074)
    o Authen::NTLM...................ok (v1.09)
  o ModPerl::Util....................ok (v2.000012)
  o Moo..............................ok (v2.005004)
  o Net::DNS.........................ok (v1.33)
  o Net::LDAP........................ok (v0.68)
  o Net::LDAP::Constant..............ok (v0.24)
  o Net::SMTP........................ok (v3.13)
  o Spreadsheet::XLSX................ok (v0.17)
  o Template.........................ok (v2.27)
  o Template::Stash::XS..............ok (undef)
  o Text::Diff::FormattedHTML........ok (v0.08)
  o Text::CSV_XS.....................ok (v1.47)
  o Time::HiRes......................ok (v1.9767)
  o Time::Piece......................ok (v1.3401)
  o XML::LibXML......................ok (v2.0134)
  o XML::LibXSLT.....................ok (v1.99)
  o XML::Parser......................ok (v2.46)
  o YAML::XS.........................ok (v0.83)

Bundled modules:

  o Algorithm::Diff..................ok (v1.1903)
  o Apache::DBI......................ok (v1.12)
  o CGI..............................ok (v4.36)
  o CSS::Minifier....................ok (v0.01)
  o Class::Inspector.................ok (v1.31)
  o Crypt::PasswdMD5.................ok (v1.40)
  o Crypt::Random::Source............ok (v0.14)
  o Email::Valid.....................ok (v1.202)
  o Encode::Locale...................ok (v1.05)
  o Exporter::Tiny...................ok (v1.002001)
  o IO::Interactive..................ok (v1.022)
  o JSON.............................ok (v2.94)
  o JSON::PP.........................ok (v2.27203)
  o JavaScript::Minifier.............ok (v1.15)
  o LWP..............................ok (v6.53)
  o Linux::Distribution..............ok (v0.23)
  o Locale::Codes....................ok (v3.52)
  o MIME::Tools......................ok (v5.509)
  o Mail::Address....................ok (v2.18)
  o Mail::Internet...................ok (v2.18)
  o Math::Random::ISAAC..............ok (v1.004)
  o Math::Random::Secure.............ok (v0.080001)
  o Module::Find.....................ok (v0.15)
  o Module::Refresh..................ok (v0.17)
  o Moo..............................ok (v2.005004)
  o Mozilla::CA......................ok (v20211001)
  o Net::HTTP........................ok (v6.17)
  o Net::IMAP::Simple................ok (v1.2209)
  o Net::SSLGlue.....................ok (v1.058)
  o PDF::API2........................ok (v2.033)
  o SOAP::Lite.......................ok (v1.20)
  o Sisimai..........................ok (vv4.25.16)
  o Sys::Hostname::Long..............ok (v1.5)
  o Text::CSV........................ok (v1.95)
  o Text::Diff.......................ok (v1.44)
  o Types::TypeTiny..................ok (v1.010000)
  o URI..............................ok (v1.71)
  o YAML.............................ok (v1.23)
  o namespace::clean.................ok (v0.27)
  o parent...........................ok (v0.236)

Post by **root** » 01 Sep 2024, 14:05

Hi,

There are two things I would start with:

1. Check the dameon logs if you see an error related to this task.
2. Run the cache delete manually: bin/otrs.Console.pl Maint::Cache::Delete

- Roy

HelpdeskMan · Post by **HelpdeskMan** » 02 Sep 2024, 10:09

Hi root

I had run that command on Friday just before I finished, checking this morning the error has now cleared!

Command to clear Cache

Code: Select all

su -c "bin/otrs.Console.pl Maint::Cache::Delete" -s /bin/bash otrs >/dev/null

Command to check Daemons

Code: Select all

su -c "/opt/otrs/bin/otrs.Console.pl Maint::Daemon::Summary" -s /bin/bash otrs

Result:

Code: Select all

Gathering summary for all daemons...

  System configuration sync:
    Daemon is active.

  Tasks to be executed in future:
    There are currently no tasks to be executed in future.

  Recurrent cron tasks:
  +---------------------------------+---------------------+-------------+---------------+---------------------+
  | Name                            | Last Execution      | Last Status | Last Duration | Next Execution      |
  +---------------------------------+---------------------+-------------+---------------+---------------------+
  | ArticleSearchIndexRebuild       | 2024-09-02 08:53:00 | Success     | < 1 Second    | 2024-09-02 08:54:00 |
  | CalendarTicketCreate            | 2024-09-02 08:52:00 | Success     | < 1 Second    | 2024-09-02 08:54:00 |
  | CalendarTicketCreateCleanup     | 2024-09-02 03:00:00 | Success     | < 1 Second    | 2024-09-03 03:00:00 |
  | CommunicationLogDelete          | 2024-09-02 04:00:00 | Success     | < 1 Second    | 2024-09-03 04:00:00 |
  | ConfigurationDeploymentCleanup  | 2024-09-01 01:40:00 | Success     | 3.0 Second(s) | 2024-09-08 01:40:00 |
  | CoreCacheCleanup                | 2024-09-01 01:20:00 | Success     | 8.2 Minute(s) | 2024-09-08 01:20:00 |
  | EscalationCheck                 | 2024-09-02 08:50:00 | Success     | 1.0 Second(s) | 2024-09-02 08:55:00 |
  | GenerateDashboardStats          | 2024-09-02 08:05:00 | Success     | 2.0 Second(s) | 2024-09-02 09:05:00 |
  | GenericInterfaceDebugLogCleanup | 2024-09-02 04:02:00 | Success     | < 1 Second    | 2024-09-03 04:02:00 |
  | ITSMChangesCheck                | 2024-09-02 08:50:00 | Success     | 1.0 Second(s) | 2024-09-02 09:00:00 |
  | LoaderCacheDelete               | 2024-09-01 01:30:00 | Success     | 1.0 Second(s) | 2024-09-08 01:30:00 |
  | MailAccountFetch                | 2024-09-02 08:50:00 | Success     | 1.0 Second(s) | 2024-09-02 09:00:00 |
  | MailQueueSend                   | 2024-09-02 08:53:00 | Success     | < 1 Second    | 2024-09-02 08:54:00 |
  | RebuildEscalationIndex          | 2024-09-02 08:52:00 | Success     | 1.0 Second(s) | 2024-09-02 08:56:00 |
  | RenewCustomerSMIMECertificates  | 2024-09-02 03:02:00 | Success     | < 1 Second    | 2024-09-03 03:02:00 |
  | SessionDeleteExpired            | 2024-09-02 07:55:00 | Success     | < 1 Second    | 2024-09-02 09:55:00 |
  | SpoolMailsReprocess             | 2024-09-02 01:10:00 | Success     | 1.0 Second(s) | 2024-09-03 01:10:00 |
  | SupportDataCollectAsynchronous  | 2024-09-02 08:01:00 | Success     | < 1 Second    | 2024-09-02 09:01:00 |
  | TicketAcceleratorRebuild        | 2024-09-02 02:01:00 | Success     | < 1 Second    | 2024-09-03 02:01:00 |
  | TicketDraftDeleteExpired        | 2024-09-02 07:55:00 | Success     | < 1 Second    | 2024-09-02 08:55:00 |
  | TicketNumberCounterCleanup      | 2024-09-02 08:50:00 | Success     | < 1 Second    | 2024-09-02 09:00:00 |
  | TicketPendingCheck              | 2024-09-02 07:45:00 | Success     | 1.0 Second(s) | 2024-09-02 09:45:00 |
  | TicketUnlockTimeout             | 2024-09-02 08:35:00 | Success     | < 1 Second    | 2024-09-02 09:35:00 |
  | WebUploadCacheCleanup           | 2024-09-02 08:46:00 | Success     | < 1 Second    | 2024-09-02 09:46:00 |
  | ReindexSMIMECertificates        | 2024-09-02 03:02:00 | Success     | < 1 Second    | 2024-09-03 03:02:00 |
  +---------------------------------+---------------------+-------------+---------------+---------------------+

  Recurrent generic agent tasks:
  +---------------------------------------------------+---------------------+-------------+---------------+---------------------+
  | Name                                              | Last Execution      | Last Status | Last Duration | Next Execution      |
  +---------------------------------------------------+---------------------+-------------+---------------+---------------------+
  | Archive tickets                                   | 2024-09-02 02:00:00 | Success     | 3.0 Second(s) | 2024-09-03 02:00:00 |
  | Close stock tickets                               | 2024-09-02 01:00:00 | Success     | < 1 Second    | 2024-09-03 01:00:00 |
  | Delete tickets from the Junk queue after 2 months | 2024-09-02 01:00:00 | Success     | < 1 Second    | 2024-09-03 01:00:00 |
  +---------------------------------------------------+---------------------+-------------+---------------+---------------------+

  Unhandled Worker Tasks:
    There are currently no tasks waiting to be executed.

  Handled Worker Tasks:
    There are currently no tasks being executed.

I did install some more modules, just as jq, and made some changes to Apache and MariaDB (performance tuning settings). No idea if these helped or not.

You mention checking the Daemon log, unsure where you mean for this as you don't mention specifics.

On the server I couldn't find anything specific and with a little help from GPT it pointed me towards Dameon::Log which was set to only keep logs for 1 day. Changed this to 7 and to redirect STDOUT as well (STDERR was already enabled).

I think the issue is now sorted. For the other error I am getting "Need Module" I'll open another post.

Thanks

Znuny Open Source Ticketsystem

CoreCacheCleanup Fail

CoreCacheCleanup Fail

Re: CoreCacheCleanup Fail

Re: CoreCacheCleanup Fail

Re: CoreCacheCleanup Fail