strange errors devurandom.pm & Random/Source/Factory.pm

[hkais]

getting issues after cross switch from centos6 / OTRS 5.0.42 to ubuntu 20.04 / OTRS 5.0.42

Various internal server errors occur, on different points.
But most have in common to do
- close of ticket
- send mail
- merge

On analyzing log i am having this errors regular, sometime a few, sometimes tons like this

Code: Select all

[Wed Feb  9 13:11:42 2022] devurandom.pm: Use of uninitialized value in subroutine entry at (eval 141) line 3.
[Wed Feb  9 13:11:42 2022] devurandom.pm: Use of uninitialized value in subroutine entry at (eval 141) line 3.
[Wed Feb  9 13:11:42 2022] devurandom.pm: Use of uninitialized value in subroutine entry at (eval 141) line 3.
[Wed Feb  9 13:11:42 2022] devurandom.pm: Use of uninitialized value in subroutine entry at (eval 141) line 3.
[Wed Feb  9 13:11:42 2022] devurandom.pm: Use of uninitialized value in subroutine entry at (eval 141) line 3.

with getting the prio messages I am getting this in a short period of time afterwards:

Code: Select all

[Wed Feb 09 13:11:42.837351 2022] [:error] [pid 121750:tid 140223543494400] Undef did not pass type constraint "ClassName" (in $self->{"weak_source"}) at /opt/otrs/Kernel/cpan-lib/Crypt/Random/Source/Factory.pm line 77\n    "ClassName" is a subtype of "Str"\n    "Str" is a subtype of "Value"\n    "Value" is a subtype of "Defined"\n    Undef did not pass type constraint "Defined" (in $self->{"weak_source"})\n    "Defined" is defined as: (defined($_))\n

after getting the message above it is only a matter of time to get this:

Code: Select all

[Wed Feb  9 13:13:05 2022] devurandom.pm: Use of uninitialized value in subroutine entry at (eval 141) line 3.
[Wed Feb 09 13:13:06.513876 2022] [core:notice] [pid 121745:tid 140223963995200] AH00051: child pid 121750 exit signal Segmentation fault (11), possible coredump in /etc/apache2

any ideas how to track down this issue?
I am pretty lost here. The next step was to get upgraded to znuny 6.0 and afterwards to 6.1
But not knowing if there is a different issue with this => stopped the migration path for now

[root]

Hi,

Which logs have you checked? The web server's error log has usually a stack trace. This helps to identify issues.

The reason for a segmentation fault might be harder to find, but wha'ts the output of `a2dismod`? Just send the output please.

And, of course, what is the output of '/opt/otrs/bin/otrs.CheckModule.pl --all'? And check if the output executed as root and as otrs user is the same.

- Roy

[hkais]

Roy, thank you for your quick anwser!

I have error_log and access_log from apache.
Stacktrace is sadly not available.

Code: Select all

a2dismod 

Your choices are: 
access_compat 
alias 
auth_basic 
authn_core 
authn_file 
authz_core 
authz_host 
authz_user 
autoindex 
dir 
env 
expires 
filter 
headers 
mime 
mpm_event 
negotiation 
perl 
reqtimeout 
setenvif 
status

sudo -u otrs /opt/otrs/bin/otrs.CheckModules.pl --all and the output as root is the same

Code: Select all

  o Apache::DBI......................ok (v1.12)
  o Apache2::Reload..................ok (v0.13)
  o Archive::Tar.....................ok (v1.34)
  o Archive::Zip.....................ok (v1.67)
  o Crypt::Eksblowfish::Bcrypt.......ok (v0.009)
  o Crypt::SSLeay....................ok (v0.73_06)
  o Date::Format.....................ok (v2.24)
  o DBI..............................ok (v1.643)
  o DBD::mysql.......................ok (v4.050)
  o DBD::ODBC........................ok (v1.61)
  o DBD::Oracle......................Not installed! (optional - Required to connect to a Oracle database.)
  o DBD::Pg..........................ok (v3.10.4)
  o Digest::SHA......................ok (v6.02)
  o Encode::HanExtra.................ok (v0.23)
  o IO::Socket::SSL..................ok (v2.067)
  o JSON::XS.........................ok (v4.02)
  o List::Util::XS...................ok (v1.50)
  o LWP::UserAgent...................ok (v6.13)
  o Mail::IMAPClient.................ok (v3.42)
    o IO::Socket::SSL................ok (v2.067)
    o Authen::SASL...................ok (v2.16)
    o Authen::NTLM...................ok (v1.09)
  o ModPerl::Util....................ok (v2.000011)
  o Net::DNS.........................ok (v1.22)
  o Net::LDAP........................ok (v0.66)
  o Template.........................ok (v2.27)
  o Template::Stash::XS..............ok (undef)
  o Text::CSV_XS.....................ok (v1.41)
  o Time::HiRes......................ok (v1.9760)
  o Time::Piece......................ok (v1.33)
  o XML::LibXML......................ok (v2.0134)
  o XML::LibXSLT.....................ok (v1.99)
  o XML::Parser......................ok (v2.46)
  o YAML::XS.........................ok (v0.81)

Bundled modules:

  o Algorithm::Diff..................ok (v1.1903)
  o Apache::DBI......................ok (v1.12)
  o CGI..............................ok (v4.21)
  o CSS::Minifier....................ok (v0.01)
  o Class::Inspector.................ok (v1.28)
  o Crypt::PasswdMD5.................ok (v1.40)
  o Crypt::Random::Source............ok (v0.14)
  o Date::Pcalc......................ok (v1.2)
  o Email::Valid.....................ok (v1.196)
  o Encode::Locale...................ok (v1.05)
  o Exporter::Tiny...................ok (v1.002001)
  o IO::Interactive..................ok (v0.0.6)
  o JSON.............................ok (v2.90)
  o JSON::PP.........................ok (v2.27203)
  o JavaScript::Minifier.............ok (v1.14)
  o LWP..............................ok (v6.13)
  o Linux::Distribution..............ok (v0.23)
  o Locale::Codes....................ok (v3.36)
  o MIME::Tools......................ok (v5.507)
  o Mail::Address....................ok (v2.14)
  o Mail::Internet...................ok (v2.14)
  o Math::Random::ISAAC..............ok (v1.004)
  o Math::Random::Secure.............ok (v0.080001)
  o Module::Find.....................ok (v0.15)
  o Module::Refresh..................ok (v0.17)
  o Moo..............................ok (v2.003006)
  o Mozilla::CA......................ok (v20180117)
  o Net::HTTP........................ok (v6.17)
  o Net::IMAP::Simple................ok (v1.2206)
  o Net::SSLGlue.....................ok (v1.055)
  o PDF::API2........................ok (v2.025)
  o SOAP::Lite.......................ok (v1.19)
  o Sys::Hostname::Long..............ok (v1.5)
  o Text::CSV........................ok (v1.33)
  o Text::Diff.......................ok (v1.44)
  o Types::TypeTiny..................ok (v1.010000)
  o URI..............................ok (v1.69)
  o YAML.............................ok (v1.15)
  o namespace::clean.................ok (v0.27)
  o parent...........................ok (v0.234)

Do you have a hint to increase log level on otrs and/or perl and/or apache?

[root]

Hi,

First of all, you're using the wrong mpm_module. This is shown in the support data collector. Fix this with

Code: Select all

a2dismod mpm_event
a2enmod mpm_prefork
systemctl restart apache2

and the segmentation issues should be history.

Increase the web server loglevel with LogLevel debug. Put this into a custom like /etc/apache2/conf-available/loglevel.conf and enable it with a2enconf loglevel && systemctl reload apache2

- Roy

[hkais]

Hi Roy,

thx for your help. I switched the module but getting this odd response from a2enmod

Code: Select all

a2dismod mpm_event

Module mpm_event disabled.


a2enmod mpm_prefork

Considering conflict mpm_event for mpm_prefork:
Considering conflict mpm_worker for mpm_prefork:
Enabling module mpm_prefork.

Is it only a confusing message to inform to prefer mpm_worker or mpm_event over mpm_prefork?
Could not find any reasonable info on googling...

Hopefully will get today a feedback from users about if environment is now more stable.

Will switch to debug in apache, but afaik under ubuntu debug is disabled. Will change and update

[root]

Hi,

Is it only a confusing message to inform to prefer mpm_worker or mpm_event over mpm_prefork?

It's just telling you that only one mpm model is possible in a running environment.

Could not find any reasonable info on googling...

My recommendation instead: reading the Apache httpd manual, section "Multi-Processing Modules (MPMs)". Second last sentence of the introduction.

- Roy

[hkais]

the switch to mpm_prefork seems to have fixed my biggest issues.
My day was a painless day after enabling prefork.

Will get back if I face any other issues anymore.

Roy, big thx for your support!