Lost of Permissions

Moderator: crythias

Post Reply
Titan
Znuny newbie
Posts: 3
Joined: 10 Jan 2022, 12:52
Znuny Version: 5.0.20
Real Name: Edgar Gastán
Company: Versia

Lost of Permissions

Post by Titan »

Hello everybody

We have a problem that occurs frequently: some agents lose permissions on the groups assigned to them and they can't see any tickets. For the rest of agents, the application works correctly and this issue seems transparent, but sometimes there is a kind of chain reaction that ends up affecting performance in general.

The error can be detected by looking at the system log, with a message like: [Error][Kernel::System::Group::_DBGroupUserGet][Line:2486]: no hay suficiente memoria para el resultado de la consulta, SQL: SELECT user_id, group_id, permission_key FROM group_user WHERE permission_value = 1. [spanish traslation: not enough memory for the query result]. A spike in CPU load graph is also detected every time permissions are lost.

In general the system recovers after executing the following commands

Code: Select all

/www/scripts/apache24.sh stop
su -c "/opt/otrs/bin/otrs.Console.pl Maint::Config::Rebuild" -s /bin/bash otrs
su -c "/opt/otrs/bin/otrs.Console.pl Maint::Cache::Delete" -s /bin/bash otrs
su -c "/opt/otrs/bin/otrs.Console.pl Maint::Loader::CacheCleanup" -s /bin/bash otrs
su -c "/opt/otrs/bin/otrs.Console.pl Maint::Loader::CacheGenerate" -s /bin/bash otrs
/www/scripts/apache24.sh start
But this is not always the case, and sometimes we need to try several times. It is also an unwanted situation, because we have to stop the service for approximately 1 minute.

We have the opinion that they are two separate errors, maybe related, but not the same:

New Agents

New agents always create problems. As soon as they connect they produce this error and it is more persistent over time. Also, the table that lists agent's permissions on the different groups is not completely filled in on a first attempt, and requires multiple executions of the recovery commands mentioned above. Could it be a cache size related issue?

General Error


The frequency of the error (with no new agents in the system) seems to be related to the number of users utilizing the application: more agents and more articles, greater the probability.

We think this kind of error could be due to differente causes:
  • the logic/configuration that has been used in our product (this will be explained later)
  • the computational limit available to us
  • agent's searches without any filter, or care
  • saturation of the database, since some scripts are launched against it through the generic agent
  • we use REST services quite intensively, and they could be interfering
  • others
Configuration

We are using an OTRS 5.0.20 running on a SUSE Linux Enterprise Server 12 SP3. The database is a PostgreSQL 9.6.1.

We are also using LDAP authentication, although with certain particular characteristics:
  • We are employing a 1 to 1 relationship between groups and queues
  • There is over 75 groups/queues and 300 agents. Many agents could be in many groups
  • There is an LDAP group, let's call it ALL_AGENTS, which gives read-only, move_into, create and notes permissions to all groups/queues (these permissions are given to all agents)
  • There is also a LDAP group, let's call it ADMIN, which gives all permissions to all queues (these permissions are only given to a few agents)
  • All specific groups also have associated total permissions on that same group. With this it is achieved that an agent with permissions on ALL_AGENTS and on the groups to which it belongs, can perform minimal actions on all the groups, and total actions on the groups to which it belongs
  • For the management of this kind of logic we use $Self->{'AuthSyncModule::LDAP::UserSyncGroupsDefinition1'} = { ... }
    I have done several tests with $Self->{'AuthSyncModule::LDAP::UserSyncRolesDefinition'} = { } to create roles, because there is redundact logic in some LDAP groups, but it doesn't seem to work with UserSyncGroupsDefinition1 at the same time (maybe I'm wrong)
  • This logic is set in Default.pm. I know that the correct way would be in Config.pm, but as it has been said before, changes in permissions can produce errors, and in this case we have thought that it could influence everyone, leaving the service unavailable for an undefined period of time. Also, as far as I understand, it would only affect in case of migrating to a higher version. Again, I could be wrong.
I hope I have made a summary of the error in a way that can be understood. Any question you want to ask will be well received.

Greetings to everyone and thanks for your time
jojo
Znuny guru
Posts: 15019
Joined: 26 Jan 2007, 14:50
Znuny Version: Git Master
Contact:

Re: Lost of Permissions

Post by jojo »

how many memory and CPUs does the machine have?
"Production": OTRS™ 8, OTRS™ 7, STORM powered by OTRS
"Testing": ((OTRS Community Edition)) and git Master

Never change Defaults.pm! :: Blog
Professional Services:: http://www.otrs.com :: enjoy@otrs.com
Titan
Znuny newbie
Posts: 3
Joined: 10 Jan 2022, 12:52
Znuny Version: 5.0.20
Real Name: Edgar Gastán
Company: Versia

Re: Lost of Permissions

Post by Titan »

jojo wrote: 18 Jan 2022, 14:48 how many memory and CPUs does the machine have?
It's running under a VMware, with 16 cores and 16 GB assigned
Titan
Znuny newbie
Posts: 3
Joined: 10 Jan 2022, 12:52
Znuny Version: 5.0.20
Real Name: Edgar Gastán
Company: Versia

Re: Lost of Permissions

Post by Titan »

The CPU load spike looks like this
Image

and the error message like this:
Image

Any thoughts?
root
Administrator
Posts: 3934
Joined: 18 Dec 2007, 12:23
Znuny Version: Znuny and Znuny LTS
Real Name: Roy Kaldung
Company: Znuny
Contact:

Re: Lost of Permissions

Post by root »

Hi,

The error message is pretty clear. Without offending, I recommend consulting someone with database / linux server configuration skills. Something seems to use all the memory.

- Roy
Znuny and Znuny LTS running on CentOS / RHEL / Debian / SLES / MySQL / PostgreSQL / Oracle / OpenLDAP / Active Directory / SSO

Use a test system - always.

Do you need professional services? Check out https://www.znuny.com/

Do you want to contribute or want to know where it goes ?
Post Reply