Performance tunings in 6.5? But no process details found

Moderator: crythias

Post Reply
hkais
Znuny expert
Posts: 280
Joined: 16 Apr 2016, 08:55
Znuny Version: see in post
Real Name: Hans
Contact:

Performance tunings in 6.5? But no process details found

Post by hkais »

Hi Znuny developers,

is where any OSS/public Issue list about the performance tunings you have on agenda regarding 6.5 release?

I could not find any details about the aspect?

Also wondering about the community integration of new features.
From other OSS projects, where is a public Issue tracker, where users can follow and see what is on the roadmap.
I am missing this, or have I simply not found it?

Why I am asking, the forum has some complaints on performance, especially about the dashboard. Also it would be nice to know if and what is planned on the next roadmap specifically to decide if help can be incorporated.

Also wondering, is there a reason to keep issue tracking closed source?

best wishes
Last edited by hkais on 22 Aug 2022, 13:43, edited 1 time in total.
Elected 2022-06 as an IT Governance Portal Expert. The portal for Znuny, OTRS and OTOBO users
Johannes
Moderator
Posts: 391
Joined: 30 Jan 2008, 02:26
Znuny Version: All of them ^^
Real Name: Hannes
Company: Znuny|OTTERHUB

Re: Performance tunings in 6.5?

Post by Johannes »

Hi,
hkais wrote: 22 Aug 2022, 08:37 is where any OSS/public Issue list about the performance tunings you have on agenda regarding 6.5 release?

I could not find any details about the aspect?
first of all: performance is a very wide aspect. We implement new features with performance in mind, but don't focus on it for now.

Specific answer for your question:

We are working on a new SearchAPI which can use Indices like Elasticsearch, Opensearch and so on. We don't wanted to lock down on one index engine.
So there is a plugin mechanism in place, which allows users to easily switch, if needed.

The old search can still be used, but I one has performance complaints, it can be adopted easily. We will implement it in the Dashboard and Ticket Overviews by default, and in other areas over the time. So this would probably solve the issue you are talking about.
hkais wrote: 22 Aug 2022, 08:37 Also wondering about the community integration of new features.
From other OSS projects, where is a public Issue tracker, where users can follow and see what is on the roadmap.
I am missing this, or have I simply not found it?
Why I am asking, the forum has some complaints on performance, especially about the dashboard. Also it would be nice to know if and what is planned on the next roadmap specifically to decide if help can be incorporated.

Also wondering, is there a reason to keep issue tracking closed source?
There is a roadmap https://www.znuny.org/en/roadmap and there is a public issue https://github.com/znuny/Znuny/issues and a public feature tracker https://github.com/znuny/znuny-feature-requests. I think everyone is able to get a good impression where we are headed and whats on the table in the future.
Features can be requested and people can work on them, if they want to.
The contributing https://github.com/znuny/Znuny/blob/dev/CONTRIBUTING.md guidelines are available as well.
So if you want to participate, this is the way to go. If you don't want to, it's also fine. Just use it.

So yes, I think you missed a lot.

We don't put everything we have in mind out there, correct. Mostly because we don't want to deliver new features to other products on the market,
before we are even close to a possible release.
We are in with a lot of money on every feature we create. So is every open source project. Everyone needs to manage their finances on their own and stay 100% open source. No business, corporate special edition is a clear statement and it should stay that way.
We release it when it is done and yes, we take the advantage of the first release. And I think this is more than fair for everyone.
We don't believe in BigBang releases to change everything, we also think that "choice" is the best way.
So everything we develop and where it makes sense, is capable of plugins. We try to reduce the need for external addons and customisations / need for custom with every release. But using the plugin mechanism, in the text templates|autocomplete for example, you can create great things with a minimum of code.
The same goes for the new search api. Get the most out of it. Not just sling around code around an existing CPAN module and leave it that way.
And as for the GUI, there will be a beta.

You might find this confusing, but we are not a heavily funded project and don't have a venture capital backing.
We are maintaining an existing code base, renew it on the go and try to keep it as compatible as possible.
Mostly to reduce the efforts for the users and companies who provide services for it.
Which also means that we can't change things, as small as they may seem. Or that we won't make changed to existing
modules because we know we will touch them in the near future.

Just because a change might be perfect for one use-case, it can cause huge problems for others.

We also maintain an existing community and an existing installed base. We simply don't wan't & often can't move that fast.
If you have a problem with this approach, this project might not be the best fit for you.
Feel free to contribute, fork, or simply modify code and provide it to your customers.
We don't mind if you do and encourage everyone to do so.

Just a side note / personal opinion:
There may be other projects who are more open than we are. Sure there are. But there is a way lot more who aren't.
We try to balance it in the speed, the quality and the detail we can.

Znuny and before OTRS where never community managed, but community influenced. So the community has a place on the table and a voice.
And some of them are paying for things which get integrated later in the process, to support the project.
But we are no community managed OS project.

The funding is provided to 99% by the Znuny GmbH, which also decides about the roadmap and future releases.
But as you might have noticed over the past two years: we take the community very seriously.
We appreciate every opened issue, feature request and we try to fix|implement them as fast as possible.

The Znuny Team
(Johannes)

P.S:
You might want to think about the post title, because you ask so many questions who are not related to the "performance of 6.5".
hkais
Znuny expert
Posts: 280
Joined: 16 Apr 2016, 08:55
Znuny Version: see in post
Real Name: Hans
Contact:

Re: Performance tunings in 6.5? But no process details found

Post by hkais »

Hi Johannes,

thank you for your clearance.
Johannes wrote: 22 Aug 2022, 11:46
hkais wrote: 22 Aug 2022, 08:37 is where any OSS/public Issue list about the performance tunings you have on agenda regarding 6.5 release?

I could not find any details about the aspect?
first of all: performance is a very wide aspect. We implement new features with performance in mind, but don't focus on it for now.

Specific answer for your question:

We are working on a new SearchAPI which can use Indices like Elasticsearch, Opensearch and so on. We don't wanted to lock down on one index engine.
So there is a plugin mechanism in place, which allows users to easily switch, if needed.

The old search can still be used, but I one has performance complaints, it can be adopted easily. We will implement it in the Dashboard and Ticket Overviews by default, and in other areas over the time. So this would probably solve the issue you are talking about.
hkais wrote: 22 Aug 2022, 08:37 Also wondering about the community integration of new features.
From other OSS projects, where is a public Issue tracker, where users can follow and see what is on the roadmap.
I am missing this, or have I simply not found it?
Why I am asking, the forum has some complaints on performance, especially about the dashboard. Also it would be nice to know if and what is planned on the next roadmap specifically to decide if help can be incorporated.

Also wondering, is there a reason to keep issue tracking closed source?
There is a roadmap https://www.znuny.org/en/roadmap and there is a public issue https://github.com/znuny/Znuny/issues and a public feature tracker https://github.com/znuny/znuny-feature-requests. I think everyone is able to get a good impression where we are headed and whats on the table in the future.
Features can be requested and people can work on them, if they want to.
The contributing https://github.com/znuny/Znuny/blob/dev/CONTRIBUTING.md guidelines are available as well.
So if you want to participate, this is the way to go. If you don't want to, it's also fine. Just use it.

So yes, I think you missed a lot.
I did following checks before posting:

review https://www.znuny.org/en/roadmap

seeing:
- New search API to connect search engines like Elasticsearch, OpenSearch, Lucene, etc.
- Performance improvements

So searched:
https://github.com/znuny/Znuny/issues?q ... erformance
https://github.com/znuny/Znuny/issues?q ... sue+search

and I did not find any relevant infos to release notes.

Therefore my question about details and progress on this topic.

Johannes wrote: 22 Aug 2022, 11:46
We don't put everything we have in mind out there, correct. Mostly because we don't want to deliver new features to other products on the market,
before we are even close to a possible release.
We are in with a lot of money on every feature we create. So is every open source project. Everyone needs to manage their finances on their own and stay 100% open source. No business, corporate special edition is a clear statement and it should stay that way.
We release it when it is done and yes, we take the advantage of the first release. And I think this is more than fair for everyone.
We don't believe in BigBang releases to change everything, we also think that "choice" is the best way.
So everything we develop and where it makes sense, is capable of plugins. We try to reduce the need for external addons and customisations / need for custom with every release. But using the plugin mechanism, in the text templates|autocomplete for example, you can create great things with a minimum of code.
The same goes for the new search api. Get the most out of it. Not just sling around code around an existing CPAN module and leave it that way.
And as for the GUI, there will be a beta.
I am fine with approaches ensuring high quality. Afaik you proofed also that this approach is very good from 6.0 to LTS and also the intermediate deliveries are considerably stable.

For me the question is, are you working with Excel as issue tracker of do you use a issue tracker. If, which I assume later case, why is the issue tracker not public visible?
You can hide client issues easily by moving the into separated projects and you could let the community see also about the next planned steps.
So on OSS projects, there is typically more support gains if the visibility and the management of the projects is more visible.
So this is a good practice from many major and mature OSS projects.

So I am wondering also about knowing of the redesigns or the UI + redesigns on the search API that where is a lot of knowledge in the community you could win with the project, but for this you need to get more open.
But of course it is your project and your decision.
Johannes wrote: 22 Aug 2022, 11:46 You might find this confusing, but we are not a heavily funded project and don't have a venture capital backing.
We are maintaining an existing code base, renew it on the go and try to keep it as compatible as possible.
Mostly to reduce the efforts for the users and companies who provide services for it.
Which also means that we can't change things, as small as they may seem. Or that we won't make changed to existing
modules because we know we will touch them in the near future.

Just because a change might be perfect for one use-case, it can cause huge problems for others.

We also maintain an existing community and an existing installed base. We simply don't wan't & often can't move that fast.
If you have a problem with this approach, this project might not be the best fit for you.
Feel free to contribute, fork, or simply modify code and provide it to your customers.
We don't mind if you do and encourage everyone to do so.

Just a side note / personal opinion:
There may be other projects who are more open than we are. Sure there are. But there is a way lot more who aren't.
We try to balance it in the speed, the quality and the detail we can.
I do not see a speed impact on being open to the community with project progress. You may got some additional queries on the issues, but honestly, what kind of persons will read the issues? Developer background guys or endusers? And from whom will you probably get good feedback during development?
Johannes wrote: 22 Aug 2022, 11:46 Znuny and before OTRS where never community managed, but community influenced. So the community has a place on the table and a voice.
And some of them are paying for things which get integrated later in the process, to support the project.
But we are no community managed OS project.

The funding is provided to 99% by the Znuny GmbH, which also decides about the roadmap and future releases.
But as you might have noticed over the past two years: we take the community very seriously.
We appreciate every opened issue, feature request and we try to fix|implement them as fast as possible.
fully confirm, but if you consider also the community, you will see there are more persons, who could contribute. But for contribution, there needs to be openness in the planning, to know if it is worth investing in the contribution or not. Especially if you have a roadmap to be more specific on it and to be more clean about architectural or system design decisions.
If you discuss internally new features, I cannot assume that everything is discussed verbally and nothing is getting documented. So this intermediate decisions, could also help to incorporate more community helpers.

An economical decision from the community members are always considering if it is worth to invest. But how to consider, if you cannot see the road on a map to decide if the paths are going in the same wished direction?
Johannes wrote: 22 Aug 2022, 11:46 P.S:
You might want to think about the post title, because you ask so many questions who are not related to the "performance of 6.5".
thank you, will fix
Elected 2022-06 as an IT Governance Portal Expert. The portal for Znuny, OTRS and OTOBO users
Johannes
Moderator
Posts: 391
Joined: 30 Jan 2008, 02:26
Znuny Version: All of them ^^
Real Name: Hannes
Company: Znuny|OTTERHUB

Re: Performance tunings in 6.5? But no process details found

Post by Johannes »

Hi,

thanks for the feedback. I try to give you some more insights.
We use GitLab in sync with GitHub, this is no secret.
And we handle our projects/milestones there.

But but you are missing the point:
We don't want this to be public during the development phase, until a beta for this feature is released.

For the reasons I described before. We don't want it to be out there before it is ready.

Thats what the beta phase is for, test the features, find issues, fix them.
And everyone involved has to wait until the beta is ready. No upfront screenshots, no videos, no code.
So I am wondering also about knowing of the redesigns or the UI + redesigns on the search API that where is a lot of knowledge in the community you could win with the project, but for this you need to get more open.
But of course it is your project and your decision.
Just to be clear, it is not my project. We are a team of ten and we make decisions together.
It is GPL and everyone can take it as he wants.

back 2 topic:
We respect your opinion on that, but no.
One does not simply dive into such projects. And the history of my own OS experience tells me, they often don't want to.
Or they simply can't due to time and other constraints.
You need to put so much effort into this, that we can also directly hire the one person who is interested in the process.

For the most features that will be implemented there is a timeline one can not simply follow or join.
We often have multiple meetings per week, before the implementation even starts. The Search API for
example started somewhere around July last year, development started in April this year. Since May we have weekly
status/discussion meetups. (Info: 4 people are working (mostly) full time on this.)

We don't have the resources to explain, everyone who joins the party in the process, why we made a decision.
And they wont read the issues.
We can discuss it during beta or after the release, but we don't want this during development.
If we discover a major design flaw, this is on us. But we can deal with it.

Same goes for the GUI. Yes there are experienced people out there, but thats exactly the reason why we
hired external resources for this task. We question every decision since the project started.
And we use it and change stuff all the time. But we don't add any more features for our version 1.0 for example.
Which does not mean that we wont add features in later versions. (Info: 3 people are working (mostly) full time on this.)

To add community feedback during this stage means: much more resources, block expectations,
explain why we drop a feature, why we don't "just add this small thing", writing post like this and so on.
This takes more time than you might think.
Believe me, the design team is annoyed enough with our "small" changes along the way.
And there is a pretty good (component) documentation and even a style guide as a result.

Another point why we don't want the community to dive in this deep for certain parts is, availability and licensing.
It is like working with freelancers, which we also don't do. You can't really rely on them as you can with your own employees.
They go on vacation, change jobs, move to other countries and are simply busy.
Without the need to inform us. I'm an active part in four OS communities and I see that in all of them.
Then we also don't want to discuss the license on a central / core parts of the application.
This is annoying for everybody.

I do not see a speed impact on being open to the community with project progress. You may got some additional queries on the issues, but honestly, what kind of persons will read the issues? Developer background guys or endusers? And from whom will you probably get good feedback during development?
We have a roadmap / timeline for exact that reason. If I update the community let's say, every three weeks.
An expectation arises, but we may change things. So we have to deal not only with progress, but with expectation. People will be disappointed, they want explanations, and so on. All the time we play with this, we loose time.

Not only for the feature itself, but for community support, documentation, bug fixes and much more.
You think this will not slow down? You are wrong with this.
And for what benefit? To get more information to the 10 people of serveral thousands wo want the
information as early as it gets, to do whatever they want.
I don't think that this is nearly worth the effort. We are as open as we can be for now.
We discussed this several times, and will keep it that way for now.
It may change, but not in the near future.
fully confirm, but if you consider also the community, you will see there are more persons, who could contribute. But for contribution, there needs to be openness in the planning, to know if it is worth investing in the contribution or not. Especially if you have a roadmap to be more specific on it and to be more clean about architectural or system design decisions.
If you discuss internally new features, I cannot assume that everything is discussed verbally and nothing is getting documented. So this intermediate decisions, could also help to incorporate more community helpers.

An economical decision from the community members are always considering if it is worth to invest. But how to consider, if you cannot see the road on a map to decide if the paths are going in the same wished direction?
The roadmap is no technical documentation. It is an outlook for users who use the software, not develop it.
It is for the admin, the key user, the people who need to know / want to know what is coming over the time.

And sorry to say it that way, but I doubt that. "The Community" are 90%+ users, not developers.
And the small amount of developers often solve bugs and participate in reporting or fixing them is more than enough.
No one need to really invest a lot, except they want to.

And if they want to, they can follow the guidelines for contributing and simply contribute.
No need to ask.
You can also create your own extensions if you think something is missing or can be improved.
There are central hubs for addons (OPAR for example).
There is also no need to ask before you publish the code.
I never understood sentences like: "I can publish it, if someone is interested."
Why wait? One may not need it now, but in a year someone might stumble upon the thread and may need
exact the solution to this problem. And the original poster is long gone.
Just release it, why wait.

If one wants to contribute, there is enough to do in the current code which does not even relate to the new features:
- Propose code improvements (performance, security, quality)
- Update extend Documentation
- Improve the tests
- QA existing features
- Just create issues for bugs you find
- Send an email to the ical instance to generate more test samples (viewtopic.php?f=53&t=43212)

the list is long and none of the points is something one can't do without knowing how the Search API is build or "what will the new gui look like".
And if you are unsure what to do or want to add something, create an issue and it will sort out itself.
See the current contributions from maxcence, sector nord and many many others. We never ask, they just help out
when they have the ressources. Because they want to help the project. We don't pay for it.

https://github.com/znuny/Znuny/pulls?q= ... s%3Aclosed


If you don't want to go into invest because you are unsure what is comming.

Fine, then just wait and be patient until the betas hit the download servers and
get going.

Greetings
Johannes
Post Reply