Huge xml_storage table

[Evanscheijen]

Hi all,

Let me start with wishing all a great bugfree 2013.....

We have some automatic imports of configuration items into our OTRS CMDB, these are stored afaik in the XML_Storage table and it seems the longer the imports are done the larger the table gets.
I am under the assumption that somehow the items are all appended to the table while only the new or updated items should be affected.
This way any action involving xmlstorage is getting slower and slower to the point that we have to wait over a minute for the reportlist e.g.
I am not sure how the import works atm but afaik it is kicked off from crontab involving a python script.
What would be the most effective way to do an import like this using a csv file?
Is there a way to check wether the imported CI's are indeed present in the table (without opening an over 10 Gb file) and if my assumption is correct is there a way to clear the table so we can shrink it back to an acceptable size?.

Regards,

Erik

[jojo]

there is an import script in bin after installation of the Import/Export module. This allows import via CSV file and only updating changed items

[Evanscheijen]

Cool, I'll look into that asap, any ideas on how to shrink the current table?

[jojo]

you might use the script to delte the CIs. And then reimport the actual version from CVS

[Evanscheijen]

I think I know what is going on, the python script does an export from an external source daily of all items not older then 1 month, resulting in a CSV that is imported daily.
The way I see it that means we get each item imported an average of 30 times.

[jojo]

if there is no change for the CI no new version will be created