Operating system: Debian Bullseye Kernel: Linux dsl9x 5.4.170+ #1 SMP Sat Mar 5 10:08:44 PST 2022 x86_64 GNU/Linux Installation method / source: GitHub, build myself
Question
Everytime I start OpenVAS I see lines like the following in /var/log/gvm/openvas.log: libgvm util:WARNING:2022-05-17 10h07.41 utc:161: NVT /var/lib/openvas/plugins/2012/amazon/alas-2012-108.nasl with duplicate OID 1.3.6.1.4.1.25623.1.0.120261 will be replaced with 2012/alas-2012-108.nasl
This makes me think like a specific path is set wrong or the cache (Redis?) is corrupt.
I’m not a hero in the C language, but I looked at the following file: https://github.com/greenbone/gvm-libs/blob/main/util/nvticache.c lines 225-256. I am trying to understand what is happening.
While I read other topics and tried to read the source code, I can’t figure out what is going wrong and how I can resolve this issue.
I’m wondering if someone can help me a bit further with this issue.
I would try to remove all the files in the /var/lib/openvas/plugins/ folder and sync the nvt files again and run then gvmd -rebuild ( 2x - ) my phone change it to a long dash.
Just to get that right, the NVT cache is part of openvas scanner not gvmd. The scanner is loading the VT information into redis and gvmd is requesting the VT changes since the last update from the scanner. Thus if you shut down redis for example due to a reboot all VT data must be reloaded from the file system.
I tried with an empty /var/lib/openvas/plugins/ but I’ll try again to be sure.
I am copying the community feed to reduce the load on Greenbone’s servers. I’ll try with a direct sync from Greenbone’s server incase the copied feed didn’t sync correctly.
Yeah this was the issue, my feed sync didn’t delete old files. That fixed the duplicate issue.
Happy to hear that it now fix this issue.
However, while the NVT cache is up-to-date almost immediately, the startup time didn’t improve. Some further investigation necessary from my side :).
What exactly is your problem with the startup time? If you mean that the complete stack is ready after a start? No, this will not work out of the box, once you sync the nvts they need to get parsed and written to the postgresql. This takes it’s time, based on you system it can take less time like 10 minutes (my cloud server) or like on my LAN it can take on a RPI 6-12h. There are some factors on HDD or SSD or NVME as storage and also of RAM and CPU.
Maybe you look to your iowait after start the services and you will probably see that it is high, I personal have on my RPI a iowait between 10 and 20.
Right now I see a startup time of 6 minutes and I’m trying to bring it down as much as possible.
What I am trying to create is a Docker image which builds every night and is (almost) ready to go. I also tried to add persistence to Redis so it doesn’t has to load its cache everytime.
In the build process of the Docker image I’m doing the NVT synchronization and wrote a script to check if the feeds are fully synced and to check if all configs are available. Only then the build process ends and it should start ready to go. That’s why I don’t understand why it has to take 6 minutes to startup when:
Feed is synced
Redis has the cache data
PostgreSQL is synced
I understand this is more a hobby project and not really the scope of the community project ;).