Openvasmd authentication error after starting big task

[tatooin]

Now, assuming it’s a bug in GVM9 (which I believe it is), is there anything I can do to help developers find the root cause of this and a cure ?

I’m not familiar with how Greenbone is organized and if there is any guidelines to follow to submit a bug report for instance ?

Thanks

[cfi]

I checked that post but unfortunately there doesn’t seem to be a fix for this available yet.

Please keep in mind that there can’t be a solution for every problem, especially not within such a short time-frame the linked post was created.

Also, this problem wasn’t happening with GVM8 (same machine, same config) where I was managing big tasks without any problems.

New major releases might introduce additional capabilities / features bringing additional load and complexity where e.g. the SQLite database is hitting its limits.

Now, assuming it’s a bug in GVM9 (which I believe it is), is there anything I can do to help developers find the root cause of this and a cure ?

You could subscribe to the referenced issue tracker linked in the thread above to see if there are any questions of the development team: https://github.com/greenbone/gvmd/issues/347

I’m not familiar with how Greenbone is organized and if there is any guidelines to follow to submit a bug report for instance ?

Generally its recommended to create a new issue at one of the github issue trackers of the problematic component if you think you have found a bug within the code and not only a configuration / environmental issue.

Greenbone Networks GmbH

[tatooin]

Just an update on this; actually you can connect back to GSA after some times. You don’t need to restart / flush anything. I’ve scanned around 50 targets from different sizes and I confirm the problem happens randomely, but almost on every targets.

I also notices that the dashboards are often broken, and everytime this happens I see the sqlite3 error in my logs.

Has anyone else seen this problem ?

Thanks !

[Lukas]

With lot´s of results, you will get much better results by using PostgreSQL SQLITE does have many locking issues, so this might be worth looking into a real database.

[tatooin]

Thanks Lukas. Well, fact is my database is already quite large (almost 3GB large) so switching to a real database as you said couldn’t harm. I will try your suggestion. Is there any guide or howto to available to do this transition ?

Thanks !

[Mogimbo]

Hi tatooin, I think the problem is similar to mine (Openvas with PostgresSQL Could not authenticate to manager daemon)
I have quite 700 Tasks and 20 daily scans (during the night).
In the beginning i used SQLite3 and had the connection problems with quite 300 tasks. The problem was SQLite timeout and I solved modifying the openvas source code.
Because this is not a good idea, I tried to use PostgresSQL but today I have the same problems.
I think the problem is still about timeout connection to the DB.
I create a my own (simple) interface in PHP end connect directly to openvas PostgresSQL and it works, so the problem shouldn’t be the DB.

Hope this helps.
Giovanni

[tatooin]

Hi Giovanni,

Thanks for your answer. What version of openvas manager are you using ? Have you tried with the latest version available on git or like me with a rather old version provided by packages maintainers (in my case, ubuntu).

Thanks a lot

[Mogimbo]

Hi tatooin,
I compiled from source openvas9-manager-7.0.3 on ubuntu 18.08

here a summary about my personal guide

sudo apt install -y postgresql
sudo apt install -y postgresql-contrib postgresql-server-dev-10
cd /usr/local/src
sudo mkdir -p openvas/debs
cd openvas
sudo add-apt-repository -s ppa:mrazavi/openvas
sudo apt update
sudo apt install -y dpkg-dev debhelper cmake pkg-config libglib2.0-dev libgnutls28-dev libgcrypt11-dev libsqlite3-dev libgpgme11-dev rats libopenvas9-dev smbclient

sudo su -
apt source openvas9-manager
cd openvas9-manager-7.0.3
dpkg-checkbuilddeps
[etc etc]

[tatooin]

Would be interesting to see if latest version available on git is fixing this. I’ve installed the latest version from git on ubuntu as well and now trying to have it work. I’ll see if I get the same results there.

I think it’s a bug so hopefully it has been fixed already.

[Mogimbo]

I have a complete guide written by myself (crawling on internet )
Maybe you have to change only the source url
If u need just ask

Giovanni

[tatooin]

Hi Giovanni,

Yes please. This would certainly help me get this done

[Mogimbo]

I create a post:

Guide to install Openvas-Manager with Postgres

This a guide I have written during my installation of openvas. All commands are taken from internet, and maybe something could be useless, but for it works Please try it and give me feedback to improve and fix the guide Hope it helps sudo apt update; sudo apt -y dist-upgrade; sudo apt -y autoremove sudo apt install -y postgresql sudo systemctl status postgresql sudo -u postgres createuser root sudo -u postgres createdb -O root tasks sudo -u postgres psql tasks create role dba…

[tatooin]

Thanks Giovanni. However it seems you’re recompiling the sources used by the package maintainer, which are probably not the latest version available.

I’m going to try by git cloning the official repository and see if I can replicate the issues.

Thanks !

[Mogimbo]

Indeed
Let me know if it works

Giovanni

[tatooin]

Hi Giovanni,

Just to keep you updated;
I have recompiled the source of the latest stable version available (GVM-9) and I confirm the problem is still there. It seems it’s a bug introduced in GVM-9, as this wasn’t happening prior upgrading to this version.

There seems to be a fix in the master branch (https://github.com/greenbone/gvmd/pull/338)

[tatooin]

But unfortunately, it doesn’t seem to have been backported.

[cfi]

As written in the topic Sqlite error / manager daemon authentication error where the posts of SQLite error and tasks stuck where split off the linked PR is unrelated to your seen issue here. This PR is only fixing an issue where tasks where not started at all when using schedules.

[Mogimbo]

Referring to SQLite:

I think the problem is that the SQLite answer to OpenVas the code: SQLITE_BUSY and after 10 retries Openvas close the connection and return the error.
In sql_sqlite3.c I have noticed these possible problems:

1. The timeout is hardcoded

#define BUSY_TIMEOUT 1000

Openvas has a config file, maybe it could be a good choice to put this configuration in this file

2. Line 441 sql_sqlite3.c in the function sql_exec_internal

int sql_exec_internal (int retry, sql_stmt_t *stmt)
[…]
while (1)
{
int ret;
ret = sqlite3_step (stmt->stmt);
if (ret == SQLITE_BUSY)
{
if (retry)
{
if ((retries > 10) && (OPENVAS_SQLITE_SLEEP_MAX > 0))
openvas_usleep (MIN ((retries - 10) * 10000,
OPENVAS_SQLITE_SLEEP_MAX));
retries++;
continue;
}
if (retries++ < 10)
continue;
return -2;
}
if (retry == 0)
sqlite3_busy_timeout (task_db, BUSY_TIMEOUT);
if (ret == SQLITE_DONE)
return 0;
if (ret == SQLITE_ROW)
return 1;
g_warning ("%s: sqlite3_step failed: %s\n",
FUNCTION,
sqlite3_errmsg (task_db));
return -1;
}

Tha variable “OPENVAS_SQLITE_SLEEP_MAX” is never set or, if it set, the default is 0.
checking in my code, I have this code in CMakeLists.txt (line 167 and 168):

if (NOT DEFINED OPENVAS_SQLITE_SLEEP_MAX)
set (OPENVAS_SQLITE_SLEEP_MAX 0)

This means that Openvas doesn’t retry to connect to DB, but only abort.
I have checked in the Github code, but in CMakeLists.txt I haven’t seen the OPENVAS_SQLITE_SLEEP_MAX 0

If you modify the code in sql_sqlite3.c line 435:

if ((retries > 10) && (OPENVAS_SQLITE_SLEEP_MAX > 0))

in

if ( ((retries > 10) && (OPENVAS_SQLITE_SLEEP_MAX > 0))
OR (ret == SQLITE_BUSY) )

The workaround should work (but it’s not the correct way to solve it)

[Mogimbo]

I’ll try to check if the workaround work also for postgres…

[tatooin]

:Thanks cfi for clarifying. Too bad