Service unavailable when verifying ospd-openvas scanner over tcp

I run two servers. The master server has everything installed. On the slave gvmd and gsad is not installed. The master communicates with the local ospd-openvas instance over socket and with the slave instance over TCP. The slave can not be verified over TCP “Service unavailable”. Starting a scan fails with “Could not send start_scan command to scanner”

GVM versions

gsad: Greenbone Security Assistant 20.08.0~git-18568a46d-HEAD
gvmd: Greenbone Vulnerability Manager 20.08.0~git-ac4498c2-HEAD
openvas-scanner: OpenVAS 20.8.0
gvm-libs: gvm-libs 20.8.0~git-a03eec19-HEAD

Environment

Operating system: Debian 10
Kernel: (‘uname -a’) 5.4.0-3-amd64 #1 SMP Debian 5.4.13-1 (2020-01-19) x86_64 GNU/Linux
Installation method / source: Git

The scanner is added using
gvmd --create-scanner="<hostname>" --scanner-host=<IP of Agent> --scanner-port=9390 --scanner-ca-pub=/opt/gvm/var/lib/gvm/CA/cacert.pem --scanner-key-pub=/opt/gvm/var/lib/gvm/CA/clientcert.pem --scanner-key-priv=/opt/gvm/var/lib/gvm/private/CA/clientkey.pem --scanner-type=OpenVAS

OSPD-Openvas is started as a service:

[Service]
Environment=PATH=/opt/gvm/bin/ospd-scanner/bin:/opt/gvm/bin:/opt/gvm/sbin:/opt/gvm/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
Environment=LD_LIBRARY_PATH=/opt/gvm/lib
Type=simple
User=gvm
Group=gvm
WorkingDirectory=/opt/gvm
PIDFile=/opt/gvm/var/run/ospd-openvas.pid
ExecStart=/opt/gvm/bin/ospd-scanner/bin/python /opt/gvm/bin/ospd-scanner/bin/ospd-openvas --pid-file /opt/gvm/var/run/ospd-openvas.pid -p 9390 -b 0.0.0.0 -k /opt/gvm/var/lib/gvm/private/CA/serverkey.pem -c /opt/gvm/var/lib/gvm/CA/servercert.pem --ca-file=/opt/gvm/var/lib/gvm/CA/cacert.pem --log-file /opt/gvm/var/log/gvm/ospd-scanner.log --config /opt/gvm/etc/openvas/ospd.conf --log-level debug
Restart=on-failure
RestartSec=2min
KillMode=process
KillSignal=SIGINT
GuessMainPID=no
PrivateTmp=true

[Install]
WantedBy=multi-user.target

Furthermore the installation of ospd-openvas on the master (connect via socket) can successfully be verified.

Issue

When trying to verify the slave scanner in GSAD I get “Scanner unavialable”.
ospd-openvas log contains:

OSPD[28175] 2020-09-28 10:16:20,851: DEBUG: (ospd.server) New connection from ('<IP of Master>', 34706)
OSPD[28175] 2020-09-28 10:16:20,870: DEBUG: (ospd.ospd) Handling get_version command request.

All seems good here.

gvmd log contains:

md   main:  DEBUG:2020-09-28 12h54.37 UTC:98909: <= client  "<verify_scanner scanner_id="afe384a6-8eb9-4ff2-baf0-c45793b430dc"/>"
md    gmp:  DEBUG:2020-09-28 12h54.37 UTC:98909:    XML  start: verify_scanner (1)
md    gmp:  DEBUG:2020-09-28 12h54.37 UTC:98909:    client state set: 519
md    gmp:  DEBUG:2020-09-28 12h54.37 UTC:98909:    XML    end: verify_scanner
lib  serv:  DEBUG:2020-09-28 12h54.37 UTC:98909:    Connected to server '<IP of Slave>' port 9390.
lib  serv:  DEBUG:2020-09-28 12h54.37 UTC:98909:    Shook hands with server '<IP of Slave>' port 9390.
lib  serv:  DEBUG:2020-09-28 12h54.37 UTC:98909:    send 14 from <get_version/>[...]
lib  serv:  DEBUG:2020-09-28 12h54.37 UTC:98909: => <get_version/>
lib  serv:  DEBUG:2020-09-28 12h54.37 UTC:98909: => done
lib   xml:  DEBUG:2020-09-28 12h54.37 UTC:98909:    asking for 1048576
md   main:  DEBUG:2020-09-28 12h54.37 UTC:98909: -> client: <verify_scanner_response status="503" status_text="Service unavailable"/>
md    gmp:  DEBUG:2020-09-28 12h54.37 UTC:98909:    client state set: 1
md   main:  DEBUG:2020-09-28 12h54.37 UTC:98909: => client  73 bytes
md   main:  DEBUG:2020-09-28 12h54.37 UTC:98909: => client  done
md   main:  DEBUG:2020-09-28 12h54.37 UTC:98909:    EOF reading from client
md   main:  DEBUG:2020-09-28 12h54.37 UTC:98909:    Cleaning up
md   main:  DEBUG:2020-09-28 12h54.37 UTC:98909:    Exiting

The error “Service unavailable” is very difficult to debug. What is unavailable? The handshake seems to work. I also verified the handshake using openssl. The connection gets succesfully established, so I assume the cert architecture generated using gvm-manage-certs -a is correct:

openssl s_client -connect 10.67.115.218:9390 -cert /opt/gvm/var/lib/gvm/CA/clientcert.pem -key /opt/gvm/var/lib/gvm/private/CA/clientkey.pem -CAfile /opt/gvm/var/lib/gvm/CA/cacert.pem -reconnect -showcerts -debug

connects and sending <get_version/> manually with openssl gets answered with

<get_version_response status="200" status_text="OK"><protocol><name>OSP</name><version>1.2</version></protocol><daemon><name>OSPd OpenVAS</name><version>20.8.0</version></daemon><scanner><name>openvas</name><version>OpenVAS 20.8.0</version></scanner><vts><version>0</version></vts></get_version_response>

This response is not indicated in the debug log of gvmd.

I struggle to do further debugging. Do I need to change a configuration? Where else can I look for the fault?

Hi ramsaut,

I have exactly the same problem. The greenbone master / slave architecture has always been a pain to make it work, but with 20.08 this is even more less understandable given the number of issues people are having with this.

Have you found a solution to this problem ?

Thanks

Note that I have created a bug request on github ospd-openvas; #341

1 Like

Hi,

just to clarify this topic from the development side. We are providing solutions for a master/sensor setup via our products and our products don’t face this specific issue. Also this is clearly a very complex enterprise feature where you can’t expect support or fixes instantly. My developer team has tried to reproduce them and fixed already several issues in this area without any payment or even relation to our products. Especially the TLS setup is difficult and error prone because of changes or even different behavior in different versions of the underlying libraries (e.g. GnuTLS and OpenSSL).

1 Like

I don’t expect instant fixes and I’m currently able to do a workaround by tunneling the sockets, so it’s not time critical. However, I would much prefer a cleaner solution, which is also usable by everyone else.
I would gladly contribute with a pull request. However, I’m stuck looking by myself. I’ll have a look on the github issue.

There is a workaround, which is to install the latest master release of gvm-libs. It was fixed with PR#394. After recompiling gvmd, it does work properly. I hope the team can backport this into the 20.08 stable release.

@ramsaut; in case you’re experienced enough to do it; this is PR greenbone/gvm-libs#394

1 Like

Seems a backport PR is now open:

Just got it to work with my own backport as well. Will try with the official backport.
Tremendous thanks!

1 Like

Thanks a lot !!

1 Like