Can't create task using any osp scanner "Given scanner_type was invalid"


#1

Hello everyone,

I hope this is the good place to ask, if not feel free to tell me where I should ask this.

I am running openVAS 9 on a fresh new install on Debian 9, regular scans work but OSP scanners don’t. In the following case this concerns ospd-w3af but ospd-debsecan and ospd-nmap don’t work the same way.
my gsad --version:

Greenbone Security Assistant 7.0.2
Copyright (C) 2010-2016 Greenbone Networks GmbH
License GPLv2+: GNU GPL version 2 or later

my pip freeze :

ospd==1.2.0
ospd-debsecan==1.2b1
ospd-nmap==1.0b1
ospd-w3af==1.0.0

my openvas-check-setup :

Step 1: Checking OpenVAS Scanner ... 
        OK: OpenVAS Scanner is present in version 5.1.1.
        OK: redis-server is present in version v=3.2.6.
        OK: scanner (kb_location setting) is configured properly using the redis-server socket: /tmp/redis.sock
        OK: redis-server is running and listening on socket: /tmp/redis.sock.
        OK: redis-server configuration is OK and redis-server is running.
        OK: NVT collection in /usr/local/var/lib/openvas/plugins contains 47727 NVTs.
        WARNING: Signature checking of NVTs is not enabled in OpenVAS Scanner.
        SUGGEST: Enable signature checking (see http://www.openvas.org/trusted-nvts.html).
        OK: The NVT cache in /usr/local/var/cache/openvas contains 47727 files for 47727 NVTs.
Step 2: Checking OpenVAS Manager ... 
        OK: OpenVAS Manager is present in version 7.0.2.
        OK: OpenVAS Manager database found in /usr/local/var/lib/openvas/mgr/tasks.db.
        OK: Access rights for the OpenVAS Manager database are correct.
        OK: sqlite3 found, extended checks of the OpenVAS Manager installation enabled.
        OK: OpenVAS Manager database is at revision 184.
        OK: OpenVAS Manager expects database at revision 184.
        OK: Database schema is up to date.
        OK: OpenVAS Manager database contains information about 47727 NVTs.
        OK: At least one user exists.
        OK: OpenVAS SCAP database found in /usr/local/var/lib/openvas/scap-data/scap.db.
        OK: OpenVAS CERT database found in /usr/local/var/lib/openvas/cert-data/cert.db.
        OK: xsltproc found.
Step 3: Checking user configuration ... 
        WARNING: Your password policy is empty.
        SUGGEST: Edit the /usr/local/etc/openvas/pwpolicy.conf file to set a password policy.
Step 4: Checking Greenbone Security Assistant (GSA) ... 
        OK: Greenbone Security Assistant is present in version 7.0.2.
        OK: Your OpenVAS certificate infrastructure passed validation.
Step 5: Checking OpenVAS CLI ... 
        OK: OpenVAS CLI version 1.4.5.
Step 6: Checking Greenbone Security Desktop (GSD) ... 
        SKIP: Skipping check for Greenbone Security Desktop.
Step 7: Checking if OpenVAS services are up and running ... 
        OK: netstat found, extended checks of the OpenVAS services enabled.
        OK: OpenVAS Scanner is running and listening on a Unix domain socket.
        OK: OpenVAS Manager is running and listening on a Unix domain socket.
        OK: Greenbone Security Assistant is listening on port 443, which is the default port.
Step 8: Checking nmap installation ...
        WARNING: Your version of nmap is not fully supported: 7.40
        SUGGEST: You should install nmap 5.51 if you plan to use the nmap NSE NVTs.
Step 10: Checking presence of optional tools ...
        OK: pdflatex found.
        WARNING: PDF generation failed, most likely due to missing LaTeX packages. The PDF report format will not work.
        SUGGEST: Install required LaTeX packages.
        OK: ssh-keygen found, LSC credential generation for GNU/Linux targets is likely to work.
        OK: rpm found, LSC credential package generation for RPM based targets is likely to work.
        OK: alien found, LSC credential package generation for DEB based targets is likely to work.
        OK: nsis found, LSC credential package generation for Microsoft Windows targets is likely to work.

It seems like your OpenVAS-9 installation is OK.

To create the scanner in openvas, I use:

openvasmd --create-scanner="w3af" --scanner-host=127.0.0.1 --scanner-port=1235 --scanner-type="OSP" \
--scanner-ca-pub=/usr/local/var/lib/openvas/CA/cacert.pem \
--scanner-key-pub=/usr/local/var/lib/openvas/CA/clientcert.pem \
--scanner-key-priv=/usr/local/var/lib/openvas/private/CA/clientkey.pem

To run ospd-w3af scanner, I use:

~# ospd-w3af -b 127.0.0.1 -p 1235 -k \
/usr/local/var/lib/openvas/private/CA/clientkey.pem -c \
/usr/local/var/lib/openvas/CA/clientcert.pem --ca-file \
/usr/local/var/lib/openvas/CA/cacert.pem -L DEBUG

When I verify the scanner with openvasmd --verify-scanner xxxxx I got

Scanner version: 2018.8.22.

note: in the logs of the scanner I got this for every verify I do, I don’t know if it’s related or no and I didn’t find a way to fix this:

2018-10-15 14:27:47,413 ospd.ospd: DEBUG: New connection from 127.0.0.1:60078
2018-10-15 14:27:49,430 ospd.ospd: DEBUG: Error: ('The read operation timed out',)
2018-10-15 14:27:49,433 ospd.ospd: DEBUG: 127.0.0.1:60078: Connection closed
2018-10-15 15:40:10,783 ospd.ospd: DEBUG: New connection from 127.0.0.1:60110
2018-10-15 15:40:12,823 ospd.ospd: DEBUG: Error: ('The read operation timed out',)
2018-10-15 15:40:12,824 ospd.ospd: ERROR: While handling client command:
Traceback (most recent call last):
  File "build/bdist.linux-x86_64/egg/ospd/ospd.py", line 491, in handle_client_stream
    response = self.handle_command(data)
  File "build/bdist.linux-x86_64/egg/ospd/ospd.py", line 792, in handle_command
    return self.handle_get_scanner_details()
  File "build/bdist.linux-x86_64/egg/ospd/ospd.py", line 739, in handle_get_scanner_details
    self.get_scanner_params_xml()
  File "build/bdist.linux-x86_64/egg/ospd/ospd.py", line 443, in get_scanner_params_xml
    ('mandatory', param['mandatory'])]:
KeyError: 'mandatory'

So, my verification made, I want to create a task that uses this scanner but I can’t save it due to error “Given scanner_type was invalid” :

I got 0 connection to the chosen scanner at this moment and I can’t find anything in the logs (maybe I can’t search). I suspect the gsad UI being responsible for this but I can’t find it.

If someone more expert than me (not very hard) and has any clue of what’s happening, I’d be very thankful

Thank you for reading


#2

I recommend to first check the ospd-based scanner setup separately before testing the full integration.

Use gvm-tools for this, like described here (from a newer ospd, but most applies for ospd 1.2 as well):

https://github.com/greenbone/ospd/blob/master/doc/USAGE-ospd-scanner


#3

Hi and thank you for your answer !

I installed gvm-tools but I don’t know how to use it. In documentation, it is said to call gvm-cli socket on the ospd-scanner socket on <prefix>/var/run/ospd-scanner.sock but I can only find openvasmd.sock and openvassd.sock at that location.

When I try to run “simple” comands like gvm-cli socket --xml “<get_version/>” I get the error [Errno 2] no such file or directory

edit : got same errors after updating to ospd 1.3.1

Thank you again for your help


#4

You may want to use “tls” instead of “socket” when starting one of the gvm-tools.
Please see the README here for examples:


#5

Yes I figured this out as well, thank you,

So I tried to create a task using gvm-pyshell through tls and found out I couldn’t create one without a configuration. That was my first mistake :frowning:
I tried creating a w3af one through GUI (I didn’t find another way to do so since gvm-tools can only copy one). I got the error "Failed to get params from scanner - the scanner may be offline or not configured correctly"
I tried with nmap and it worked correctly. So now my problem is with ospd-w3af configuration and not with ospd-scanners in general.

In the logs of ospd-w3af I have the following:

2018-10-17 11:33:21,048 ospd.ospd: ERROR: While handling client command:
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/ospd/ospd.py", line 522, in handle_client_stream
    response = self.handle_command(data)
  File "/usr/local/lib/python2.7/dist-packages/ospd/ospd.py", line 826, in handle_command
    return self.handle_get_scanner_details()
  File "/usr/local/lib/python2.7/dist-packages/ospd/ospd.py", line 773, in handle_get_scanner_details
    self.get_scanner_params_xml()
  File "/usr/local/lib/python2.7/dist-packages/ospd/ospd.py", line 468, in get_scanner_params_xml
    ('mandatory', param['mandatory'])]:
KeyError: 'mandatory'

Is my ospd-w3af badly installed or something else ?

Thank you very much again !


#6

Ok that was again my fault, w3af can only run with ospd 1.0.0
I had changed the dependency ospd==1.0.0 to ospd>=1.0.0 and now I can create the config and the task. I changed it again and reinstalled ospd 1.0.0

When I start it tho the task stops at 1% with the logs:

2018-10-17 11:58:01,335 ospd.ospd: INFO: c8d0ef74-611b-4742-b7a2-f8b91323519d: Scan started.
2018-10-17 11:58:01,336 ospd.ospd: DEBUG: SSL close error: 
2018-10-17 11:58:01,337 ospd.ospd: INFO: 127.0.0.1: Scan started.
2018-10-17 11:58:01,337 ospd_w3af.wrapper: INFO: w3af scan using audit_high_risk profile.
2018-10-17 11:58:01,427 ospd.ospd: DEBUG: New connection from 127.0.0.1:48954
2018-10-17 11:58:03,445 ospd.ospd: DEBUG: SSL error: The read operation timed out
2018-10-17 11:58:03,446 ospd.ospd: DEBUG: 127.0.0.1:48954: Connection closed

When I resume it I have:

2018-10-17 11:58:39,251 ospd.ospd: DEBUG: New connection from 127.0.0.1:48986
2018-10-17 11:58:43,331 ospd.ospd: DEBUG: SSL error: The read operation timed out
2018-10-17 11:58:46,482 ospd.ospd: DEBUG: 127.0.0.1:48986: Connection closed
2018-10-17 11:58:46,489 ospd.ospd: INFO: d064be4c-c381-40e4-a30a-863b98f8cd3d: Scan started.
2018-10-17 11:58:46,490 ospd.ospd: INFO: 127.0.0.1: Scan started.
2018-10-17 11:58:46,490 ospd_w3af.wrapper: INFO: w3af scan using audit_high_risk profile.
2018-10-17 11:58:46,678 ospd.ospd: DEBUG: New connection from 127.0.0.1:48992
2018-10-17 11:58:48,699 ospd.ospd: DEBUG: SSL error: The read operation timed out
2018-10-17 11:58:48,700 ospd.ospd: DEBUG: 127.0.0.1:48992: Connection closed
2018-10-17 11:59:04,309 ospd_w3af.wrapper: DEBUG: Unknown severity information.
2018-10-17 11:59:04,309 ospd.ospd: ERROR: While scanning 127.0.0.1:
Traceback (most recent call last):
  File "build/bdist.linux-x86_64/egg/ospd/ospd.py", line 375, in start_scan
    self.exec_scan(scan_id, target)
  File "build/bdist.linux-x86_64/egg/ospd_w3af/wrapper.py", line 206, in exec_scan
    options['headers'], options['res_status'])
  File "build/bdist.linux-x86_64/egg/ospd_w3af/wrapper.py", line 239, in store_scan_results
    request = vuln.getElementsByTagName('httprequest')[0]
IndexError: list index out of range
2018-10-17 11:59:04,315 ospd.ospd: INFO: c8d0ef74-611b-4742-b7a2-f8b91323519d: Scan finished.
2018-10-17 11:59:34,048 ospd_w3af.wrapper: DEBUG: Unknown severity information.
2018-10-17 11:59:34,048 ospd.ospd: ERROR: While scanning 127.0.0.1:
Traceback (most recent call last):
  File "build/bdist.linux-x86_64/egg/ospd/ospd.py", line 375, in start_scan
    self.exec_scan(scan_id, target)
  File "build/bdist.linux-x86_64/egg/ospd_w3af/wrapper.py", line 206, in exec_scan
    options['headers'], options['res_status'])
  File "build/bdist.linux-x86_64/egg/ospd_w3af/wrapper.py", line 239, in store_scan_results
    request = vuln.getElementsByTagName('httprequest')[0]
IndexError: list index out of range
2018-10-17 11:59:34,049 ospd.ospd: INFO: d064be4c-c381-40e4-a30a-863b98f8cd3d: Scan finished.