Passing arguments to gvm-tools

systemgeek · May 28, 2019, 12:17pm

No matter what I do I am unable to pass arguments to the gvm-tools. I am running:
Centos 7.6
gvmd-8.0.0-6928.el7.art.x86_64
greenbone-security-assistant-8.0.0-6932.el7.art.x86_64
greenbone-vulnerability-manager-10.0.0-6947.el7.art.noarch

And via pip gvm-tools 2.0.0b1

I have configured my gvm-tools.conf like so:
[main]
timeout=60
hostname=127.0.0.1
tls_path=/var/lib/gvm/CA
[gmp]
gmp_username=USERME
gmp_password=PASSSSS
[ssh]
username=USERME
password=PASSSSS
port=9392
[unixsocket]
socketpath=/usr/local/var/run/gvmd.sock
[tls]
port=9392
certfile=%(tls_path)s/clientcert.pem
keyfile=%(tls_path)s/servercert.pem
cafile=%(tls_path)s/cacert.pem

And it seems that regardless of the position of the options or what options I give gvm-pyshell or gvm-cli it keeps getting confused over the options.

[USERME@pci-sec02 .config]$ gvm-pyshell --config ssh  --hostname=host.domain.net
usage: gvm-pyshell [-h] [-c [CONFIG]]
                    [--log [{DEBUG,INFO,WARNING,ERROR,CRITICAL}]]
                    [--timeout TIMEOUT] [--gmp-username GMP_USERNAME]
                    [--gmp-password GMP_PASSWORD] [-V] [--protocol {GMP,OSP}]
                   CONNECTION_TYPE ...
gvm-pyshell: error: the following arguments are required: CONNECTION_TYPE
[USERME@pci-sec02 .config]$ gvm-pyshell ssh  --config --hostname=host.domain.net
usage: gvm-pyshell [-h] [-c [CONFIG]]
                   [--log [{DEBUG,INFO,WARNING,ERROR,CRITICAL}]]
                   [--timeout TIMEOUT] [--gmp-username GMP_USERNAME]
                   [--gmp-password GMP_PASSWORD] [-V] [--protocol {GMP,OSP}]
                  CONNECTION_TYPE ...
gvm-pyshell: error: unrecognized arguments: --config
[USERME@pci-sec02 .config]$ gvm-pyshell ssh  --hostname=host.domain.net --config
usage: gvm-pyshell [-h] [-c [CONFIG]]
                    [--log [{DEBUG,INFO,WARNING,ERROR,CRITICAL}]]
                    [--timeout TIMEOUT] [--gmp-username GMP_USERNAME]
                    [--gmp-password GMP_PASSWORD] [-V] [--protocol {GMP,OSP}]
                    CONNECTION_TYPE ...
gvm-pyshell: error: unrecognized arguments: --config
[USERME@pci-sec02 .config]$ gvm-pyshell ssh  --gmp-username=USERME --gmp-password=PASSSSS --hostname=host.domain.net
usage: gvm-pyshell [-h] [-c [CONFIG]]
                   [--log [{DEBUG,INFO,WARNING,ERROR,CRITICAL}]]
                   [--timeout TIMEOUT] [--gmp-username GMP_USERNAME]
                   [--gmp-password GMP_PASSWORD] [-V] [--protocol {GMP,OSP}]
                   CONNECTION_TYPE ...
gvm-pyshell: error: unrecognized arguments: --gmp-username=USERME --gmp-password=PASSSSS
[USERME@pci-sec02 .config]$ gvm-pyshell --gmp-username=USERME --gmp-password=PASSSSS --hostname=host.domain.net ssh
usage: gvm-pyshell ssh [-h] --hostname HOSTNAME [--port PORT]
                       [--ssh-username SSH_USERNAME]
                       [--ssh-password SSH_PASSWORD] [-i]
                       [SCRIPT] [ARG [ARG ...]]
gvm-pyshell ssh: error: the following arguments are required: --hostname
[USERME@pci-sec02 .config]$ gvm-pyshell --ssh-username=USERME --ssh-password=PASSSSS --hostname=host.domain.net ssh
usage: gvm-pyshell ssh [-h] --hostname HOSTNAME [--port PORT]
                       [--ssh-username SSH_USERNAME]
                       [--ssh-password SSH_PASSWORD] [-i]
                       [SCRIPT] [ARG [ARG ...]]
gvm-pyshell ssh: error: the following arguments are required: --hostname
[USERME@pci-sec02 .config]$ gvm-pyshell --ssh-username=USERME --ssh-password=PASSSSS ssh --hostname=host.domain.net
usage: gvm-pyshell [-h] [-c [CONFIG]]
                   [--log [{DEBUG,INFO,WARNING,ERROR,CRITICAL}]]
                   [--timeout TIMEOUT] [--gmp-username GMP_USERNAME]
                   [--gmp-password GMP_PASSWORD] [-V] [--protocol {GMP,OSP}]
                   CONNECTION_TYPE ...
gvm-pyshell: error: unrecognized arguments: --ssh-username=USERME --ssh-password=PASSSSS

What am I doing wrong here???
Louis

bricks · May 28, 2019, 12:33pm

Hi,

first of all I am wondering why a beta version got packaged for end users. Beta version are likely to have bugs and are mostly intended for early adopters.

I’ve re-written the parser of gvm-tools in master since the this last release. So you could try to use the master branch instead.

systemgeek · May 28, 2019, 2:31pm

Not sure how I got a beta release but I did with pip install. I will un-install it and try to re-install it again.

Thank you.

systemgeek · May 28, 2019, 3:55pm

I just download the master branch via GIT and when I ran a pip install . It told me that it was installing:
Successfully built gvm-tools
Installing collected packages: gvm-tools
Found existing installation: gvm-tools 2.0.0b1
Uninstalling gvm-tools-2.0.0b1:
Successfully uninstalled gvm-tools-2.0.0b1
Successfully installed gvm-tools-2.0.0b1

This is what I used to download the tools: git clone https://github.com/greenbone/gvm-tools.git

Louis

bricks · May 28, 2019, 6:19pm

Sorry I did miss these lines. I thought you were using some gvm-tools packages for centos.

Did using gvm-tools from master fix your issue? If not could you please create a ticket at Sign in to GitHub · GitHub ? Thanks in advance!

systemgeek · May 28, 2019, 6:26pm

I tried re-cloning from master (which is what I always do anyway) but it was the same version. On the github page it says:
Latest commit 1d558cd on Apr 23

So I am not sure where you updated too.

I can open a ticket if you still want me to. Or I can try the version in the repo:
gvm-tools.noarch 1.4.1-6952.el7.art

However, the reason I uninstalled the version in the repo was because gvm-cli and gvm-pyshell were not found as a normal user but only as root. And this was not a path problem in the shell since my user could see /usr/bin/gvm-cli. The python scripts in gvm-tools could not find things.

Louis

bricks · May 28, 2019, 6:33pm

Yes it seems I did not change the version number after the beta1 release. My fault. Should have been incremented.

Nevertheless you should be able to install the correct version via pip install --user https://github.com/greenbone/gvm-tools from the master branch.

systemgeek · May 28, 2019, 6:35pm

Not sure how but I am game to try…

systemgeek · May 28, 2019, 6:41pm

Since I have it cloned I did a git pull to try to pull any updates. Said it was up to date.

A git log -1 master shows me this:
[user@pci-sec02 gvm-tools]$ git log -1 master
commit 1d558cdc190ac0e8282b1f01862f8744b8ef5ae5
Merge: 025b6f0 8a21bd0
Author: Juan José Nicola jjnicola@gmail.com
Date: Tue Apr 23 10:50:58 2019 +0200

    Merge pull request #190 from bjoernricks/update-dependencies

    Update dependencies

So I do not think your commits have been pushed to master branch yet. But anyway. A pip install . --user gives me this:

Successfully built gvm-tools
Installing collected packages: gvm-tools
Successfully installed gvm-tools-2.0.0b1

I am happy to test again but I do not think its been updated.

bricks · May 28, 2019, 7:34pm

Sorry I guess we both misunderstood each other.

I did assume you did use pip install gvm-tools for the installation. Using this command will install the version from pypi.org which is the same as https://github.com/greenbone/gvm-tools/tree/v2.0.0.beta1

Since this release/tag I’ve changed the code in the master branch regarding parsing the console line arguments. This code still has some small issue I would like to fix before the final 2.0.0 release but may already have fixed your issues hopefully. If not please create a new ticket via https://github.com/greenbone/gvm-tools/issues/new

The last commit in the master branch is https://github.com/greenbone/gvm-tools/commit/1d558cdc190ac0e8282b1f01862f8744b8ef5ae5 .

systemgeek · May 29, 2019, 12:30pm

So that I fully understand what you are asking do you want me to install with:
pip install gvm-tools --user (so it installs from pypi.org)
Or do you want me to do:
git clone https://github.com/greenbone/gvm-tools.git
cd gvm-tools
pip install . --user

Louis

bricks · May 29, 2019, 3:27pm

You should try to use gvm-tools from git at the master branch.

This should work. Maybe also add the -e flag like pip install -e --user .

systemgeek · May 31, 2019, 12:48pm

Regardless if I do:
git clone https://github.com/greenbone/gvm-tools.git
cd gvm-tools
pip install . --user
or
pip install gvm-tools #(to install from pypi.org)
I end up with the same version of gvm-tools-2.0.0b1. Is there another place I can down load it? Or is there a branch I can down load???

bricks · May 31, 2019, 1:51pm

As I already wrote that’s ok and expected. I did forgot to increment the version in master branch after the 2.0.0beta1 release. Therefore both will show 2.0.0b1 as version but contain different code.

systemgeek · May 31, 2019, 2:33pm

Ok tried that. Now getting this error:
[lbohm@pci-sec02 ~]$ gvm-cli ssh --hostname=127.0.0.1 --xml="<get_version/>"
ERROR:paramiko.transport:Exception: Error reading SSH protocol banner
ERROR:paramiko.transport:Traceback (most recent call last):
ERROR:paramiko.transport: File “/usr/lib/python3.6/site-packages/paramiko/transport.py”, line 1930, in _check_banner
ERROR:paramiko.transport: buf = self.packetizer.readline(timeout)
ERROR:paramiko.transport: File “/usr/lib/python3.6/site-packages/paramiko/packet.py”, line 331, in readline
ERROR:paramiko.transport: buf += self._read_timeout(timeout)
ERROR:paramiko.transport: File “/usr/lib/python3.6/site-packages/paramiko/packet.py”, line 487, in _read_timeout
ERROR:paramiko.transport: raise EOFError()
ERROR:paramiko.transport:EOFError
ERROR:paramiko.transport:
ERROR:paramiko.transport:During handling of the above exception, another exception occurred:
ERROR:paramiko.transport:
ERROR:paramiko.transport:Traceback (most recent call last):
ERROR:paramiko.transport: File “/usr/lib/python3.6/site-packages/paramiko/transport.py”, line 1782, in run
ERROR:paramiko.transport: self._check_banner()
ERROR:paramiko.transport: File “/usr/lib/python3.6/site-packages/paramiko/transport.py”, line 1934, in _check_banner
ERROR:paramiko.transport: raise SSHException(‘Error reading SSH protocol banner’ + str(e))
ERROR:paramiko.transport:paramiko.ssh_exception.SSHException: Error reading SSH protocol banner
ERROR:paramiko.transport:
(‘SSH Connection failed’, SSHException(‘Error reading SSH protocol banner’,))

Googling that it seems related to this issue: https://github.com/paramiko/paramiko/issues/673. But I am running:
[lbohm@pci-sec02 ~]$ pip list
Package Version
------------ -------
asn1crypto 0.24.0
cffi 1.9.1
cryptography 2.3
defusedxml 0.6.0
gvm-tools 2.0.0b1
idna 2.7
lxml 4.3.3
paramiko 2.1.1
pip 19.1.1
ply 3.9
pyasn1 0.1.9
pycparser 2.14
python-gvm 1.0.0b2
setuptools 41.0.1
six 1.11.0
wheel 0.33.4
And from what I can see the problem is not really in paramiko but with the code used to called it. Specifically the time out.

bricks · May 31, 2019, 3:32pm

SSH is NOT provided out of the box. You need to setup SSH by yourself to work with openvasmd/gvmd.

Please take a look at the gmv-tools docs for the different connection types

https://gvm-tools.readthedocs.io/en/latest/connectiontypes.html

Also you should take a look at the GVM architecture to understand the daemons and their communication

systemgeek · June 5, 2019, 12:05pm

Can you please explain to me why this does not work:
[lbohm@pci-sec02 ~]$ gvm-cli socket --gmp-username=admin --gmp-password=8gUtP*zeUhefmpT --xml="<get_tasks/>"
usage: gvm-cli [-h] [-c [CONFIG]]
[–log [{DEBUG,INFO,WARNING,ERROR,CRITICAL}]]
[–timeout TIMEOUT] [–gmp-username GMP_USERNAME]
[–gmp-password GMP_PASSWORD] [-V] [–protocol {GMP,OSP}]
CONNECTION_TYPE …
gvm-cli: error: unrecognized arguments: --gmp-username=admin --gmp-password=8gUtP*zeUhefmpT

But this does work;
gvm-cli socket --xml "<commands><authenticate><credentials><username>admin</username><password>8gUtP*zeUhefmpT</password></credentials></authenticate> <get_tasks/> </commands>"

The documentation at: https://gvm-tools.readthedocs.io/en/latest/connectiontypes.html and gvm-cli -h clearly show that --gmp-username and --gmp-password should work.

bricks · June 5, 2019, 12:10pm

Could you try moving the --gmp arguments before socket like

gvm-cli --gmp-username=admin --gmp-password=foo socket ...

This should be a bug in the current argument parser code. It’s still in beta state.

systemgeek · June 5, 2019, 5:51pm

That worked. Thank you.