Still no feed update

The new messages are looking quite different to the initial ones. Could this be related to the following problem in the Linux Kernel in conjunction with rsync:

Thx cfi,

Hi Lukas,

checked again this morning. Client is directly behind the router, only ipv6 is enabled and waited many hours without any action.

pre2008/zyxel_pwd.nasl
3,084 100% 5.26kB/s 0:00:00 (xfr#74993, to-chk=115/76168)

sent 4,562,718 bytes received 18,113,521 bytes 223,411.22 bytes/sec
total size is 356,153,916 speedup is 15.71

At this point the system is hanging.

Checked kernel:

└─$ uname -r
5.10.0-kali9-amd64
└─$ uname -a
Linux kali 5.10.0-kali9-amd64 #1 SMP Debian 5.10.46-1kali1 (2021-06-25) x86_64 GNU/Linux
└─$ cat /proc/version
Linux version 5.10.0-kali9-amd64 (devel@kali.org) (gcc-10 (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2) #1 SMP Debian 5.10.46-1kali1 (2021-06-25)
└─$ sudo dmesg | grep Linux [ 0.000000] Linux version 5.10.0-kali9-amd64 (devel@kali.org) (gcc-10 (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2) #1 SMP Debian 5.10.46-1kali1 (2021-06-25)

So 5.10.0 is the latest kernel for kali, but guess before rc1?

Compiling my own kernel does not make sense as i will loose future kernel updates…

But - why did it work with this kali installation over months? shouldn’t the bug be a problem in olders setups also?

Strange …

Also - if i read the source code right, this is a IPv4 issue. But i am using IPv6 only?

That is really strange but at least i am not alone with this problem …

If you send me your source address i can have a look …
At the moment our community server is distributing 1.5TB community feed per day.

Are you 100% sure that your ISP is not messing up with any NAT technology ?

Did you asked your ISP ?

1 Like

guess it will not be possible to ask ISP - they do not know this.

I will try again tomorrow and send my v6IP via PN if this is O.K. for you.

Regarding NAT - i am using this since years / decades and never had such problems. But - in IT “never” is only true until sommething happens the first time :wink:

Meanwhile you can try with our VM, GSM Trial that is the “only” supported platform to ensure this is not a issue due to uncoordinated integrations.

1 Like

i have installed GSM Trial as VM and started in LAN. Here’s the first result:

grafik

With this setup the machine is running on IPv4. It did not get an IPv6 adress automatically.

Next i move the network adapter into my WAN zone, directly behind my router.

Let’s look what is happening there …

guess i will give up now.
I was told that my system is resetting the connection - (i do believe this due the log entry shown) my system (wireshark) is telling me the opposite.
Also - even the trial VM is unable to synch the feed…
Next i tried with a different access (normal is vodafone and zyxel, tryout was telekom and huawei) - still no persistent thread update.
sometimes the nvt’s worked but the system stopped the next step.
So i am not only confused but also at the end of what i could try.
here’s the result of my findings:

The idea that it is a problem with rsync and old bug can not be true as it does not work with the appliance.
The hardware and provider should not be the problem - i tested two different combinations.
My firwall can not be the cause - i tested directly behind the router.
IPv4 / NAT - i tested with IPv6 also without getting a solution.

I do know that it worked fine without problems in the previous version. The problems started after the update to the latest release

So all i can try is to install the previous release.

On the other hand it is still strange, that so many other people do have problems with the feed update …

so here’s my last try before giving up.
I made a fresh install of kali 2021.2 as predefined VM.
After “apt update && apt full-upgrade” i installed sudo apt install openvas gvm following this guideline:
https://www.ntbrad.com/2020/12/08/kali-and-openvas-gvm-setup/
had a small problem with postgresql due to “locales” that was fixed soon. postgresql 13 was default and the only one installed.
Next was gvm-setup with this result:
└─$ sudo gvm-setup 1 ⨯
Creating openvas-scanner’s certificate files

[>] Creating database
CREATE ROLE
GRANT ROLE
CREATE EXTENSION
CREATE EXTENSION
[>] Migrating database
[>] Checking for admin user
[] Creating user admin for gvm
[
] Please note the generated admin password
[] User created with password ‘xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx’.
[
] Define Feed Import Owner
[>] Updating OpenVAS feeds
[*] Updating: NVT

At this point the system hang without any action.

So after some time i made a reboot and tried again. Still the same “hang” at the same position.

use iftop to check if there’s any action on the network but - nothing.

So to all using kali and having the same problems - i did not find any solution for this problem.

Maybe i will find some help at the kali forums …

I just checked we had last day 340.000 feed syncs, so there must be something wrong at the infrastructure or your ISP. I suggest you contact your support and ask about CGN and real IPs ?

1 Like

Thx Lukas,

guess vodafone will not be willing to change something for me.

I will need to check for a different solution that i can offer my customers instead.

Have a good time.

1 Like

I am back …
I found out a hint regarding

sync error: received SIGINT, SIGTERM, or SIGHUP (code 20) at rsync .c(276)

on

Gentoo Forums :: View topic - rsync keeps failing in io.c: rsync error: error in rsync pro.

So i just downloaded the source for rsync following this guide:

Compile rsync from source on Ubuntu - BeginningLinux.com

!!! Take the latest version !!!

Needed to get some missing dependencies but finally i was able to build a current rsync version.

I did not yet made my own “make.conf” as mentioned in the article so there’s no config above the given source.

After this action i was able to run my “gvm-setup” without problems.

Came back to the known problem with:

    ERROR: Greenbone Security Assistant too old or too new: 21.4.1~dev1
    FIX: Please install Greenbone Security Assistant >= 21.04.

ERROR: Your GVM-21.4.1 installation is not yet complete!

but this could be fixed very fast following this guide:

GSA is too old or too new -- error - Greenbone Community Edition - Greenbone Community Forum

I will now continue testing.

As far a i can understand the problem it might occur with “no so fast” network connections like the one i have (LTE, <20 MBit/s).

I’d be happy if others could try this out to see, if this might be the solution for the problem…

2 Likes

If its working for you, that’s awesome. The reason we’ve raised the issue is that its not working for us, could we possibly address that? I’m just not sure I’ll get anywhere by raising it with my ISP

Got the following error:
*] Updating NVT (Network Vulnerability Tests feed from Greenbone Security Feed/Community Feed)
rsync: [Receiver] failed to connect to feed.community.greenbone.net (45.135.106.143): Connection timed out (110)
rsync: [Receiver] failed to connect to feed.community.greenbone.net (2a0e:6b40:20:106:20c:29ff:fe7f:d2ae): Connection timed out (110)
rsync error: error in socket IO (code 10) at clientserver.c(137) [Receiver=3.2.3]

Added an entry to /etc/hosts:
45.135.106.142 feed.community.greenbone.net

Ran the gvm-feed-update successfully after that.

1 Like

That is the old server, try the new one after your DNS resolution …

The 143 IP Address was failing, the .142 worked. What is the new address?

So new Server is in place, we migrated to nft from IPTables, with the following important rules.

More then 10 TCP SYN (rsync tries) per IP per minute will result in a one day (1440 minutes) block, one simultaneous IP is allowed.

All lists are dynamic.

Here:

# host feed.community.greenbone.net
feed.community.greenbone.net has address 45.135.106.143
feed.community.greenbone.net has IPv6 address 2a0e:6b40:20:106:20c:29ff:fe7f:d2ae

1 Like

That is the address that was taking an error. I just commented out the /etc/hosts entry and reran the update and it worked this time. Mostly like that server is getting overloaded. Thanks for the insight.

That is for the power abusive user, if we got more then 10 connections from the same IP within 1 minute, they have to wait 24h. One IP parallel as well. That will prevent scans, internet scam, brute force on ssh as well abuse of our public feed services.

Here our block rule to be as transparent as possibe:

[..]
 set denylist {
                type ipv4_addr
                size 65535
                flags dynamic,timeout
                timeout 1d
                elements = { 
[..]
 ip protocol tcp ct state new,untracked limit rate over 10/minute add @denylist { ip saddr }
 ip saddr @denylist drop
 tcp dport 873 meter rsync1 size 65535 { ip saddr & 255.255.255.0 ct count over 3 } counter packets 0 bytes 0 reject with tcp reset
 tcp dport 873 meter rsync2 size 65535 { ip saddr ct count over 1 } counter packets 1 bytes 64 reject with tcp reset
 tcp dport 873 tcp flags syn / fin,syn,rst,ack counter packets 2792 bytes 165204 accept
1 Like

That is good to know. I will have to let the students in my class know that the install may fail occasionally and they’ll have to keep trying the gvm-feed-update until it eventually works as was the case this morning.