Important: IP and bandwith limitations on Community Feed

Came to check on the feeds and saw this:

rsync: failed to connect to feed.community.greenbone.net (45.135.106.143): Connection timed out (110)
rsync: failed to connect to feed.community.greenbone.net (2a0e:6b40:20:106:20c:29ff:fe7f:d2ae): Network is unreachable (101)
rsync error: error in socket IO (code 10) at clientserver.c(127) [Receiver=3.1.3]

Not sure if my IP got flagged again. If it did, I’m at a loss as to why.

Thank you.

We don´t block anything at the moment, if the machine is full it is full … wait 30 min an try again.
IPv6 generally works lot better then IPv4.

Thanks Lukas. Will try at night.

If you are in dire need of a scanner, and not the most current feeds, you can use my container. It has a base set of data available. You can start the container with “-e SKIPSYNC=true” and it will start up and be useable in < 15m most of the time.

The date on the feeds in the container is currently 26 October. I normally update it once a week, but have not this week because of the ongoing issues.

Once it’s up and running you can try to get the sync to run with:
docker exec -it /sync.sh

Hy community!
We hope you all experience a good feed sync performance again. We still have to deal with attacks and misuse, and will continue working on our firewall rules. But we have already achieved some improvements.

  • We now have >200Mbit/s average transfer, therefore we assume we have a lot of happy syncing people.
  • We have > 100GiB transfer volume per h currently, so things seem to work well.

We have setup following rules:

  • We have a total rsync limit of 800 connections, thus supporting more than 55000 syncs per day,

Many thanks to our tech team (especially @Lukas) for doing night shifts and working hard on this solution.

We will continue improving our systems and add more redundancy in the next weeks.

3 Likes

Thanks all, i’ve successfully ran the update yesterday after added an entry to /etc/hosts:
45.135.106.142 feed.community.greenbone.net

To me, i just need to the update every week in my scanner which externally facing to internet and then copy it manually to other scanners (e.g. airgapped ones) by running simply following commands:

sudo tar -czvpf cert.tar /opt/gvm/var/lib/gvm/cert-data/
sudo tar -czvpf pugins.tar /opt/gvm/var/lib/openvas/plugins/
sudo tar -czvpf scap.tar /opt/gvm/var/lib/gvm/scap-data/
sudo tar -czvpf data-objects.tar /opt/gvm/var/lib/gvm/data-objects/

Some new information.

My GVM and Scanner IPs are still getting the “connection timed out error”. However if I use a different machine with a different IP address, I can down load the feeds with out problem. I know it was mentioned that nothing is being blocked but I don’t know why the GVM and Scanner machines are still not able to update feeds directly.

So, for now what I’ve done is set up an rsync of the feeds on this other machine with different IP address and I’ve modified the feed update script on the GVM and Scanner machines to rsync from this other machine instead.

This other machine will automatically download the feeds from feed.community.greenbone.net once daily at 9pm SGT (consecutive, not simultaneous downloads for the plugins, data-objects, cert-data and scap-data). I hope this will reduce some traffic from Greenbone.

Hello,

Same problem here also. I have an infrastructure of 9 scanners set up worldwide, behind a company network. None of them are being able to update anymore since November 9th.

All scanners are updating once a day, at various times. So it shouldn’t be considered as “excessive use”.

Any news on your side ?

Thanks

Hi,

I’m still facing this problem -on all my systems- despite I’m only syncing once a day. Who should I contact at greenbone to have my IPs unblocked and understand why it happened ?

Please help; I’m out of sync since November 9.

Thank you

The new rules work for the vast majority of community users, although problems like yours might occur, but not intended by us. Sorry for that. We need your IP adresses to investigate this further.

3 Likes

Thanks for your reply. I’ve just sent you my public IP addresses via private message.

1 Like

@tatooin Message received. Our tec team will investigate, lets see what we can do :-). . Most likely @DeeAnn will take over from here.

Elmar

3 Likes

Hi @tatooin,

our Tech Ops took a look and none of the IPs are showing as blocked. A suggestion was to make sure you are trying to sync with: feed.community.greenbone.net and also check your firewall rules to make sure the new IP address of 45.135.106.143 is allowed. Hope that helps and please let us know. Thanks!

1 Like

Hello;

Oh; I didn’t notice the community feed ip did change to 45.135.106.143.
On my firewalls, only 45.135.106.142 is allowed, which I guess was the old IP. Do you confirm ?

Sorry for that !

2 Likes

I’d actually written that (142) the first time also and had to edit it :sweat_smile:. Yep, it’s old. And no worries! :smiley:

1 Like

Sorry, I really havn’t seen that. Where did you post the feed repository IP did change to 143 ? Can you point me to the link ? Sorry again for the lost of time.

@tatooin no idea, I remember we published this but I cannot find the post, sorry that this change caused trouble. But I am happy to hear we could help to solve your problem, and wish happy scanning :slight_smile:

1 Like

Hello,

I have the following error when trying to update on three of my nodes:

@ERROR: max connections (40) reached – try again later
rsync error: error starting client-server protocol (code 5) at main.c(1675) [Receiver=3.1.3]

While it works fine on my other nodes. Two of the affected nodes are hosted on Amazon, the last one is hosted internally (IP is 165.225.20.92)

Could you check ?

Thanks !

Hi tatooin,

There is no guarantee or SLA of any resources for our community feed.
Please try again in a couple of hours. If you get this message, it means that feed server is full with simultaneous syncs. At lease we tell that the server is full, and no longer drop the connection.

2 Likes