Important: IP and bandwith limitations on Community Feed

Hi @asawula ,

Maybe you should check just now, as it seems to be working now…!

If some one runs a installation with 10.000 Units trying to sync simultaneously he will definitively overload our service. Bedsides this we see attacks to our service so the total sync bandwidth gets down on the back of everyone. We implement some counter measure and then again we got a new attack. Now we wen´t from SYN Cookies to random early drop and total limit of 150 simultaneous connections.

As you can see the traffic gets down due to attacks, and yes you are absolutely right we need to think about a free key that can sync once a day with a limited amount of installation to act fair to the community. The plan was to keep the entry level low and enable a anonymous service but it seems that some take unfair advantage from this.


     2021-10-18    64.58 GiB |    2.07 TiB |    2.13 TiB |  216.94 Mbit/s
     2021-10-19    69.21 GiB |    1.68 TiB |    1.75 TiB |  177.94 Mbit/s
     2021-10-20    71.70 GiB |    1.71 TiB |    1.78 TiB |  181.56 Mbit/s
     2021-10-21    91.24 GiB |    1.81 TiB |    1.90 TiB |  193.18 Mbit/s
     2021-10-22    64.23 GiB |    1.78 TiB |    1.84 TiB |  187.54 Mbit/s
     2021-10-23    57.36 GiB |    1.47 TiB |    1.52 TiB |  155.13 Mbit/s
     2021-10-24    58.24 GiB |    1.83 TiB |    1.89 TiB |  192.49 Mbit/s
     2021-10-25    62.04 GiB |    1.98 TiB |    2.04 TiB |  207.68 Mbit/s
     2021-10-26    59.62 GiB |    1.78 TiB |    1.84 TiB |  187.46 Mbit/s
     2021-10-27    55.03 GiB |    1.50 TiB |    1.55 TiB |  157.95 Mbit/s
     2021-10-28    71.29 GiB |    1.41 TiB |    1.48 TiB |  150.65 Mbit/s
     2021-10-29    54.74 GiB |    1.34 TiB |    1.40 TiB |  142.23 Mbit/s
     2021-10-30    48.73 GiB |    1.29 TiB |    1.34 TiB |  136.43 Mbit/s
     2021-10-31    51.59 GiB |    1.61 TiB |    1.66 TiB |  169.25 Mbit/s
     2021-11-01    43.60 GiB |    1.19 TiB |    1.23 TiB |  125.35 Mbit/s
     2021-11-02    37.57 GiB |  624.34 GiB |  661.91 GiB |   65.81 Mbit/s
     2021-11-03    27.65 GiB |  946.15 GiB |  973.79 GiB |   96.82 Mbit/s
     2021-11-04    14.49 GiB |  601.45 GiB |  615.94 GiB |   61.24 Mbit/s
     2021-11-05    38.85 GiB |    0.98 TiB |    1.02 TiB |  103.37 Mbit/s
     2021-11-06     3.45 GiB |  225.12 GiB |  228.57 GiB |   22.72 Mbit/s
     2021-11-07     2.04 GiB |   93.49 GiB |   95.53 GiB |   20.26 Mbit/s

Nope I still get the rsync error.

Can we / someone help with load balancing or similar ?

What about using a CDN (CloudFlare, CloudFront, Akamai, etc…)? The feeds are static content, changing once per day or less.

We are working actively on a solution, first upgrade our firewall to get us better protection and then a load-balancer to more then two servers. But if someone abuse our service even a CDN will not help and comes with a heavy price tag.

Hi all,

gvm-check-setup 21.4.3
Test completeness and readiness of GVM-21.4.3
Step 1: Checking OpenVAS (Scanner)…
OK: OpenVAS Scanner is present in version 21.4.3.
OK: Server CA Certificate is present as /var/lib/gvm/CA/servercert.pem.
Checking permissions of /var/lib/openvas/gnupg/*
OK: _gvm owns all files in /var/lib/openvas/gnupg
OK: redis-server is present.
OK: scanner (db_address setting) is configured properly using the redis-server socket: /var/run/redis-openvas/redis-server.sock
OK: redis-server is running and listening on socket: /var/run/redis-openvas/redis-server.sock.
OK: redis-server configuration is OK and redis-server is running.
OK: _gvm owns all files in /var/lib/openvas/plugins
OK: NVT collection in /var/lib/openvas/plugins contains 77656 NVTs.
Checking that the obsolete redis database has been removed
OK: No old Redis DB
OK: ospd-OpenVAS is present in version 21.4.3.
Step 2: Checking GVMD Manager …
OK: GVM Manager (gvmd) is present in version 21.4.4.
Step 3: Checking Certificates …
OK: GVM client certificate is valid and present as /var/lib/gvm/CA/clientcert.pem.
OK: Your GVM certificate infrastructure passed validation.
Step 4: Checking data …
ERROR: SCAP DATA are missing.
FIX: Run the SCAP synchronization script greenbone-feed-sync.
sudo runuser -u _gvm – greenbone-feed-sync --type SCAP.

ERROR: Your GVM-21.4.3 installation is not yet complete!

Please follow the instructions marked with FIX above and run this
script again.

I can’t finish setting up my OpenVAS because of this issue.

Where in the setup does need to be done? Is there a set of instructions?

Thank you!

Hi,

Where can I check to see if my IP has been banned? I have my own public IP address but we are within the university so I don’t know if it traffic gets routed through some common proxy or something.

Thank you.

I’m similar, I have a dedicated public IP and have been trying to install openvas for lessons. I fear I have ran setup a few too many times… I would like to also check if my IP is banned, please.

Thank you for the people who is working hard to solve this problem (Lukas, DeeAnn, etc.). Hopefully it can get back to normal soon. I really enjoy using this service.

5 Likes

Thanks for all working on the issue, appreciate how frustrating these things can be.

I’m wondering if I sync incorrectly, I do it manually once a month running…

greenbone-nvt-sync
greenbone-certdata-sync
greenbone-scrapdata-sync

Although a bit slower, the first two commands worked without issue, but the last one is timing out with "rsync: [Receiver] failed to connect to feed.community.greenbone.net Network is unreachable (101) error in socket IO (code 10) at clientserver.c(137) [Receiver=3.2.3]

Is that expected at the moment? More to the point should i not be running all three, one after another, once per month?

To enable SYN COOKIES, you must have access to /etc/sysctl.conf. To see if you have it enabled, enter sysctl -n net.ipv4.tcp_syncookies. This will show if it is either enabled or not. However, the main issue are the rsync feeds. Please be patient as Greenbone fixes this issue.

1 Like

Thank you @roma. I tried earlier and it managed to update for my the main gvm, but the 2 remote scanners failed for updating.

Will wait for a resolution from GB.

Is there any other way of updating the SCAP db, etc.?

Please look Lukas post here: Still no feed update - #40 by Lukas

Please look Lukas post here: Still no feed update - #40 by Lukas

Saw this, checked that my machines are resolving feed.community.greenbone.net correctly to the new IP address (45.135.106.143), but it still won’t sync. Same ‘connection timed out’ message.

I strongly encourage you to consider CloudFlare for CDN. They have a free tier that I think you can still use for basic CDN capabilities. I run an MSSP and we stand CloudFlare up in front of websites and WebApps all the time to improve site performance and security.

Welcome jeffleder to our community!
Thanks for the advise. We are in the process evaluating different solutions, CDN like cloudflare is one them. We want to fix this persistently, so it may take some time to the final solution (as we already are in the end-of-year business noise).

3 Likes