Threat Level: green Handler on Duty: Didier Stevens

SANS ISC: How did this Memcache thing happen? - Internet Security | DShield SANS ISC InfoSec Forums


Sign Up for Free!   Forgot Password?
Log In or Sign Up for Free!
How did this Memcache thing happen?

As pointed out earlier (https://isc.sans.edu/forums/diary/Why+we+Dont+Deserve+the+Internet+Memcached+Reflected+DDoS+Attacks/23389/) this memcached reflected DDoS thing is pretty bad.  How bad?  Well, US-CERT updated its UDP-Based Amplification Attacks advistory (https://www.us-cert.gov/ncas/alerts/TA14-017A) to add Memcache to the list of potential attack vectors.  The really telling bit is the chart that shows the Bandwidth Amplification Factor.  Before memcache was added the largest factor was 556.9 from NTP where each byte sent in to a vulnerable server would return about 557 bytes in attack traffic.  Memecache is listed as 10,000 to 51,000.  That's remarkably large.

How common is an internet-facing memcache daemon?  I did a little poking with Shodan (https://www.shodan.io/) and it had identified 100k or so systems.  Mostly in cloud provider spaces as you would expect.

So how did this happen?  I don't know much about memcache myself, so I think that makes me particulary qualified to take a stab at the answer.  Since I don't know what it is, or how to install it, I turn to google to help me out.  Buried in the wiki for memcached (https://github.com/memcached/memcached/wiki/ConfiguringServer) under the networking section one might read:

"By default memcached listens on TCP and UDP ports, both 11211. -l allows you to bind to specific interfaces or IP addresses. Memcached does not spend much, if any, effort in ensuring its defensibility from random internet connections. So you must not expose memcached directly to the internet, or otherwise any untrusted users. Using SASL authentication here helps, but should not be totally trusted."
 

But who has time for instructions, right?  What's Google tell me to do?  The top answer gives us:

Change the memcached configuration setting for CACHESIZE and -l :

  1. Open /etc/memcached.conf in a text editor.
  2. Locate the -m parameter.
  3. Change its value to at least 1GB.
  4. Locate the -l parameter.
  5. Change its value to 127.0.0.1 or localhost.
  6. Save your changes to memcached.conf and exit the text editor.
  7. Restart memcached. 

Source: http://devdocs.magento.com/guides/v2.1/config-guide/memcache/memcache_ubuntu.html 

Hey, that looks right.  What about Digital Ocean's instructions (they have a lot of these open)?  https://www.digitalocean.com/community/tutorials/how-to-install-and-use-memcache-on-ubuntu-14-04

It has this as how it's started:

/usr/bin/memcached -m 64 -p 11211 -u memcache -l 127.0.0.1

That looks fine too.  So it's not a rash of bad instructions on the Internet.

Okay, how about stackoverflow/serverfault?  In the top question about memcache (https://serverfault.com/questions/347621/memcache-basic-configuration) we find an illuminating answer:

"The -l option regulates to which interfaces memcached will listen to, not which hosts will be served."
 

I wonder how many web-admins thought they needed to open up access to all of their clients. 

Kevin Liston

292 Posts
ISC Handler

Sign Up for Free or Log In to start participating in the conversation!