edit: I'm OP, not the content author. I serve a media website, which is where I noticed and from where my concern stems. Comcast users should also be concerned about this.
Just scanned my logs more fully and have serious concerns.
As people have noted, this really does make requests every 5 seconds. My 404 page is currently 18KB, which means these users (who are being warned about their bandwidth) are being forced by their ISP to download extra web traffic from the site they're sitting on. For me that number is 1/3MB / minute and I'm seing users who sit around a very long time.
Also, this isn't restricted to the two metros Tuscon and Nashville people have mentioned. Here is a sample of hits I'm seeing (removing final octet from IP/hostname):
c-75-65-181-xxx.hsd1.la.comcast.net
West Monroe, LA
c-174-52-141-xxx.hsd1.ut.comcast.net
Provo, UT
c-69-137-179-xxx.hsd1.az.comcast.net
Tuscon, AZ
c-76-109-127-xxx.hsd1.fl.comcast.net
Miami, FL
cpe-72-225-230-xxx.nyc.res.rr.com
New York, NY
c-68-48-154-xxx.hsd1.md.comcast.net
Washington, DC
Just scanned my logs more fully and have serious concerns. As people have noted, this really does make requests every 5 seconds. My 404 page is currently 18KB, which means these users (who are being warned about their bandwidth) are being forced by their ISP to download extra web traffic from the site they're sitting on. For me that number is 1/3MB / minute and I'm seing users who sit around a very long time.
Also, this isn't restricted to the two metros Tuscon and Nashville people have mentioned. Here is a sample of hits I'm seeing (removing final octet from IP/hostname):
c-75-65-181-xxx.hsd1.la.comcast.net West Monroe, LA
c-174-52-141-xxx.hsd1.ut.comcast.net Provo, UT
c-69-137-179-xxx.hsd1.az.comcast.net Tuscon, AZ
c-76-109-127-xxx.hsd1.fl.comcast.net Miami, FL
cpe-72-225-230-xxx.nyc.res.rr.com New York, NY
c-68-48-154-xxx.hsd1.md.comcast.net Washington, DC
c-98-224-83-xxx.hsd1.ca.comcast.net Fresno, CA
c-66-41-214-xxx.hsd1.mn.comcast.net Minneapolis, MN
So what do we do about this?