Hacker Newsnew | past | comments | ask | show | jobs | submitlogin
ISPs more likely to throttle netizens who connect through CG-NAT: Cloudflare (theregister.com)
76 points by throw0101a 50 days ago | hide | past | favorite | 39 comments


The Register is adding very little on top of https://blog.cloudflare.com/detecting-cgn-to-reduce-collater...

Previously discussed (a bit) at https://news.ycombinator.com/item?id=45746509


The usual story is:

1. Add an IP that has been freed from another use to a CG-NAT Pool.

2. Get complaints from customers about being hard banned from things like Netflix, Sport Streaming and VPNs or other utilities.

3. Investigate, no IP reputation issues. Find some random GEO IP database that has a side business in selling lists of VPNS or other geo breakout tools. They have listed this IP for some random reason. Almost never nefarious.

4. Give it 3 weeks for the Geoip nard to update from the wrong classification (harmful) to some kind of also wrong but unharmful classification like "Datacentre"

5. Customers can stream The Witcher again. Yay.

Really while ipv6 should be a solution here, another very good solution would be the removal of such useless middlemen from the face of the earth.


As someone who read all books and played the third game: People should totally be banned from watching Netflix's Witcher.

On a more serious note, I used GeoIP when it was free and it was a godsend to reduce malicious connection attempts to my webserver without impacting 99 % of my "clients" (not paying customers).

These kind of services *are* helpful. Wehterh you should rely on them when you have millions of customers is a different story altogether.


>On a more serious note, I used GeoIP when it was free and it was a godsend to reduce malicious connection attempts to my webserver without impacting 99 % of my "clients" (not paying customers).

Yeah, I think the issue is that, if you have 30 paying customers with, lets say, a canadian ip address and a canadian credit card, and then one bloke connects from the UK by that ip address, its daft that you just blacklist the ip and take all those customers offline. Personally I would start with a polite notice to the ISP, as its likely someones running a proxy and they should theoretically be as interested to remove it as you are.


I'm talking about Indian, Pakistani, Chinese IPs. Countries I don't do business with.


What I'd give at #2 to have some kind of backdoor ISP contact to these vendors to be able to ask "what exactly the hell is going on here?".

Having to troubleshoot things like Playstation Network bans ("or is it a ban?") behind CGNAT is an interesting adventure that typically leads nowhere.


Few if any of them permit outside contact. I remember having to get a customer to escalate their PSN case after removing the GEOIP configuration because PSN didnt reflect the change immediately. Its really difficult from a customer service perspective, because the error messages tend to pin the blame on the ISP, despite it wholly being a circumstance of the content owner.

You do get lucky, IIRC theres a few blog posts where people go into detail about how to contact some providers, but I found that some of them rotated those emails pretty regularly.


> Because CGNAT is more prominent, and more heavily used, in Africa and Asia […]

Isn’t essentially the entire US on CG-NAT for IPv4 on mobile data?

I’ve also had DOCSIS connections, i.e., fixed lines, with only CG-NAT in Europe years ago.


I wonder if that's not very visible in Cloudflare's data because those mobile devices will likely use IPv6 to connect to Cloudflare-hosted sites.


That’s what I was thinking. Anyone coming from Cloudflare will end up getting there via IPv6.


I use cloudflare to make my weather station available over T-Mobile. They don’t filter inbound ipv6 on regular phone lines (they do for TMHI) so you can host a simple page on ipv6, only set the AAAA record in cloudflare, and they will proxy it for ipv4 users so I can ignore being CGNAT’d for ipv4. Make sure if you do this setup with a tool like ddclient to keep the record current as T-mobile rotates ipv6 frequently


Well, what I mean to say was “anyone coming from Verizon will get to Cloudflare via IPv6”.

My current endpoint lacks IPv6, so I use Cloudflare so IPv6 clients can get to it. Verizon’s IPv6 is noticeably faster than their IPv4 CG-NAT.


mobile devices dont get ip6 do they? last i looked my cheapo gateway only provided v4 cgnat


Phones have been on IPv6 for years.


> Isn’t essentially the entire US on CG-NAT for IPv4 on mobile data?

T-Mobile, for one, has had their handsets IPv6-only for a few years, so if your Android/iOS does a DNS lookup and gets an AAAA record back, it will skip CG-NAT. T-Mobile presenting at NANOG in 2018 on IPv6:

* https://www.youtube.com/watch?v=d6oBCYHzrTA

And Rocky Mountain IPv6 Taskforce in 2017:

* https://www.youtube.com/watch?v=nNMNglk_CvE

Further data:

> Chances are if you use the Internet on your smartphone, you are connecting via IPv6. According to the Internet Society’s 2018 State of IPv6 Deployment,[1] 80% of smartphones in the US on the major cellular network operators use IPv6 and major mobile networks are driving IPv6 adoption with Verizon Wireless at 84%, Sprint at 70%, T-Mobile USA at 93%, and AT&T Wireless at 57%. Plus, some mobile networks are taking the step to run IPv6-only to simplify network operations and reduce costs.

* https://www.arin.net/blog/2020/01/16/mobile-edge-of-the-inte...

Though some folks aren't happy with the implementation:

> But from my own experience, neither T-Mobile nor AT&T allows inbound traffic to the phone's IPv6 address. This negates some of the advantages of having a globally routable IPv6 address.

* https://isc.sans.edu/diary/27814


> Plus, some mobile networks are taking the step to run IPv6-only to simplify network operations and reduce costs.

I think this has also already happened. You don't really notice it as a user, because the phone itself can act as a 4-to-6 translator (similar to how DS-Lite in DOCSIS transports CG-NATted v4 traffic in v6) for both native apps doing incompatible things with literal v4 addresses and tethering clients.


In Brazil I think basically more than half of fixed broadband should be CGNAT.

Basically only one single ISP don't use CGNAT...

Would be interesting if Cloudflare could give this info!


For those of us who don't have many options other than satellite internet, it generally uses CG-NAT, specifically Starlink.


AFAIK all mobile networks use NAT unless you pay a lot more for a special service with a public static IP.


I’ve had IPv4 CG-NAT on mobile LTE since ever and for a decade on residential cable in europe. Cloudflare is being lazy, Google too. I get served “Captacha’s” at least 10 times a day.

Even when i have IPv6 assigned, iOS and macOS seem to prefer A TXT RR and proceed just using IPv4 almost always. On LAN mDNS link-local IPv6 is always prefered.


They need to come up with an ip solution that is useful enough that people actually want to upgrade to it.

When you compare it to other technologies like https, tls1.3, unicode, 5g cellular, wifi 6, wifi 5 or bluetooth versions, etc. It’s clear that ipv6 adoption is not what it should be if they launched a protocol with clearer benefits to the end user.


> It’s clear that ipv6 adoption is not what it should be if they launched a protocol with clearer benefits to the end user.

The "end user" has no idea about TLS 1.3 or many other things. It's the techies that work behind the scenes that make the changes 'on behalf' of everyone else.

And IPv6 traffic is, according to Google, the majority of traffic it sees in many countries (including the US at >52%):

* https://www.google.com/intl/en/ipv6/statistics.html#tab=per-...

The 'real' holdouts are enterprise companies and corporate networks as evidenced by the fact that IPv6 usage goes up on weekends (i.e., when most people aren't at work on said corporate networks). See also:

> Chances are if you use the Internet on your smartphone, you are connecting via IPv6. According to the Internet Society’s 2018 State of IPv6 Deployment,[1] 80% of smartphones in the US on the major cellular network operators use IPv6 and major mobile networks are driving IPv6 adoption with Verizon Wireless at 84%, Sprint at 70%, T-Mobile USA at 93%, and AT&T Wireless at 57%. Plus, some mobile networks are taking the step to run IPv6-only to simplify network operations and reduce costs.

* https://www.arin.net/blog/2020/01/16/mobile-edge-of-the-inte...

Being able to connect your smartphone to the Internet seems like a clear benefit to the end user IMHO. Would hate to see what every mobile phone being behind CG-NAT would be like.


I think smartphones are a special case, because they generally cannot run public facing services that open up ports and are specifically designed to be hardened for ipv6 and designed to work in conjunction with the carrier's ipv6 firewall/network.

Compare that to a home network where, printers are shared, iot devices have open ports, computers and nas share drives. IPv6 may address the needs of cellular carriers and devices, but it doesn't adequately address the needs of small local networks connecting to the internet.


> I think smartphones are a special case, because they generally cannot run public facing services that open up ports […]

Except for peer-to-peer applications, like Skype used to be originally: clients (tried to) talked directly to each other. IMHO it'd be great if we could have app(lication)s that worked like that again: less centralization.

> Compare that to a home network where, printers are shared, iot devices have open ports, computers and nas share drives.

Off the top of my head: your CPE/home router has an internal CA; you tell a local app(lication) to 'connect to' the CA and get a certificate (ACME, SCEP, etc); your home IoTs/NAS/etc also connect to the CA and get certificates; so all your personal devices have a root of trust. You 'bookmark' the IPv6 address of your printer/NAS/whatever. When you are away from home you want to connect to (e.g.) NAS, so you tell your smartphone to connect to it, and it knows the IPv6 address, but how can the CPE or NAS know that this random IP that is trying to connect is trusted?

Well, it uses IPsec negotiation and sends the X.509 certificate, and the other end of the tunnel (NAS) sees that the cert is trusted, and so allows the tunnel to be connected. If a connection attempt is made with an untrusted certificate the negotiation fails.

Of course if you don't want your NAS to allow external connections you don't enable the feature (default: off), and so it never punches a hole (PCP, UPnP IGD). And given a IPv6 subnet is /64 (the equivalent of four billion IPv4 Internets), good luck trying to scan that address space (and it is generally recommended to give residential users a /56).

As it stands now, you have to have third party tunnels (Wiregaurd, Tailscale, etc) and 'extra' protocols on top of IP (often dynamic DNS as well) to do the above, whereas with IPv6 universal connectivity can become part of the 'base network' architecture.


All that I’m saying is that the marketplace is not convinced that IPv6 works better for a local network than IPv4 with NAT and DHCP.

It’s more secure and more private for users that aren’t security or network engineers.

My prosumer $1,000 networking setup isn’t sufficient to run certificates and IPv6 firewall the way you’ve described and I don’t feel qualified to setup what you are suggesting. I can get a $50 router and setup a reasonably secure IPv4 with NAT and DHCP in 15 minutes.


> My prosumer $1,000 networking setup isn’t sufficient to run certificates and IPv6 firewall the way you’ve described and I don’t feel qualified to setup what you are suggesting. I can get a $50 router and setup a reasonably secure IPv4 with NAT and DHCP in 15 minutes.

My several-year-old Asus AC68 does IPv6 (my previous ISP had it), (Open)VPNing:

* https://www.asus.com/ca-en/support/faq/1008713/

* https://www.asus.com/support/faq/1049180/

and Let's Encrypt:

* https://www.asus.com/us/support/faq/1034294/

Just because you're not qualified does not mean it wouldn't be handy to those who are, but not-high IPv6 adoption is hampering them. Further, some of this would currently have to be done manually (mostly the cert provisioning: IPsec/IKEv2 can otherwise be fairly automated), but if there was more uptake there's no reason it couldn't be more automatic.


It's the Internet protocol. End users are not supposed to interact with it directly.

What exactly would replace IPv6? It's just an implementation detail, but an important one if you want to make the rest of the stack suck less.


Yeah, IPv6 is heavily tuned to the needs of the large-scale network operators, and is actively worse for the regular user and small networks.

From user/small admin standpoint, the goal is to re-use as much admin knowledge as possible - and what's on the wire does not really matter. So the ideal IPv4 upgrade _for users_ is IPv4 with larger addresses, but otherwise behaving identically. Ideally all the admin tooling stays the same, and the software needs changing some struct names, and tweaking IP regex. And sure, it'll all be different on the wire and all the OS'es need to be upgraded - but that is not a problem, consumer OS'es live only for a few years anyway.

From large network operator standpoint, the goal is improve efficiency of the huge networks. So lets eliminate NAT everywhere, completely redo host addressing, get rid of DHCP, and so on - redesign everything from scratch so it's "better". Sure, it's a huge learning curve but they have departments full of network engineers, they can do it. They are not some part-time sysadmins who just want their network to keep functioning.


It does not behave identically.

I have MultiWAN on OPNsense. My PC IP is always 192.168.0.12. My router decides which upstream it should go. If I go full IPv6, router should derive double IPv6 from both WANs and if main upstream goes down, stop advertising IPv6 from main upstream. Or stop advertising gateway. I don't know what is the right IPv6 way of doing MultiWAN.

Not only PC may change IP, but also servers. Legacy IPv4 DNS can be extended to IPv6, but that mechanical action is not flexible enough. With IPv6 we need to be able to mass replace IPv6 /64 prefix leaving all suffixes intact. We probably need /64 prefix alias system. Software is not prepared for this. In IPv4 SNAT and DNAT were being these "aliases". If NAT is not an option anymore, then DNS must step in.

For many server software it just not possible to listen on multiple IPv6 address. Last time I tried MySQL, it just could not listen on multiple addresses. I could not make it listen on IPv4 and IPv6, specifying two addresses. MySQL server wanted just one address. This address could be [::], which means all interfaces and all protocols. And Linux implements some stupid hack to accept IPv4 connections to IPv6 socket. And Windows Vista also adopted this brainrot. But this is all wrong. Servers have to learn to listen to multiple IPs. This is normal. And for good IPv6 servers should learn to not only listen on multiple IPs, most wanted multiple IPv6, but also rebind listeners on the fly. If I got disconnected from ISP, reconnected by DHCPv6, and ISP assigned another IPv6 prefix, then DynDNS should update all my zones to new /64 prefix, and all servers in my network should rebind listeners.

Or else we may abandon all that TRUE IPv6 philosophy and do SNAT in DNAT in IPv6 just like in IPv4, but with wider address space. But then again, software (another software) is not quite ready for this. Software is expecting public IPv6 address to be just reachable. And private IPv6 address to be just unreachable.


That's illustrates my point well - the "TRUE IPv6" philosophy is major changes in every network-facing user software.. that's why it has been 20+ years and it's not done yet.

And the justification of "Software is expecting public IPv6 address to be just reachable" is super silly. You have to be crazy in this day-and-age to operate without firewall. Every office, every home network should have "default-deny" policy from the internet. So no, your software should not expect to be reachable even once IPv6 adoption is complete.


Isn’t that just a correlation with crappy internet?

If you’re on non cg Nat it’s likely a pretty high end connection


I grew up in Australia, and have spent a fair bit of time in India for over a decade, and now live in India (1⅓ years).

Every ISP that I have experienced, mobile and broadband, is using CGNAT. The easiest way I’ve seen this on broadband is https://iknowwhatyoudownload.com/ showing several movie downloads per day.

Cloudflare isn’t the only problem, but they are the worst, probably by dint of popularity. I get blocked outright occasionally (presented dishonestly as because my request matched attack patterns due to things like SQL injection in query string parameters, when I’m actually just trying to load any regular page), and blocked with hCAPTCHA frequently (normally presented dishonestly with their stock page as “example.com needs to review the security of your connection before proceeding”, though a few like blender.org customise it). It’s draining.

In Cloudflare’s actual article, they claim their bot detection to be resilient to CGNAT <https://blog.cloudflare.com/detecting-cgn-to-reduce-collater...>. Frankly, if it is so, I wonder if they just have a rule that amounts to “is user in India”. I definitely feel prejudged and discriminated against. I am idly curious if leasing a static IP from my ISP would help anything, in the short or long term.

In Australia, I think I experienced Cloudflare’s blocking page once in my life, and no others.


>Every ISP that I have experienced, mobile and broadband, is using CGNAT. The easiest way I’ve seen this on broadband is https://iknowwhatyoudownload.com/ showing several movie downloads per day.

From memory, APNIC was handing out a /22 to every new member, then a /23, then a /23 worth. Now it asks you to submit a plan on how you would allocate a /23 if you received those ips.


>and blocked with hCAPTCHA frequently (normally presented dishonestly with their stock page as “example.com needs to review the security of your connection before proceeding”

Isn't that from cloudflare, not hcaptcha?


I meant, in the context: blocked by Cloudflare with hCAPTCHA recourse. But as I consider it more carefully, I don’t think they don’t use hCAPTCHA in their challenges any more. They used reCAPTCHA at first, moved to hCAPTCHA around 2020, then they made their own thing Turnstile in ~2022 and migrated challenges to that at probably? the same time.


It's almost impossible to edit Wikipedia on mobile networks due to ip hopping vandals on CGNAT.


The same applies to us rich folk on mobile. Not sure what the point of this article is.


I agree. A bunch of platforms use CGNAT, like Tailscale: https://tailscale.com/kb/1015/100.x-addresses


My understanding is that they use the IP range assigned for CGNAT as their private address range to avoid conflicts, but do they use CGNAT?




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: