Shilling for a living

Good lord, let’s read someone else who has no fucking clue how ISPs work, how bandwidth is purchased, what bandwidth even means. This person just could not be more clueless.

Given that the comment is reasonably well-written, I’m betting this one is actually a paid industry shill, though, on further consideration.

The person says, “For years, Netflx has been paying 3rd party internet delivery services such as Cogent and Level 3 to deliver streaming video to Netflix subscribers.”

No. Absolutely not. Netflix does use Cogent and Level 3 for their bandwidth exactly like you’d use an ISP at home, but on a much, much (much!) larger scale. Nothing is delivered directly to Netlfix subscribers. That is just not how the internet works. What is an “internet delivery service,” anyway? That phrase there is an attempt to mislead. That is not a term of art in my field — infrastructure engineering — nor in any related field.

I’ve used both Cogent and Level 3, bought loads of bandwidth from them. In fact I bought so much bandwidth from Cogent in NJ they had to run an entirely new fiber line. Cogent and Level 3 provide Netflix’s bandwidth, just as your ISP provides yours. That’s all that’s happening.

“These companies deliver the streaming video to ISPs such as Comcast, Verizon, ATT, et al, through commercial agreements based on the amount of video transmitted back and forth between the ISPs and the 3rd party delivery services, i.e: you take my data transmissions, I’ll take yours and we will call it even. As Netflix became more popular, the flow of video became more one sided – from Netflix and the 3rd party services to the ISPs. The ISPs want compensation for the imbalanced data exchange (your “careful engineering” point).”

I can tell this guy(?) is a shill as there is enough truth mixed with the falsehoods that he is obviously technically knowledgeable to some extent. That’s why I suspect the person is actually in industry employ.

Let’s be clear here, though. What Shill is talking about is peering agreements, which are indeed made between backbone providers. However, Comcast is NOT and never has been a backbone provider. It is a last mile provider. Peering agreements are not made between last mile providers and backbone providers because bandwidth here is almost always asymmetrical.

And why is this?

It’s because almost anyone on any ISP (except for a very few) have much more downstream than upstream bandwidth allocation. For instance, Comcast’s 105mbs service only offers 10Mbs upstream, which is less than 1/10 as fast.

Traffic from a residential ISP can’t help but BE asymmetrical since not only is upstream bandwidth much lower than downstream. In fact most of the time — to force us to be good little consumers — you can’t even run servers from your home per the TOS. This traffic then will always, always be asymmetrical and peering agreements which presume roughly symmetrical bandwidth would never work.

What really happened is that Netflix is in competition with Comcast, and more and more of Comcast’s customers started requesting traffic of content hosted by Netflix. Comcast deliberately degraded this content by (I’m guessing here, I have no details of internal Comcast network management) reducing the number of agg ports the Level 3 or Cogent bandwidth flows through at some crucial routing center. (That’s the way I’d do it, anyway, if I were an evil ISP!)

That way, they can argue that they aren’t actually prioritizing traffic, the “hole” just isn’t big enough for the bandwidth. Which is correct, as they’ve made it too small, all without formal traffic management of any sort.

Traditionally, what a residential ISP would do if their customers started requesting more traffic than could be delivered would be to upgrade their infrastructure. Say Comcast customers are requesting 1Tbs at peak times and most of this flows through Cogent as its backbone provider. Comcast would say to Cogent, “This link is getting saturated. Can we drop another fiber from our core router to yours in Poughkeepsie?”

Often these crucial internet routing points are in the same datacenters, by the way, just for this ease of functioning and upgrading as needed.

And Cogent would say, “OK, it’ll be ready in a week. Please have the datacenter plug it into Port 29 on patch panel 37G.” Then a new fiber would go across the datacenter from Cogent’s racks to Comcast’s racks, and 1Tbs available traffic would become 2Tbs just like that.

In the case of Netflix, though, this wasn’t even necessary. Comcast was offered and refused Netfix’s caching boxes that would’ve made the traffic free to them essentially. Thus Comcast chose to punish their own customers — which seems a bit of modus operandi for them.

In other words, this whole problem could’ve been solved if Comcast had accepted for free a few very expensive servers. It would not even have required the traditional infrastructure upgrades, in other words.

“As the demand has increased, the quality of the 3rd party delivery has declined. Netflix has allowed their customers to get crappy service from the 3rd party delivery services in order to put pressure on the ISPs to do a delivery deal directly with Netflix.”

This is not how the internet fucking works!

Netflix has absolutely no desire to do a deal with this ISPs directly. None. That is harmful to them and always will be. These “3rd party delivery services” Shill is talking about are Level 3, Cogent, et al., and they always provide exactly as much bandwidth you ask for. It is their job, and they honestly do it pretty well. They are very good at it. If you order 1Gbs, you get it. Etc. No, it is Comcast that did not upgrade its infrastructure to accommodate increased demand from customers (or deliberately degraded it).

“That strategy didn’t work with Comcast because they are too big to ignore. There is nothing ‘miraculous’ about quality of the Netflix transmission on Comcast improving after they reached an agreement with the cable company. Instead of paying a Cogent or Level 3 to transport their streaming video to Comcast for delivery to Comcast customers, Netflix has cut out the middleman (whose technology hasn’t kept pace with the demand for Netflix video) and will now pay Comcast directly to cache their “instant video” for more immediate access by Netflix subscribers via Comcast.”

This so-called strategy of delivering content over the internet to customers who request it is just how the internet works. Treating it as some strange state of affairs is really bizarre and is yet another sign of the commenter’s complete shill status. The “middlemen” here are the backbone providers. Trust me, their technology has more than kept pace and is far beyond anything Comcast has. The infrastructure failure here was Comcast’s, not Cogent’s or Level 3’s. They are more than happy to give you just as much bandwidth as you ask for.

What Netflix offered to Comcast for free was those aforementioned caching boxes so that Comcast would not have to upgrade their infrastructure (though they should have done so anyway). Comcast said no, even though it would’ve benefited them. Instead, they chose to extort money from their competition, harm their own customers and weaken the underpinnings of the internet all at the same time.

“If only Netflix would negotiate similar deals with every ISP. Instead, Netflix sees an opportunity for a dishonest bait and switch. They want to stop paying the middlemen for transport of Netflix videos, give the videos directly to all ISPs, but not pay for this service.”

Netfix would in fact much rather pay the “middlemen” than be extorted by ISPs. You can tell Shill is a shill because of how cleverly he or she turns around who is the greedy and at fault one here. Classic shill technique, that reversal. All Netflix wanted was for its content to be delivered. Comcast refused to provide the infrastructure for it to be delivered, and refused to provide traditionally-used caching boxes to make the infrastructure upgrades unnecessary.

And then apparently paid shills to lie about it.

Please note that I am glossing over or ignoring completely many, many technical details here. There’s a reason 1,200 page books are written about this stuff. However, nothing I’ve said up there is incorrect, just much-simplified.

*Note that it’s a bit more complicated than this. I know because I’ve done it a lot, though not with Comcast.┬áIf anyone really wants to know the details of BGP, OSPF or how to do an ISP interconnect at a commercial data center, I’d write it, but my blog audience would go from 6 to 2.