Mar 24

Shilling for a living

Good lord, let’s read someone else who has no fucking clue how ISPs work, how bandwidth is purchased, what bandwidth even means. This person just could not be more clueless.

Given that the comment is reasonably well-written, I’m betting this one is actually a paid industry shill, though, on further consideration.

The person says, “For years, Netflx has been paying 3rd party internet delivery services such as Cogent and Level 3 to deliver streaming video to Netflix subscribers.”

No. Absolutely not. Netflix does use Cogent and Level 3 for their bandwidth exactly like you’d use an ISP at home, but on a much, much (much!) larger scale. Nothing is delivered directly to Netlfix subscribers. That is just not how the internet works. What is an “internet delivery service,” anyway? That phrase there is an attempt to mislead. That is not a term of art in my field — infrastructure engineering — nor in any related field.

I’ve used both Cogent and Level 3, bought loads of bandwidth from them. In fact I bought so much bandwidth from Cogent in NJ they had to run an entirely new fiber line. Cogent and Level 3 provide Netflix’s bandwidth, just as your ISP provides yours. That’s all that’s happening.

“These companies deliver the streaming video to ISPs such as Comcast, Verizon, ATT, et al, through commercial agreements based on the amount of video transmitted back and forth between the ISPs and the 3rd party delivery services, i.e: you take my data transmissions, I’ll take yours and we will call it even. As Netflix became more popular, the flow of video became more one sided – from Netflix and the 3rd party services to the ISPs. The ISPs want compensation for the imbalanced data exchange (your “careful engineering” point).”

I can tell this guy(?) is a shill as there is enough truth mixed with the falsehoods that he is obviously technically knowledgeable to some extent. That’s why I suspect the person is actually in industry employ.

Let’s be clear here, though. What Shill is talking about is peering agreements, which are indeed made between backbone providers. However, Comcast is NOT and never has been a backbone provider. It is a last mile provider. Peering agreements are not made between last mile providers and backbone providers because bandwidth here is almost always asymmetrical.

And why is this?

It’s because almost anyone on any ISP (except for a very few) have much more downstream than upstream bandwidth allocation. For instance, Comcast’s 105mbs service only offers 10Mbs upstream, which is less than 1/10 as fast.

Traffic from a residential ISP can’t help but BE asymmetrical since not only is upstream bandwidth much lower than downstream. In fact most of the time — to force us to be good little consumers — you can’t even run servers from your home per the TOS. This traffic then will always, always be asymmetrical and peering agreements which presume roughly symmetrical bandwidth would never work.

What really happened is that Netflix is in competition with Comcast, and more and more of Comcast’s customers started requesting traffic of content hosted by Netflix. Comcast deliberately degraded this content by (I’m guessing here, I have no details of internal Comcast network management) reducing the number of agg ports the Level 3 or Cogent bandwidth flows through at some crucial routing center. (That’s the way I’d do it, anyway, if I were an evil ISP!)

That way, they can argue that they aren’t actually prioritizing traffic, the “hole” just isn’t big enough for the bandwidth. Which is correct, as they’ve made it too small, all without formal traffic management of any sort.

Traditionally, what a residential ISP would do if their customers started requesting more traffic than could be delivered would be to upgrade their infrastructure. Say Comcast customers are requesting 1Tbs at peak times and most of this flows through Cogent as its backbone provider. Comcast would say to Cogent, “This link is getting saturated. Can we drop another fiber from our core router to yours in Poughkeepsie?”

Often these crucial internet routing points are in the same datacenters, by the way, just for this ease of functioning and upgrading as needed.

And Cogent would say, “OK, it’ll be ready in a week. Please have the datacenter plug it into Port 29 on patch panel 37G.” Then a new fiber would go across the datacenter from Cogent’s racks to Comcast’s racks, and 1Tbs available traffic would become 2Tbs just like that.

In the case of Netflix, though, this wasn’t even necessary. Comcast was offered and refused Netfix’s caching boxes that would’ve made the traffic free to them essentially. Thus Comcast chose to punish their own customers — which seems a bit of modus operandi for them.

In other words, this whole problem could’ve been solved if Comcast had accepted for free a few very expensive servers. It would not even have required the traditional infrastructure upgrades, in other words.

“As the demand has increased, the quality of the 3rd party delivery has declined. Netflix has allowed their customers to get crappy service from the 3rd party delivery services in order to put pressure on the ISPs to do a delivery deal directly with Netflix.”

This is not how the internet fucking works!

Netflix has absolutely no desire to do a deal with this ISPs directly. None. That is harmful to them and always will be. These “3rd party delivery services” Shill is talking about are Level 3, Cogent, et al., and they always provide exactly as much bandwidth you ask for. It is their job, and they honestly do it pretty well. They are very good at it. If you order 1Gbs, you get it. Etc. No, it is Comcast that did not upgrade its infrastructure to accommodate increased demand from customers (or deliberately degraded it).

“That strategy didn’t work with Comcast because they are too big to ignore. There is nothing ‘miraculous’ about quality of the Netflix transmission on Comcast improving after they reached an agreement with the cable company. Instead of paying a Cogent or Level 3 to transport their streaming video to Comcast for delivery to Comcast customers, Netflix has cut out the middleman (whose technology hasn’t kept pace with the demand for Netflix video) and will now pay Comcast directly to cache their “instant video” for more immediate access by Netflix subscribers via Comcast.”

This so-called strategy of delivering content over the internet to customers who request it is just how the internet works. Treating it as some strange state of affairs is really bizarre and is yet another sign of the commenter’s complete shill status. The “middlemen” here are the backbone providers. Trust me, their technology has more than kept pace and is far beyond anything Comcast has. The infrastructure failure here was Comcast’s, not Cogent’s or Level 3’s. They are more than happy to give you just as much bandwidth as you ask for.

What Netflix offered to Comcast for free was those aforementioned caching boxes so that Comcast would not have to upgrade their infrastructure (though they should have done so anyway). Comcast said no, even though it would’ve benefited them. Instead, they chose to extort money from their competition, harm their own customers and weaken the underpinnings of the internet all at the same time.

“If only Netflix would negotiate similar deals with every ISP. Instead, Netflix sees an opportunity for a dishonest bait and switch. They want to stop paying the middlemen for transport of Netflix videos, give the videos directly to all ISPs, but not pay for this service.”

Netfix would in fact much rather pay the “middlemen” than be extorted by ISPs. You can tell Shill is a shill because of how cleverly he or she turns around who is the greedy and at fault one here. Classic shill technique, that reversal. All Netflix wanted was for its content to be delivered. Comcast refused to provide the infrastructure for it to be delivered, and refused to provide traditionally-used caching boxes to make the infrastructure upgrades unnecessary.

And then apparently paid shills to lie about it.

Please note that I am glossing over or ignoring completely many, many technical details here. There’s a reason 1,200 page books are written about this stuff. However, nothing I’ve said up there is incorrect, just much-simplified.

*Note that it’s a bit more complicated than this. I know because I’ve done it a lot, though not with Comcast. If anyone really wants to know the details of BGP, OSPF or how to do an ISP interconnect at a commercial data center, I’d write it, but my blog audience would go from 6 to 2.


Mar 23

Got comms

Simply brilliant.

My old Informationweek editor, Mitch Wagner, once discovered some young girls holding a gossipy chat in the comments section of an old blog post of his; when he asked them what they were doing there, they told him that their school blocked all social media, so every day they picked a random blog-post somewhere on the Internet and used it as a discussion board for the day.

Reminds me of using early PDAs with infrared transmitters in high school to hold text conversations line-of-sight across the classroom. This was in the early ’90s so no one had the first clue what we were up to.

Mar 22

Rabbit warren of Wall Street sycophants

Brief prediction: though she will almost certainly run for office, if she does Elizabeth Warren will never, ever be allowed by the Democratic establishment to be president.

It just won’t happen. Can’t happen. Every large corporation in the US will oppose it, and thus behind the scenes all the Democrat power players will oppose it.

Elizabeth Warren has only a slightly better chance of being president than I do.

Mar 21

Getting a little stabby in there

The first time someone ever tried to stab me, I was fifteen years old.

That person was my mother.

I doubt she even remembers it. She was drunk, as she was almost all the time she was home. Luckily I was awake when she decided sticking a knife in my ribs was the thing to do. If I’d been asleep, well, not sure I’d be here now.

I don’t need sympathy for it. It wasn’t even traumatic to me at the time. I am just not constitutionally set up to experience trauma of that sort.

And no, that’s not some armor against experiencing negative emotions. I have just always been that way, ever since I can remember.

The night of the attempted stabbing my mother as mentioned was drunk and thus slow, so it was fairly trivial to disarm her. However I pushed her a little too hard as I was taking the knife away and she fell back into the coffee table and hit her head.

Blood fucking everywhere. People bleed more when they are drunk.

She got up and threatened to call the police, picking up the phone and starting to dial. I wasn’t about to go to jail for her or for anyone, so I jerked the phone off the wall and broke it into a million pieces on the floor.

As drunk as she was, I knew she wouldn’t be able to walk far to make any other call. This was before the era of cell phones.

After that, she tried to attack me again, so I held her immobile until she yielded.

That night, I slept in the car with the doors locked, covered in blood and with my sister screaming at me for what seemed like hours for “hurting mama.”

So that’s what life in North Florida is like. Bet it hasn’t changed much for many people who still live there.

Mar 21

La vie

Definitely in a hip hop mood today.

Her voice is even better than Lauryn Hill’s, in my opinion. I really like the Fugees’ version, but Rudimdental’s live version is better than the Fugees’ studio version. And that is pretty amazing.

Mar 19

Not improving

Internet access in the US is actually getting worse — unknown to most — as more and more ISPs push data caps onto their customers.

It doesn’t matter if you have a 100Mbs connection if you can use all of your 150GB cap in less than five hours downloading at top speed.

The reason for this retrogression is that most of the ISPs are also content providers and producers, and they are attempting to eliminate competition from Netflix, Hulu and other streaming services.

Most people — being too ignorant for their own good — will not complain until it is already too late. That 150GB cap which seems generous now won’t seem so great when in five or 10 years you are watching a 4K movie streamed over the internet, and three of those movies in a month can exceed your entire cap.

Or in 10 years one game does it. (Even now, some games are more than 40GB. That will only increase.)

In most of the rest of the world, caps for landline broadband don’t exist or are disappearing. In the US, they are becoming nearly inescapable. It will only get worse.

For this reason, many 3rd-world countries (for lack of a better term) now have considerably better and cheaper internet access than the US.

Bandwidth is just insanely cheap. It is nothing like electricity or water (which is what your ISP wants you to believe). Nothing. If a network is idle, it costs the ISP absolutely zero to move data across it. And adding more capacity these days is so cheap you can do with a million dollars (nothing to an ISP) what 10 years ago when there were no caps would’ve cost $100 million. Really.

Well, it was a nice internet while we had it.

Mar 13

Marge Piercy

I recently finished reading Marge Piercy’s Woman on the Edge of Time.

I don’t intend to write a formal, full review but it was one of the most literate and grounded novels I’ve read. It called to mind — as it would for many others, I suspect — Ken Kesey’s One Flew Over the Cuckoo’s Nest in its exploration of the inhumanity of institutionalized violence.

Since I have been consciously attempting to read more fiction by women, as I’ve noted the quality of my reading material hasn’t suffered in the least. This novel, though, stands near (but under) Jo Walton’s Among Others as among the very best I’ve read.*

Published in 1976, Woman on the Edge of Time feels much more contemporary than its publication date would suggest. In fact, Piercy accurately predicts many technologies that did not yet exist making it feel even more current than by all rights it should.

It’s rare enough that a novel takes the perspective of an overlooked, welfare-dependent woman who is past the age of being appealing to men. Even rarer is that it doesn’t snidely wink at the reader, seeming to say, “We both know that you are better than this.”

In short, Piercy’s work is highly recommended.

*If someone forced me to choose a favorite novel of all time, I’d say Among Others by Jo Walton.

Mar 08

Not very bright is Bright

Is this guy some sort of idiot? The evidence says yes.

What’s a bit strange is that people are even looking for it in the first place.

While early PCs had dumb power switches that unceremoniously cut the power to the machine, the use of electronic power buttons—buttons that can tell the operating system to perform an orderly system shutdown before killing the power—have been a feature since the introduction of the ATX specification in late 1995. Any system on the market today can be shut down by pressing the same button that you used to turn it on. On the face of it, at least, this should make the on-screen button fairly unimportant.

I can think of dozens of scenarios (not all of which I will list) that this dumbass apparently can’t imagine where having a software power switch is far better and easier than having to find the physical power button each time. To wit:

1) The machine is in a cabinet where getting to the physical power button is incovenient.

The machine is in a locked kiosk for public use.

3) The household has lots of kids and/or cats and the physical power button is disabled in software to avoid being accidentally turned off once booted.

4) The user is handicapped and is physically unable to reach the button.

5) The PC is in another room, and is remoted into.

You get the picture. This idiot Peter Bright (I guess it is opposite day, as “bright” he is not) seems to be unable to conceive of any use for an operating system, or any way or reason for using it, that he has not personally taken part in.

People like him are why I sometimes hate being associated with IT.

Mar 07

Apps for the dumb

We had the internet, but we were too stupid to keep it.

Apps aren’t the only thing killing it, but that’s a big part. I fucking hate apps. Hate using them, hate how limited they are, how slow they are and how they are designed for 3-year-olds.

But that seems to be about all people can handle. When we all go extinct, I hope some smarter species rises in our place.

Mar 04


The collapse of Jared Diamond.

Another one — along with Malcolm Gladwell and Jonah Lehrer — I thought was bullshit long before anyone else seemed to catch on to the scent. I don’t know exactly how I tell — reading scientific papers with their (justified) hedging and uncertainty, versus the stone certainty of people like Diamond helps immunize one from being duped, even if not an expert in the field.

As I am an expert in no field nor do I wish to be (it limits creativity and flexibility enormously), having reliable heuristics for judging validity and relevance is important.

There was a post on Making Light recently about how everyone has a superpower. If I had to list one, I’d say it is being able to look at a massive amount of data (whether a book, some numbers, it really doesn’t matter) and say, “That doesn’t look right.” Often without being able to articulate why. And then being right about that incorrectness at far better than the rate of chance.

Diamond, Lehrer and Gladwell immediately set off that detector in me, with Lehrer making it go off like the foghorn of a steamer ship.