Etherplex

I’ve decided to try out audio blogging for fun.  I recently got a Google TV unit from Logitech, and these are some of my initial thoughts about it.

Audio

The “Google-Verizon Deal“, as everyone seems to be calling it, is not a deal at all, but rather a proposal whose intended audience is primarily the government and perhaps other corporate stakeholders in the internet service provider regulation.  The reactions to the proposal have been overwhelmingly negative, and while I understand why, I thought this might be a good time to engage in discussion about the current understanding of where we are, and where the proposal would take us.

A Working Definition of “Net Neutrality”

It is important to outline exactly where we stand on net neutrality.  First, a working definition: net neutrality is intended to ensure that internet service providers do not prioritize (or throttle) traffic based on the owner of the traffic or the specific application of that traffic.  In general, this is intended to allow actions like optimizing VoIP traffic to minimize latency (so your internet calls don’t have a high delay because someone next door is torrenting the latest Ubuntu ISO).  On the other hand, it is intended to prevent the prioritization of one VoIP service (or client) over another: Vonage, TMobile @ Home and Skype should all receive the same priority because they are all VoIP applications. 

As a good friend of mine said when we discussed it: if you own a website, net neutrality dictates that you shouldn’t be able to go to your customers’ ISPs and buy better access to them.

Do We Even Have Net Neutrality?

Many people believe neutrality is a good idea, including Google.  But there seems to be a misunderstanding on where we currently stand with respect to net neutrality.  Much of the coverage seems to indicate that we somehow already have network neutrality.  We don’t.  We never have.  It is well known that various ISPs reduce bandwidth to particular sites like YouTube because of disproportionate traffic consumption coming from those sites.  Bittorrent has been a whipping boy as well; Comcast actually won in court after throttling consumers’ bandwidth if they used Bittorrent.  So, this concept that seems to floating around that we all live in a net neutral world is false.  NPR’s coverage of the “deal”, as they insist on calling it, is but one example of this misunderstanding:

In a move that could change the landscape of the Internet, Google and Verizon are reportedly close to a deal that would allow Internet service providers to prioritize traffic from certain websites.

Not only does the article call it a “deal” (it is not at all clear that there is any deal) but the phrase “that would allow Internet service providers to prioritize traffic from certain websites” makes it sound like they currently can’t.  But NPR is not the only reputable news source to get it wrong.  In the article from the New York Times covering Comcast’s win in court, they said:

A federal appeals court ruled on Tuesday that regulators had limited power over Web traffic under current law. The decision will allow Internet service companies to block or slow specific sites and charge video sites like YouTube to deliver their content faster to users.

Note the phrasing: “The decision will allow Internet service companies to block or slow specific sites”.  Embedded in that phrasing is the notion that prior to this decision, companies could not do this.  But, there wouldn’t have ever been a court case if that were true.  The reason Comcast was brought to court was because they engaged in this activity.  The court essentially clarified the current situation on net neutrality: there isn’t any.

I don’t know how to make this any clearer, so I’ll just say it: we do not currently live in a world in which net neutrality is a reality.  Net neutrality is an idea that a lot of people agree with (like world peace), but it isn’t real, either in the wired networks that run to our homes and businesses or the wireless networks that we use on our phones.  The only news coverage I’ve been able to find that actually acknowledges this reality is the article on TorrentFreak discussing this exact issue:

Seriously, we don’t understand where all the hatred towards Google comes from. The proposal is not going to destroy Net Neutrality, simply because Net Neutrality doesn’t exist yet.

My sentiments exactly.

But, that doesn’t stop the vitriol from spewing forth from publications like eWeek, The Huffington Post, The Last Watchdog, and even the very respectable Ars Technica, who dubbed the proposal a “flip-flop”.

A Step Towards a Common Understanding

The ISPs’ Position

So, it is in this context that we come to the Google-Verizon proposal.  After a decade of stagnation on the net neutrality front (or even a regression, if you count Comcast’s win) both ISPs and websites want some certainty in a highly uncertain landscape.  On one hand, many internet service providers also engage in other businesses, and may wish to provide “private” services alongside the broadband internet service.  This usually includes legacy information dissemination mechanisms, most notably television service, but also other services like VoIP.

Let’s take an example using VoIP, which would be a bundled service in a package like AT&T’s UVerse.  Back in the old days when VoIP was newer, I remember having trouble with Vonage; if I were downloading a large file (let’s say I was executing a large apt-get install command and my Linux computer) and my friend called, I would answer the phone and they would immediately complain: “Hey, Rick, I can’t hear you.  What’s going on?”  This was a real problem, and while it’s gotten better (as bandwidth has increased), there is an idea that providers should be able to identify latency-sensitive types of traffic and prioritize them accordingly.  Hopefully that is something we can agree on, because it has nothing to do with net neutrality, because it isn’t about service providers buying better access to customers.

Here’s where it gets interesting.  If you do decide to buy AT&T UVerse, should AT&T be able to bundle in phone service and make guarantees that the phone service won’t have choppy audio, even when you saturate your entire bandwidth with a torrent you’re downloading?  What about TV?  Should AT&T be able to guarantee that your TV channels still come in clearly and without horrendous compression artififacts even if someone in your home is watch Netflix for Hulu or downloading Bioshock off of Steam?  As I read the Google-Verizon proposal, this is what they mean when they talk about service providers being able to provide “differentiated services”.  Everyone should be asking “Differentiated from what, exactly?”  I think the answer is differentiated from “normal” or “public” internet.  The internet you’re using to read this post.

But let’s explore what happens if we don’t let providers do this.  Consider the same example.  Currently, cable providers already carve out a chunk of the data coming over the wire for channel information, but no one really cares, because it operates independently of the piece carved out for the “public” internet.  The provider is sending prioritized data, but since people don’t think of it as being on their internet connection, they don’t seem to mind.  Now consider the case in which cable providers wanted to switch to an IP-based distribution mechanism, opening up the rest of that coax cable for internet use, but prioritizing their own video streams so that if someone was watching TV, they would be guaranteed a good picture.  Even though this scenario would be a win for all players, it would violate net neutrality in its purest form.  Enforcing pure net neutrality in that case would hinder progress.  So, Google and Verizon introduced the concept of these “differentiated services” that I imagine would include things like VoIP and TV service.

But that’s not the whole story.

The Web Sites’ Position

On the other side of the debate is the web sites that are engaging in significant research and development to introduce new, competitive offerings that challenge traditional business models.  Netflix, Youtube and Hulu come to mind.  These websites have to deliver their services over a wire that is owned by the very companies they are competing with.  In the absence of network neutrality, that poses a very risky proposition.  Since businesses like these have enormous up-front investments, there is a big risk that as soon as they make those investments and become relevant, their competitors like Verizon, Comcast and other digital media providers will drop the hammer on them.  So, in the interest of maintaining innovation on the web, it is vital that these companies have some kind of assurance that their competitors won’t unfairly deprive them of their right to compete on the basis of their products’ and services’ merit.  That assurance is network neutrality.  The need for some kind of regulation in this regard is quite clear.  But why?

The one remaining piece of the puzzle lies in the fact that like so many utilities (water, power, telephone), wired internet providers have a virtual monopoly.  The right way to handle all this would normally be through the free market; if an ISP ruined your experience watching Hulu so they could push you their own video streams, you would simply switch to another that didn’t do that.  But that’s not possible, since providers have a stranglehold on the geographic location in which you live.  Unless you’re willing to pick up and move so you can get better internet access, the lack of competition is a real problem.  That’s the motivation for government regulation.

Wrapup

So where does that leave us?

Well, given that we don’t have net neutrality in either wired or wireless forms yet, and given that we all agree that it would be a good thing to have, I have to say that while I’m a bit disappointed by portions of the current proposals (both those coming from the FCC and those drafted by Google-Verizon), after years of making no progress, it is good to finally get some movement on this.  Getting formal, public agreement that public internet should operate in a neutral way is a big step: a step we haven’t seen before.

I think it is clear that we will need wireless net neutrality as well, although I agree that we’re still figuring out exactly what wireless networks will look like.  The market is still changing very fast in the wireless space, with very distruptive changes (like flat rate, capped or metered data plans) still being figured out.  As phones become more like mini-laptops and laptops replace laptops, the importance of wireless is undisputed.  How exactly to manage the complexity of a such a dynamic market is another question which Google and Verizon have rightfully (in my humble opinion) decided to postpone resolution of.

One of my first projects to push to GitHub was my Emacs initialization files. Emacs is precisely as powerful as Neal Stephenson suggests, but I find that people are unwilling to use it because they find it difficult to configure to do what they want. I thought that I might help by not only making my Emacs configuration public, but by using Org-Mode and Org-Babel to make my Emacs initialization a literate program — one that doesn’t just have comments, but that is itself an executable document meant to be read by humans.

I completed the port this weekend, posted it to GitHub, and am making a weave of the file available as a static page here on Etherplex. I’ll be updating it as it changes, and hope someone finds it to be useful.

I’m a big fan of distributed version control for a lot of reasons, but sites like GitHub and BitBucket really strike at the heart of the issue: real coding should be a social phenomenon, if for no other reason than code is really about sharing ideas, and code should be addressed to human beings rather than to a computer.

The self-hosted Mercurial repositories I used to host at http://code.etherplex.org are now gone, and the subdomain is now a redirect to my GitHub profile, where I’ll be migrating all my active and psuedo-active coding efforts. While I tend to prefer Mercurial, the community at large has spoken, and I’m happy to use Git for the majority of my work if it helps engage the community. So, head on over to Code Etherplex and check out what I’ve posted. Not much yet, but I’ll be pushing a lot of my in-progress work to GitHub in the next couple of weeks.

Computers and Connections Are Unreliable

Solution: Federated Services

The Trend Away From Federation

What We’ve Lost, And How We Can Get It Back

The Three Critical Questions

My Gambit Scheme code is taking 500 times longer than the equivalent C program, and I don’t know why.

So, there’s a really horrible way to spend processor time to calculate pi that involves taking the unit square and inscribing a circle inside it with radius 1/2.  You then partition the square into N partitions on each axis (the code below uses 12,000).  This gives you a unit square and places it on a grid with 12,000 on each side, for a total number of grid squares of 12,000^2 = 144,000,000.  The algorithm then iterates through the center of each square and checks to see if it is inside of the circle.  It sums the number of points inside the circle, divides it my the total number of points, and multiplies by 4, which produces an approximation of pi.

For those already initiated, this is like a deterministic Monte Carlo approach.

Here’s the C implementation (from the professor of my Architecture of Parallel Computers course):

#include <stdio.h>

int main(int argc, char*argv[])

{
  int numPartitions = 12000;
  int circleCount = 0;
  double interval = 0, pi = 0;

  int i = 0, j = 0;

  interval = 1.0/(double)numPartitions;
  for (i = 0; i < numPartitions; i++) {

    double a = (i + .5)*interval;
    for (j = 0; j < numPartitions; j++) {
      double b = (j + .5)*interval;

      if ((a*a + b*b) <= 1) circleCount++;
    }
  }

  pi = (double)(4*circleCount)/(numPartitions * numPartitions);

  printf("Estimate of pi is: %10.8lf\n", pi);

  return 0;
}

Here’s the (mostly equivalent, I hope) Scheme implementation:

(define (main args)
  (let* ((partitions 12000)
         (increment (expt partitions -1))
         (count 0))

    (do ((y 0 (+ 1 y)))
        ((= partitions y) (* 4 (/ count (* partitions partitions))))
      (let ((b (* increment (+ y .5))))
        (do ((x 0 (+ 1 x)))

            ((= partitions x))
          (let ((a (* increment (+ x .5))))
            (if (< (+ (* a a) (* b b)) 1)
                (set! count (+ 1 count)))))))))

(display (string-append (number->string (exact->inexact (main '()))) "\n"))

OK, so, running these on my three-year-old Intel Core 2 Duo T7200 (under Linux 2.6.31), the C version takes 1.03 seconds, consistently.  The Scheme version, compiled with Gambit Scheme 4.6.0, takes about 8 minutes 19 seconds.  I am using a complied binary of the Scheme code, and Gambit was built for the machine I’m running it on, i.e. –enable-single-host was supplied to the configure script.

So, the point of this post is not really that Scheme is slow — in fact, I vastly prefer it.  Rather, I’m looking for some input from people more knowledgeable in Scheme as to what I’m doing wrong here.  I assume do is well optimized, but I’m I suffering because of the let* or nested lets?  What’s killing my performance?

I’ll try to profile it, but I’m not sure how to do that in a way that traces back to the Scheme source after it’s been compiled and linked.  The is the first time I’ve used Gambit (I used PLT in the past, and I’ve done quite a bit of work in Clojure on the JVM), so I could use advice from anyone that knows the C or Gambit environments.