Addendum (1:12 PM): Google's Larry Page and David Drummond are categorically denying that Google gives the government open-ended, back-door access to user data. This appears to confirm my speculation (for Google at least) that these firms are still tightly controlling data access by reviewing and addressing each data demand on an individual and responsible basis. And keep something in mind -- the government can use legal means to try force you to be silent about a matter, but they can't force you to lie, unless they're resorting to waterboarding and shock collars for Internet executives.

Addendum (10:05 PM): Hangout On Air video discussion of this and related topics recorded this evening (55 minutes) is now available for viewing at this link and via the embed below.

Yesterday in The Soviet Surveillance States of America we began connecting some of the dots associated with the new disclosures of the U.S. federal government's collection of telephone and Internet data.

Since the initial reports, we've now been informed by officials that they only actually look at the telephone connection "metadata" in the course of specific, targeted investigations, and that the Internet data slurping associated with PRISM is directed at foreign nationals in foreign countries (though Americans can be accidentally sucked into the system as well).

We're told by administration spokesmen and top members in Congress that this is all for our own good, presumably as are ubiquitous CCTV cameras, license plate readers, DNA swabbing of innocent persons, and all the other varied inputs (some of which we possibly don't know about) feeding to our law enforcement and intelligence agencies.

Our fearless leaders seem startled that there's such a negative reaction to these new revelations. "Calm down children, we know what's best for you!" appears to be the common refrain.

What they forget -- or more likely are conveniently ignoring -- is that we Americans are a historically rather strange breed when it comes to an innate distrust of government. Frequently these concerns go completely overboard, but when government actually does play into the hands of the conspiracy theorists it does nobody any good at all. (On the other hand, we continue to have evidence that our government is so leaky that keeping a really big secret for long is an intense challenge.)

If you really want to incur the ire of most honest Americans, treat them all like they're criminal or terrorist suspects.

Now, what's really going on with PRISM? The government admits that the program exists, but says it is being "mischaracterized" in significant ways (always a risk with secret projects sucking up information about your citizens' personal lives). The Internet firms named in the leaked documents are denying that they have provided "back doors" to the government for data access.

Who is telling the truth?

Likely both. Based on previous information and the new leaks, we can make some pretty logical guesses about the actual shape of all this.

Here's my take.

First, I believe it's reasonable to assume that significant targeted use of DPI -- Deep Packet Inspection -- is in place, most or all of it outside the control (or even perhaps knowledge) of major Internet sites (but quite possibly associated specifically with major ISPs and backbone providers).

Just as I doubt that "all phone calls are being recorded," I doubt that a mass collection of non-targeted Internet data is going on. Not only would this be technically enormously difficult when you consider traffic patterns and volumes, but would not likely be useful from an analysis standpoint compared with more careful targeting of specific communications, even with the improvements in analysis tools we are aware of (and/or can speculate exist in the shadows).

We do know for certain that the government has become very insistent on two fronts -- wanting virtually instantaneous access to specific stored and real-time user data on demand, and getting it in the clear (that is, unencrypted).

So long as most people don't bother to encrypt their email and other data the latter point is largely moot. The government is mostly concerned that someday down the line ubiquitous encryption will take hold -- that is, strong encryption by default -- that would be time consuming for the spooks to crack on an independent basis.

An intriguing outline becomes clear. The government likely doesn't have "back doors" into major Internet sites that would allow government access to those sites' user data on a "willy-nilly" basis. But it does seem reasonable to assume (especially based on the historical record associated with telephony, e.g. CALEA) that the government has pressured major Internet sites to deploy the means for rapid access to specific data requests that would be mediated by gatekeepers at those firms.

That is, NSA (or whomever) would have an expedited means to present a firm with (for example) a court order or National Security Letter. If legal counsel at the firm determines that this is a valid and sufficiently narrow demand, the mechanism would be in place to immediately provide access (perhaps one-shot, perhaps ongoing for some period) to that specific data (likely related to specific user accounts).

In other words, what we're likely talking about with PRISM isn't a "back door" for rummaging around through data in an uncontrolled manner, but rather a technical and legal protocol for the government to quickly gain access to specific data under order when the firm involved agrees that the order is valid and chooses not to challenge it.

Overall, this regime would replace much slower, largely ad hoc systems for responding to data demands, with a pipeline that can provide that data to government directly -- but the firms still control the valve on that pipe and which data is permitted to flow into it, allowing the firms to fight orders that they do not consider reasonable, focused, or otherwise valid.

This kind of scenario may help to explain the seeming contradictions of what we're now hearing about PRISM, and seems to sync well with the battles over government access to user data that we already know about, and with government demands that when they do get such access, they have some way to get the data in unencrypted form.

But even if my speculation about the relatively constrained nature of PRISM is correct, the potential for government abuse of such deployed systems is still enormous.

Such surveillance environments drastically undermine our own ability to criticize similar and worse abuses by other countries. And here at home, the "you have nothing to fear from surveillance if you have nothing to hide" argument does not play well with most honest Americans. Faith in cloud computing and storage models -- which I feel are enormously important to us all in so many ways and bring with them vast benefits to consumers -- are predicated on users trusting that their cloud data will be at least as safe from government abuses as their data would be on their own local hard drives.

The rise of ubiquitous encryption will over time likely be unstoppable, and will change the face of these issues in major ways that we cannot predict with confidence.

We can, however, predict with considerable assurance that any government and any officials -- regardless of political parties -- who insist on treating the American people as suspects, as ignorant children whose personal data should be available to government prying merely at its beck and call, are ultimately helping to destroy critical underpinnings of what has made this country great.

If we continue to permit this, the ultimate fault and blame will not be with our government or our leaders, but rather with ourselves.

--Lauren--