This is a random observation while listening to some killer ambient music on a pair of good headphones:

We like to in hindsight attribute the success of disruptive products to practicality when in truth these products succeeded because they elicited some previously-impossible emotional experience in their user.

Here’s the crux of the epiphany: When I was eight I got one of the very first Sony Walkman’s (this is the closest pic I could find but I swear mine was even more old school). I had a bunch of teeth pulled right when this miraculous device debuted and my parents figured it would a good distraction to fill my head with music during the recovery period. This turned out to be a genius move on their part and worked really well. In spite of having awful pain from every corner of my mouth, this new incredible way to experience music trumped everything and transported me beyond the pain. Now here’s why this is relevant:

Clayton Christensen (and by proxy many others) have cited the Sony Walkman in their explanations of disruption usually saying something to this effect: “The Walkman achieved disruption because it enabled young people to listen to music out of earshot of their parents for the first time.” This is a functional/practical motivation (ie. listening privacy, facilitated quiet rebellion) and while it may partially account for its success, I would submit that there was a more fundamental emotional-based motivation: “it enabled for the first time the undeniably cool & irreproducible experience to have music originate from between one’s ears.” This is akin to experiencing dry ice, static electricity or pop rocks for the first time- it’s just freakin’ cool!

No doubt practical motivations overtake the cool factor at some point and most disruptive products with any longevity can’t subsist indefinitely on coolness alone. But in the vast majority of past product success analysis from today’s vantage point, the coolness factor gets way undervalued. The ubiquity of white earbuds now makes it difficult for us to imagine thirty years back to a time when experiencing music that originated between your ears instead of from external speakers was as untangible as anti-matter & black holes are to us today.

Anyways, there’s no call-to-action here other than to observe that our “coolness bar” is perpetually raised higher each year and it’s impossible to see those case study products via the same lens of wonderment we would have had at the time they presented. I don’t dispute Christensen’s ideas on disruption re: underserved markets, competing against non-consumption, etc. but I think we need as entrepreneurs to acknowledge the role of a more parsimonious “I just gotta have the music inside my head!” motivation in explaining the success of a product like the Walkman.

No doubt when Apple someday develops the ability to deliver any smell on demand via the appstore, we’ll all run out and purchase an iSniff because “I gotta have any smell on demand!” And years afterwards the business historians will all concoct elaborate theories about the runaway success of this product explaining how we were economically-motivated and seeking to reduce trips to flower stores.

Ok here’s a plea for any developer who knows how to write browser extensions to write one that lets me do basic spreadsheet operations right in the web page. I would pay $20 for this add-on in its most basic buggy incarnation and up to $50-75 for a pro edition depending on how well it worked. Here’s the issue:

It’s too cumbersome to ask simple questions and do basic data wrangling of tabular numbered data in web pages.

I play with data probably five times a day via various web sites (sometimes our own, sometimes ones in the wild). Here’s a practical example from right now- we’re running some email campaigns for one of our startups and I get this report:

Which is just a set of numbers and has no meaning until you can see relative %’s and how campaigns compare across iterations. I would like to be able to quickly calculate the open rate, CTR and bounce rates of each of these five campaigns. And then get average totaled across all mailings.

Now sometimes you luck out and can copy/paste the table into Excel or Numbers and do basic summing / averaging / math ops there. But it’s a crapshoot – half the time it pastes the entire table into a single column which makes it useless. You wind up w/ this:

(sorry if you haven’t seen this Spinal Tap scene that bread reference will make no sense at all).

Pasting to a desktop app makes you leave the browser and adds just enough friction to the process to where you might not ask a question of the data that you would have otherwise. Google Docs is getting us closer and their copy/paste tends to work better, but that too is still an extra step and cumbersome & flakey. The other alternative on small datasets like this is to Command-Space to open Spotlight and manually run some calculations there typing in the numbers. But alas that sucks as well.

What would be truly spectacular is a FF or Chrome extension that gave me this right in the context of the web page:

aaaand… boom:

Select. Click. Done. Two motions to get immediate insight into tabular data on web pages. Like I said, I’d pay $20 no question for the basic version and if you start adding spiffy extra spreadsheet functionality, that number goes up to $50 and beyond very quickly. This is a valuable/painful enough situation where it would be pretty easy to make me happy even with a crappy extension.

So my question is “would you pay for such an extension?” Heck, I’ll setup a Pledgebank and hire a programmer to create this if enough people want it. I think it could do miracles for startup founders in terms of wiping out the friction associated with casually asking questions of data in web pages. My hunch is some developer could give away the very most basic version and charge a grip for the professional edition similar to how iMacros has done it. Leave a comment or a tweetback if this is something you’d use.

I had a bit of an “ah-hah” moment this past Thursday evening at the Phoenix Ableton Usergroup (a free monthly group I organize for musicians who use this piece of software). Here’s the gist of what changed in my thinking: prior to Thursday if you would have asked me “what makes a great integrated development environment?” I would have said:

The toolset allows you to manifest what’s in your head with the least friction and most fidelity.

Post Thursday night, here’s my new opinion:

A tool whose design is so inevitable, irreverent, spartan or unique that the tool itself inspires creative ideas that weren’t there before its use.

A friend of mine Brandon conducted an experiment called awhile back called “month of music” where he forced himself to author a new piece of original music every day for a month using new and different instruments. He found that the introduction of new and unfamiliar tools generated musical ideas which hadn’t existed before.

Ableton is a piece of music recording software that has (for the past four months since I began using it) continued to blow my mind at each turn with untold possibilities. It’s admittedly one of the most daunting interfaces I’ve ever had to learn and reminiscent of 3D Studio Max in its complexity. But it’s one of those rare software programs that comes along, shatters the traditional paradigm and opens up a world of possibility that sends your mind reeling with ideas. For me it’s been a “Don’t know what you don’t know” advancement – the equivalent of taking off a pair of dirty sunglasses you didn’t even realize you were wearing.

If you’re a musician and have plateaued with your musical inspiration via your current multi-track DAW recording software, check out Ableton Live. The learning curve is going to feel like standing at the base of El Capitan looking up so you’ll ideally want a trainer, a user group or a bunch of time to dedicate to watching the various Youtube video tutorials out there. But once you can get over the “suck threshold” with it, it’s amazing.

I’ll write more on Ableton specifically as I become more advanced with it but think about whatever tools you’re using now and how you might shake loose some new creative inspiration by going out of your way to do things using a decidedly unfamiliar instrument.

Let me explain this convoluted title. Ignite Phoenix #9 was this past Saturday and it frigging ruled. I just happened to run into the entire Ignite organizing crew at a coffee shop in Scottsdale that actually came about via a talk at the first ever Ignite. Oh and there happened to be one of the Ignite presenters here that I got to randomly chatting with so it was basically a lot of ignition going on.

Anyways the event on Saturday was flawless in every respect (mad props to the organizing committee for continuing to improve on something that was already amazing – I’ve been to 8/9 and they’ve gotten progressively better). My only ounce of negative feedback for Jeff and team was that one of the talks was clearly just a pitch for this lady’s barter business. And hey I get it: you get a captive audience of 800 people and the temptation is to pitch your biz and advance your cause. I understand, but it makes the audience roll their eyes and basically ends up being a buzzkill.

So anyways, here’s a suggestion I want to propose: we need the equivalent of a super-budget, anonymous jury system to shame people out of trying this at future events. And now here’s an even wackier proposal for what that might look like:

What if every member of the audience got a snapple cap upon coming through the door and the moderator explained the protocol that if presenters pull shenanigans and start Amwaying the crowd, the audience is to “cap their ass” with a collective gong of popping their Snapple caps to make that annoying clicking sound?

If you know you’re going to be publicly shamed on stage when you willingly violate the presentation guidelines and slang your own stuff, you’ll either a) steer clear of this practice or b) violate it and provide some serious amusement for the audience. Either way- WIN!

Anyways, I leave this suggestion in Jeff’s capable hands and will pledge to buy a few cases of Snapple to arm the audience if they decide to adopt it. Even if they just gave the front row this duty I think it would work (but it’d be way more impressive to see 800 people capping a presenter that did this). If you’re down with this idea or have feedback on how to improve it, chime in with a comment.

The title is a reference to the final scene of one of the radest 80′s movies ever: “Back to the Future.” I remember walking out of that theater as a kid hopped up on red vines, Huey Lewis, the prospect of time travel, and all the possibility that a flying delorean represented. It seemed like anything was possible.

I have a similar optimism today with this swirling curling storm of a revolution that’s promising to change how products will be test marketed, built and delivered. I predict this fundamental change is going to do for product development and business model generation what test-driven development did for software dev. And it’s pretty freakin’ exciting to be swimming in this stew of startup activity while this storm is developing. To explain the essence of this mentality let me first tell a story that will reveal a double entendre in this post’s title:

I don’t have the original source on this anecdote but supposedly at a California college (Cal Poly?) they were redesigning the campus and trying to figure out where to build the new sidewalks. It was a complex arrangement of buildings and there were a bunch of conflicting opinions about where the sidewalks belonged. Someone had the ingenious idea that rather than speculating, they should instead run an experiment and let the market speak. So they planted grass the first year and waited. At the end of the year they took an aerial photo and the tread-worn ground became the blueprint for the optimal sidewalk routes as chosen perfectly and implicitly by the student body.

So what does this have to do with startups?

I believe we’re on the cusp of seeing some major changes in how products are brought to market. If you follow the Lean Startup, Four Steps to the Epiphany, Customer Development movements then you have the core philosophy already. But what’s interesting is the emergence of tools that allow you to apply these concepts very rapidly on a large and targeted scale via online experiments. We in the tech industry no longer have to build and tear up sidewalks – we can just plant grass first. Rather than explain the techniques for “virtual grass planting,” I figure it will be easier to simply publish the data and methods for experiments I’m conducting now with a local Phoenix startup that I advise. Here’s the gist of it though:

You can think of this mentality like test-driven development for business.

Test-driven development (TDD) is a methodology for creating software where you seemingly put the cart before the horse and write the tests up front. You then go back and do the necessary coding to satisfy the tests. Once the code meets the test, then (and only then) do you go back and fine-tune, refactor and optimize things. Having been a confessed “cowboy coder” back in the day this style of development sounded completely absurd until I saw it in practice at the San Diego Java User Group. Writing the tests first forces you to think differently by getting consensus on the destination and then worrying about the implementation details of how you get there after the fact. In the same way it’s now possible with all these tools to front-load much of the learning about product-market fit, price elasticity & messaging before you ever actually do an ounce of engineering. It’s all about systematically removing uncertainty and converting unknowns to knowns before charging ahead with the concrete.

Anyways, I don’t mean to leave anyone with startup blue balls but we’re not quite ready at this point to open source our experiments. This is an exciting time to be in this space though. To get a good flavor for this type of thinking check out Kent Beck’s talk from the Startup Lessons Learned conference on the logical extension of Agile development to business. And if you’re new here sub the RSS of this feed or this Twitter account to follow along on how we’re validating and iterating at 88mph and 1.21 gigawatts.

Very simple: make it possible to loan a digital book to a friend. Not authorize the same book simultaneously across multiple computers on the same account, but actually de-auth it from one and give it to someone else.

IMHO the first service to do this becomes the dominant eReader format and here’s why: this is the last inadequacy that still drives people like myself to purchase physical books. The reading experience of eReaders has become adequate in every other respect and has other added advantages like search, portability, convenience of sync across multiple devices, instant gratification of being able to download immediately, etc.

I use the Mac client to read Kindle books now and I’ve tinkered with the Apple iBooks. Both are comparable but neither offers this ability to pass a book on after you read it. If there were limitless lending then it could be argued that it would wreck the eBook market and create a secondary blackmarket of people scalping loaned eBooks. But it would also cement that provider’s eBook format as the dominant format and force everyone get an account on their system. Because they still control the auth/de-auth lending process they could mitigate this problem by throttling the frequency or absolute number of times a book could be lent.

This opens a lot of doors. A lot more people would start buying eBooks knowing they could later loan them (for me personally there would never be reason to purchase a physical book again). Once everyone is using their format they make it so easy to purchase new books that whatever sales they lose from people passing on a loaned copy would be more than made up for in new eBook sales. They gain the opportunity to sell into a massive new base of account holders who are lured in initially by the prospect of a free book loan from a friend who already has an account. And they get a HUGE amount of useful data from tracking the reading behaviors and the lineage of lending. Lastly, they enable a crazy new capability if they make it so annotations can be separated from the lent copy and shared across other copies. For instance I would love to be able to subscribe to Derek Sivers’ book markups and flip on his annotations to see the notes he made while I’m reading one of the books on his list. This type of “co-reading” makes it possible to read not just the author’s message but select people’s takeaways inline.

With the release of iTunes 10 and the Ping service, Apple has finally added a social layer to its media player. I would expect eventually the social layer which is being rolled out around music will extend to all forms of their digital content be it a book, movie, TV show, song, podcast, or whatever comes next. Once the loaning capability is baked in, game over. Amazon should preemptively strike and enable this for all current Kindle owners. Turn all the old eBooks currently collecting dust on the proverbial digital shelf into a powerful, free viral campaign for its current subscribers to signup their friends.

Is there a flaw in this strategy or does this seem like an obvious move to anyone else?

Check out the full list of Google services and then think about one that’s not on there now but that could be in a big way: Google Soulmate Finder. If you subscribe to the notions that a) there’s a person out there who uniquely complements you and b) it’s possible to qualify/quantify experiences, traits & behaviors to identify that person and match him/her to you, then Google is in probably the best position of anyone to help you find your soul mate.

Here are the four main reasons why they could (and may choose to at some point) pull this off:

1. Hands down they have the most comprehensive, indexable data set of both explicit and implicit behavioral, personality and intention info in the world. And they have it for a huge population. Folks drink the Google Koolaid in varying degrees (I would be best classified as an intravenous Google Koolaid consumer at this point) but for even the people who only use search, they have access to an incredible amount of material that could be mined for insight into what makes one tick. For instance, for users of Gmail & Google Apps they know: where you’ve been, what you care about, who you correspond with, activities that define you, etc.
2. Their core expertise is in serving relevant results – they rule at pattern matching and developing algorithms that weight results based on what’s working. Their whole search business revolves around improving the quality of results that are delivered and they have more expertise than anyone on refining results via empirical data.
3. Were they to offer a matchmaking service they would have access to a pretty interesting feedback loop by virtue of knowing how things worked out. Their matches might suck at first but pretty quickly the AI could validate which algorithms yielded successful results because they’d know which people stayed together. Other matchmaking services have to rely upon explicit input from the participants to know how well the suggestions worked – Google has data that gives them this implicitly. That means the speed and accuracy with which they could iterate their algorithms isn’t gated by reliance upon explicit feedback from participants.
4. They’re motivated to do things that draw in new users & drive increased usage across all services. Their stated goal is to index the world’s info and make it more accessible. Anything they can do that gets more people hooked on using their services helps their cause. The matchmaking service would be a killer app for both attracting new people and getting existing ones to further embrace all the Google services. If they reliably demonstrated a string of successes in matching people, I guarantee you’d see a bunch of the people that currently subscribe to paid dating sites flock to the Google equivalent if it were a) free and b) more effective. Provided Google sold the story well about “the more you do on our system, the better the quality of our matches will be,” those new users would be heavily motivated to go all-in on using Google services. They would pick up not just new users but die-hard ones motivated by the promise of finding their soulmate.

Now my hunch is that a significant countervailing force here that prevents them from doing this now is the “creepiness factor.” It would make it all too real how much they truly know about you if they were to offer this service today and it might actually have a detrimental effect of driving users to defect from their service. This is definitely something that changes with the times though, we’ve seen the “boiling frog comfort” effect in the last four years with Facebook. Kids growing up today will have never known life without exposing everything via social networks and may be more comfortable with this type of service. At any rate, if you see a heart icon and the “Google Matchmaker” app appear on Google labs, you heard the prediction here first ;-)

Matt Asay wrote a post about a month ago and I’ve been meaning to respond. He argues that the comments section on blog posts tend to devolve into ghettos of expletives and personal attacks instead of productive discussion (I agree with this part of his post). He then proposes that Twitter is the medium we should be using to hold conversations around blog posts (this is the part I take issue with). Rather than describe at length why, here’s a graphic that summarizes the argument:

Examining what’s worked empirically

I believe commenting channels can be examined on the above seven dimensions. Admittedly there is zero science behind the above chart – this is purely subjective analysis via thinking about aspects of each medium and how they contribute to effective or ineffective discussion. In thinking about groups I’ve been involved with and which ones worked well the most interesting realization is that there’s a factor outside of the qualities of the channels themselves that trumps everything here: it’s the glue of interaction beyond the channel. The most meaningful discussions I’ve been a part of were in groups where we had in-person or real-time interaction punctuated by periods of asynchronous online exchanges. AZCFUG, CFUG manager list, Cambrian House, Refresh Phoenix, AZIPA – these are all groups that produced valuable insight and relationships, every one of them anchored by a good level of accountability through real-time interaction outside of the online channels.

When you strip everything away it’s not even the real-time aspect that’s critical though- it’s the accountability / reputation factor that ultimately drives quality discussion. When you know your words stick with you wherever you go, you behave differently. You show respect, humility, ensure what you’re typing actually adds unique value, etc. Remove this factor and you get the faceless, ghetto problem of Digg comments and the troll activity on our city’s newspaper site. You lose pride in ownership, start getting a few broken windows and the whole neighborhood adopts a license to behave badly. BUT… Twitter is not the antidote folks- starting a personal blog and supplementing it with Twitter is. Twitter alone is too short-form of a medium to communicate real substance and the “one-way follow” aspect makes it impossible to see the whole conversation as an outsider (unless people happen to be using explicit #hashtags). Yes, it addresses the accountability concern but it does so at the expense of introducing new issues of an “ADHD/Sound-byte-speak” and fragmented dialogue for the people involved. It’s like having UN delegates hold a debate where everyone can talk into their mic but nobody knows who’s hearing who because each set of headphones is tuned pickup a select fraction of the participants involved.

So what do you suggest?

The question becomes, “short of being able to have face-to-face interaction serve as an anchor of civility in discussion between online exchanges, what do you propose as the most effective means of holding productive debate online?” If you look at the above graphic you’ll understand why I believe the answer is to return to using the personal blog w/ a combination of verified comments for short responses and trackbacks for more substantive responses. Twitter has IMHO shoplifted people’s mojo and derailed this practice that used to be commonplace. I believe we’ll see Twitter fatigue and a resurgence of the way it used to be with volleys of blog posts that mutually link amongst one another. Facebook link shares and Tweets are pointers to content – like a fluid, more social RSS feed. But the real substance of discussion has always and will continue to reside in blogs.

If you were tasked with re-architecting the typical high school experience with the end goal of “better equipping students for whatever they do next,” what changes would you make? Don’t confine your ideas to simple curriculum changes either, go nuts. Change anything about the full experience. Some aspects to consider:

• how are classes delivered?
• what is the best use of classroom time and what interaction can be conducted electronically?
• how are parents involved?
• how are grades determined?
• how are grades reported?
• is there another system besides standardized testing that will yield objective results?
• is striving for objective results even the real goal?
• how early is too early to fork the paths of college-prep hopefuls from others?
• what skills are timeless and is there a better approach than the current one for developing these?
• what’s the new interaction model look like in the classroom?
• if the agile manifesto were written from the perspective of teachers today, how would it read?
• what aspects of how classes are rendered is completely legacy and can be thrown out
• are there any fundamental sea changes occurring with skill development like shift from knowledge-retention as a goal to development of knowledge acquisition strategies?
• how do you amplify “street smarts,” social skills & intangible benefits?
• what can be entirely eliminated from the existing system?

Every time I hear the phrase “education reform” I cringe. Not because I don’t believe it needs reforming, but because the people throwing the phrase around are thinking in terms of “How can we boost standardized test scores? How can we graduate more students? What counseling programs can be introduced to reduce dropouts?” In other words, they’re thinking purely in terms of optimizations to the existing flawed system. The phrase “lipstick on a pig” comes to mind. These people unfortunately suffer from the same “curse of knowledge” that will guarantee their thinking remains constrained to the current paradigm.

Try this Innovation Games exercise: Try to imagine forward ten years. The kids that are currently in high school now will be a few years into the workforce (and possibly more if they skipped the college step). Hold an image of prosperity. Picture a scene of advanced living standards, a world successfully meeting challenges from all sides with our environment, urban development, natural resources, international relations, medical challenges, etc. Picture the younger people who are either responsible for these advancements or who are delivering the labor to render these developments. NOW…

Project yourself back ten years from this future point of prosperity and ask “What must the high school environment for these young adults had to have looked like for them to develop the qualities they possess today?” Note: this is a waaay different question than the typical “what should we change today about schools?”

I haven’t thought through all these suggestions entirely but here are some raw ideas that I’ve been thinking about:

Curriculum substitutions

Allow students to nix their current electives and choose from the following:

• basic accounting & finance: reconcile statements, concepts of interest, budgeting, investing
• speed reading
• typing
• practical web knowledge: effective search techniques, web fundamentals, intro web apps
• GTD methodology
• collaborative web applications 201
• negotiating
• persuasive writing
• public speaking
• sketching
• team collaboration
• scientific method
• critical reading
• logical fallacies
• entrepreneurship
• personal branding
• nutrition, health & wellness
• money management
• time management
• stress management
• basic auto maintenance
• minor home repairs: clothing fixes, appliance maintenance
• incentives & economics
• career safari

While these are typically thought of as college subjects I believe they’re core enough to merit pulling them down into the latter half of high school. The fact that things like money management, critical reasoning skills, basic auto maintenance and time management were never taught has always seemed like a major omission. And how obvious is it to have a course that exposes students to the breadth of careers available and plant early seeds which the student can investigate? While my high school experience was 90% quality, we had courses like Biology, Calculus and European history which I would have gladly traded in a heartbeat for any of the above.

Interaction model changes

To me it’s clear the interaction model needs to move away from the primarily unidirectional “hub and spoke” teacher-broadcasting-to-students approach. It needs to move towards a clustered student-to-student group mesh interaction with the teacher as facilitator. The immediate response is likely “wouldn’t this just be pure chaos with blind leading the blind?” The key is to think of this mesh spanning vertically across grades. Look to the interaction model of a martial arts dojo where experienced students must teach smaller groups of novice students in order to advance to their next belt. When you must learn something to the point at which you can teach it to others, it necessitates an entirely different level of understanding (not to mention it amplifies the power of your teaching force and the new “teachers” are much closer in demographic profile to their students and therefore better able to relate).

Course goal changes

The concept of courses as we know them today seems at best geared towards “imparting knowledge” and at worst “teaching to the test.” Perhaps the goal of school needs to be rethought more in terms of one’s “essence discovery,” cultivating raw talents, inspiring students to seek to develop these and most importantly, getting to a point of being excited to pursue whatever is next? We need to think of educating kids more like developing an athlete. I had this discussion with a good friend over lunch the other day: it seems like schooling for us mostly consisted of loading our mental computers up with software. The game now has changed and it’s more about developing our CPU/RAM capacity, installing a better OS and learning strategies for finding the software we need at runtime.

Grading model changes

To support the above goal, the concept of grades needs to be reassessed entirely. Given how the world works clinging to standardized testing is like if eBay were to discard its peer rating system in favor of a 100pt multiple-choice test administered to each buyer to determine buyer ratings. Sure, it’s a more standardized approach… but that’s all it is. It sacrifices meaningfulness for the preservation of “standards.” I don’t have a concrete proposal for what this new system should be but I know when something’s broken and standardized testing (especially the AIM’s test) is horrifically broken. We need to consider moving to a market-driven peer-review type grade system. Investigate marketplaces like Elance and “asymmetric follow” systems like Twitter to get ideas here.

Changes to Parent involvement

When I was in high school we had a parent teacher conference once a semester. One night. Likely the only interaction our folks would have with our teachers (unless we really f’d up in class that semester). With all the low-friction collaborative tech that’s available these days there has to be a way to channel more meaningful input to the parents and get more pro-active involvement out of them. Invested parents can compensate for a crappy teacher (I know this first hand) while it’s nearly impossible for the best teachers in the world to overcome a crappy parenting situation. And involved parents are better parents. Private microblogging platforms are what I’m envisioning here.

Getting Real

Cordon off at least 1/4 of the time for teaching skills in a Startup Weekend-esque way (ie. not explicitly but rather via pursuit of finishing a real project). Pick from non-profit endeavors or other public works projects and deliver some aspect as part of your class. Fieldtrips weren’t always about ditching the tedium of the classroom and goofing off, they were reminders of how what you were learning was applicable to real life. There’s a lot of room to move what’s currently being taught in classrooms into field exercises. The result is students that are more engaged and learning that gets more deeply encoded and anchored with real experience so it can be recalled later.

Anyways, this post is getting too long. I have a bunch of other ideas, but what do you think? What changes will have needed to have occurred for the workforce ten years from now to be flourishing?

It can’t be assumed it will reach its intended recipient.

It’s not actually a new phenomenon but it seems the deliverability of application-generated email has fallen to a point where a letter sent via the US Postal Service is more likely to reach its intended recipient. Let me explain.

Many services (including our own) use email as an integral part of the service itself. Account activation, critical system notifications, trial key issuance, software update alerts, billing-related communications: email is the transport mechanism we rely upon because it’s realtime and it’s the lowest common denominator for reaching a user. The recent preponderance of SPAM however and (consequent aggressiveness of spam filters) has rendered email unreliable for this purpose.

Person-generated emails still seem to make it 99% of the time but I’d guess the deliverability of our automated emails is maybe 85%. In scenarios of account activation it’s merely an annoyance but in scenarios of proactive notifications of important events this is a real issue. Failing to receive those communications can have real material impact to the customer.

How are folks dealing with the unreliability of email in their apps? Are you staying within the realm of email and seeking better ways to ensure delivery? Exploring alternate communication mediums like SMS or IM’s? Offering personalized, protected RSS feeds of account activity? Or has someone developed a web service that can launch carrier pigeons?