On the merits of plate-spinning

Chris Messina posted his framing of open source via the metaphor of spinning plates. In a nutshell, volunteer efforts help keep the plates spinning. How well a project can leverage their help (to actually spin the plates) and reward that help will determine how well the project will retain and recruit volunteer effort.

Here’s some more food for thought on that line…

• It’s not just open source projects that follow this metaphor — pretty much any volunteer program does, whether that volunteer effort is backed solely by other volunteers (like most open source projects) or is backed by some non-profit or for-profit enterprise (like church fundraisers or get-out-the-vote political activities).
• However, there is some threshold of visibility before this metaphor really applies. To paraphrase a hackneyed aphorism, if a spinning plate fell in a forest and nobody was there to hear it, was the plate really spinning in the first place? So, the vast majority of open source projects on SourceForge, for example, have either one person spinning a plate, or have an image of a spinning plate emblazoned on the project’s tombstone. Only once some community notices and values the project might others decide that plate-spinning sounds like fun.
• The advantage that open source projects and other online-based volunteer efforts have is a wider potential set of volunteers (not limited by geography) and a lower switching cost (it’s fairly cheap for people to give the plate a love tap and see if they like the spinning process). Of course, the lower switching cost for volunteer time is also a disadvantage. In contrast, local volunteer efforts, like helping with your town’s 150th anniversary celebration, have a limited set of volunteers and a high switching cost into the project.
• Project efficiency should, in theory, help determine long-term winners. I believe this is what Mr. Messina refers to as “build[ing] infrastructure to harness the additional marginal effort that is contributed as social capital”. Since I have a physics undergrad degree, I tend to think of this in terms of friction — the more energy that is lost to friction and not applied to the project, the less likely it is that the volunteers will feel their contributions are worth their effort and the more likely it is that they will switch. Friction can arise from many sources, such as requiring copyright assignment, constant flamewars on project discussion lists, and a slow release cycle. Lubrication too can come from many sources, such as facilitating many venues for code contribution (develop the core, develop something distributed as a contrib/ item, develop against an API or standard and be a “friend of the family”, etc.).

 

Government Needs an API…And A Thunk

Tim O’Reilly recently raised the notion of Congress getting a version control system, so interested citizenry could see changes made to legislation.

xDebate is focused on another facet of the same problem: putting an API on government. We need a open standard for raising issues and taking positions on those issues so those results can be aggregated and presented to government to help them understand what their citizens want done.

However, in this case, an “API” is insufficient. Let’s suppose we get lots and lots of people taking positions on issues of relevance. We cannot assume that legislators at all levels of government will be paying attention to the issues’ results. After all, at the Congressional level, they’re so swamped with email that if you try sending it to them, it’s as likely as not to wind up in the bit bucket. On the other end of the spectrum, at the municipal level, many legislators in Middle America aren’t comfortable with computers, for those who actually own or use one.

Hence, we need a thunk, to bridge the open standards of online opinion gathering with the varied and more-offline needs of our legislators. We will need to compile the results into reports, nicely bound, with summaries and reprints of highly-linked-to commentary, and we will need to mail those reports to Congress. We will need to provide “flash” updates by fax, or by FedEx. Over time, as the legislature gets more Net-savvy, we can reduce or eliminate the thunk, but until then, we have to get the opinion where the legislature (and the media) will not miss it.

 

Going Wide: Themability

In many cases, when two Web applications need to be integrated to support a customer or market, the fact that the applications look different is not a big deal. For example, lots of people integrate with Google Maps, yet they don’t make their own sites look like Google.

However, if the customer or market needs to see a single brand — with the illusion that the integrated Web applications are really all from one provider — that means one or both applications will need to change their look and feel to be consistent with each other. This is a common customer demand, and the probability that you’ll want to honor that demand depends on how desperate you are for that customer.

Remember that “themability” (i.e., the ability to change look and feel) can be scoped to this specific customer if you can use a custom domain name (e.g., customer.myapp.com or myapp.customer.org). Just watch for the Host header and switch out the theme on the fly as needed. In fact, one theme hack is to leave your application alone and add a module that rewrites your HTML for the customer’s benefit as needed (e.g., via a Java servlet filter in a J2EE app).

 

Going Wide: Clear Content Licensing

For other applications to be able to use APIs made available by your application, there needs to be a clear set of Terms of Use, including clear content licensing, so potential integration authors know what is and isn’t kosher vis a vis your hosted application.

“Clear content licensing” can run the gamut from Creative Commons licensing to simply a “no, you can’t use it” policy. Bear in mind that the latter will limit the effectiveness of your API program — if people feel it’s illegal for them to use your API, there’s less value in having the API in the first place.

Having nebulous or unwritten content licensing will give you a short-term gain, in that people will assume the best and integrate with your APIs and such. However, should you turn around and lock down the content licensing, you’ll outrage a bunch of folks, which may lead to bad publicity or more energized competition for your application.

 

Going Wide: APIs

A specific type of contact point (per the previous post) is the API. Does your application specifically allow other applications to retrieve, or maybe update, data from yours? If so, you need to spell out how. If not, you need to realize that APIs may well be forced upon you.

As examples of the latter, take eBay and Craiglist. Early on, eBay offered no APIs to speak of. But, eBay became popular. As a result, engineers “rolled their own” APIs via “Web-scraping”: simulating an ordinary Web browser with another program and “scraping” out the data needed from the response to a URL request. This violated eBay’s terms of use, which limited how popular such a third-party-imposed API could be. Similarly, Craigslist even today offers no API (to the best of my knowledge) and historically has fought the use of their data for services like cross-community classified searches.

The needs of other applications that wish to integrate with yours, vis a vis an API, may be modest. Many times, it will simply be gathering enough information to help a user link to the right material in your application. For example, suppose your application has the notion of “categories”, and another application wishes to allow users to link from their app to yours, going to a specific category. To do that, the other application needs a roster of categories and contact-point URLs for each. Either you have to provide that via a supported (or at least endorsed) API, or they’ll have to scrape it out of your application, or they’ll have to skip that capability, possibly to the detriment of users or yourself, if your application becomes less popular due to this missing bit of integration.

This doesn’t mean you necessarily have to spend zillions of staff hours implementing APIs for all possible requests up front. You could, for example, build an API for a small handful of expected requests, endorse Web-scraping for things outside the current API, and slowly expand the API to support things that lots of people have been scraping.

 

Going Wide: Contact Points

And the “Going Wide” series continues…

Given SSO, per the previous blog post, it is now possible for two sites to cross-link and have a seamless user experience, right?

Well, not really. SSO is necessary but not sufficient for seamlessness. Next up: points of contact.

If SSO allows Web 2.0 Application A to link to Application B without the user having to re-login…that says nothing about the specific places in Application B where it makes sense to have such links. Some applications support arbitrary linking, such as being able to link to just about any page in Wikipedia. Many other applications, though, have limits as to where you can safely link in, and those limits need to be documented.

For example:

• Sites that use context parameters in URLs (e.g., the old ;jsession= parameter you see in some J2EE sites) probably can’t support you linking to an old session.
• Sites may have legal restrictions as to where you can link without repercussions, as evidenced by various “deep-linking” lawsuits.
• Some URLs may be transitory (e.g., URLs that have embedded transaction information, like shopping cart IDs) even if they appear normal. Linking to one of those will likely bring up an error page, or at least not take the user any place useful.

If you are making a Web 2.0 application that you expect others to link into, you need to document the limits your application has for safe places to link to.

 

Going Wide: SSO

In the model I’ve been discussing over these blog postings, there are three layers of interested parties:

• You
• Your customers — which you’re trying to raise from tens to tens of thousands
• Your customers’ customers

In the example I’ve used (online ordering service for smaller restaurants), your customers are restaurants, and their customers are the patrons of the overall service. Most of the actual users of your services are patrons of restaurants; however, the users that pay your bills are the employees of the restaurant who, presumably, use the administrative side of your service (e.g., uploading menus and prices).

In my previous post, I discussed the notion that your Web 2.0 service may not be the complete picture for your users, that your service may need to integrate with other services to provide a “whole solution” to an audience. So, for patrons, your service might need to integrate into the online delivery service’s Web site. For restaurant staff, your service might need to integrate into their existing order tracking system (if they have one), or perhaps an accounting system.

The first place that the users might trip over a seam between these disparate services is at sign-on time. If Service A requires a login and Service B requires a separate login, with separate registration and separate password rules and all that, you’re toast. Restaurant workers might put up with that; patrons won’t.

Hence, the first thing you need to address in your service is single-sign-on (SSO), so patrons who place an order on your site can then seamlessly slide over to the delivery service’s site to choose an address to where dinner should be delivered.

The closest thing to a standard in this area that has widespread attention is OpenID. With OpenID, both you and your peer services rely on third parties for actual authentication. The OpenID specifications support SSO, in that once the user has logged into the first service, they probably don’t have to provide their password a second time to use the second service (“probably” because that’s up to their Identity Provider, but usually it’s not required for two authentication requests within the same browser session).

 

Going Wide: Integration

Continuing the series of posts on what Web 2.0 purveyors need to consider as they scale their application up from tens of markets to tens of thousands, let’s talk about integration.

Most likely, if you’re reading this, you do not work for Microsoft, you don’t work for Red Hat, and your name is not Mark Shuttleworth. That means that your firm is in the business of delivering an application or two to customers. Those applications, as awe-inspiring as they may be, won’t deliver everything that markets need. In my example of an online ordering service for small restaurants, restaurants still need an accounting system, general office tools (word processor, etc.), and the like.

 

Going Wide: Hosting Models

So you want to take your Web application from tens of markets to tens of thousands of markets, huh? You’ll need to figure out how the heck you’re going to host all of that!

Here are some potential answers you might give to the question “Will we be willing to host tens of thousands of branded and configured versions of our application?”:

 

Going Wide: Instantiation Cost

The “Going Wide” series of posts is to collect my thoughts on Web 2.0 applications “going wide” — working with other parties to get an existing application used in many disparate markets. The common case for this is where you have an application that has some sort of geographic focus (e.g., an online ordering service for small independent restaurants) that has applicability elsewhere, but where you’re not in position to be trying to sell it nationwide yourself.

Issue #1 is the cost of “instantiating” your application.