This is the second half of our interview. Part one can be found here. In this half we focus on Couchappspora, Ogden's open source social networking project.

Sponsor

Klint Finley: How many Couchappsora users are there?

Max Ogden: I'm not sure at this point to be honest. Probably on the order of hundreds. It is incredibly alpha.

The nature of it is that you can set up a CouchDB somewhere and then tell your Couch to start replicating from anyone else's Couchappspora instance or start your own disconnected network. Replicating in that sense is similar to 'following' someone. So it is a big experiment in creating small decentralized networks.

One of the comments that I always think of came from a user named 'Chauncey' on Couchappspora. He left a comment that said "Anything 'small' like this can be really 'big' in China market, and web app like this is no exception." I think it was in reference to the aggressive censorship that is prevalent there.

The development timeline of Couchappspora so far hasn't been that speedy. Originally we did a couple weeks of work after the initial Diaspora release back in September of 2010. And then it laid dormant for a few months. I actually picked it back up at the beginning of February because I got motivated by the turmoil in Egypt. In the last few months I had been ruminating on what other projects people are working on in this space and how I can make Couchappspora something relevant. I've decided to use it as an excuse to write interfaces for the OStatus stack.

In terms of OStatus, the most exciting part of it to me is the ActivityStreams spec, which is being spearheaded by some folks at IBM, Google, Myspace and Microsoft. See the bottom of this document for references.

ActivityStreams is implementing what is called "actor-verb-object" logic to represent social streams. So if you look at the front page of Github, of Twitter, of Facebook or any other social network (or feed) you can extract actor verb object triples such as "Max pushed a new piece of functionality to his project on github" or "Tyler posted 'hello max' to your wall" or "Klint uploaded a photo of his cat to icanhazcheezburger."

So if we can convince these major players to implement ActivityStreams as a syndication format...

You could pull stuff from all around the web into Couchappspora?

Exactly. Jeremie Miller's Locker Project is really exciting. I had a chance to have dinner with Jeremie and the rest of his team at Sing.ly a few weeks ago and they got me really excited about Locker and Telehash and I got them really excited about CouchDB and Ostatus. It was a productive meeting.

At a high level I am promoting the idea that data should be able to flow freely. I'm actually getting mentored by Stewart Brand this year, which is one of the most exciting things to ever happen to me. He famously stated "information wants to be free," and I think that idea gets more and more important as we move forward. Especially when governments start shutting off the Internet.

To state it even more concisely, I want users to have the option to be totally private and "off the grid." Right now it is a binary system. You're on Facebook or not. But I want to introduce the option to host your data anywhere if you are motivated to do so.

I would like to see Linux nerds running social networks from their cellphones and when they turn their phones off they become invisible online.

I think that most people don't care about their privacy. A small percentage does care a lot and it's important to facilitate the rights of all users.

I want to see Couchappspora or other related technologies be combined with mesh networking hardware in situations like Egypt or Iran where you run a social network server on your cell phone and it will automatically 'hop' your data around the crowd to find the one guy who has internet access.

I would love to see something like that. I've been covering mesh networks a lot lately.

Couch.io/CouchOne/CouchBase are working on getting Couch running everywhere, which is really exciting in that respect. You can run it on Android now and iOS and in browser via HTML5 are in the pipeline.

Is it possible to run Couchappsora on Android then?

Yep, although I don't have an Android.

What advice would you give people just starting out with Couch?

The hardest thing to 'grok' with couch is how to get data in and out, the workflow of pushing and pulling. There are some great developer tools that make it really easy, but like any other command line utilities they can sometimes be a pain to install since the entire API is implemented in HTTP. It is hard to wrap your head around sometimes, but you can literally do anything in couch over HTTP, which is one of it's strongest points.

Back to part one.

This is the second half of our interview. Part one can be found here. In this half we focus on Couchappspora, Ogden's open source social networking project.

Sponsor

Klint Finley: How many Couchappsora users are there?

Max Ogden: I'm not sure at this point to be honest. Probably on the order of hundreds. It is incredibly alpha.

The nature of it is that you can set up a CouchDB somewhere and then tell your Couch to start replicating from anyone else's Couchappspora instance or start your own disconnected network. Replicating in that sense is similar to 'following' someone. So it is a big experiment in creating small decentralized networks.

One of the comments that I always think of came from a user named 'Chauncey' on Couchappspora. He left a comment that said "Anything 'small' like this can be really 'big' in China market, and web app like this is no exception." I think it was in reference to the aggressive censorship that is prevalent there.

The development timeline of Couchappspora so far hasn't been that speedy. Originally we did a couple weeks of work after the initial Diaspora release back in September of 2010. And then it laid dormant for a few months. I actually picked it back up at the beginning of February because I got motivated by the turmoil in Egypt. In the last few months I had been ruminating on what other projects people are working on in this space and how I can make Couchappspora something relevant. I've decided to use it as an excuse to write interfaces for the OStatus stack.

In terms of OStatus, the most exciting part of it to me is the ActivityStreams spec, which is being spearheaded by some folks at IBM, Google, Myspace and Microsoft. See the bottom of this document for references.

ActivityStreams is implementing what is called "actor-verb-object" logic to represent social streams. So if you look at the front page of Github, of Twitter, of Facebook or any other social network (or feed) you can extract actor verb object triples such as "Max pushed a new piece of functionality to his project on github" or "Tyler posted 'hello max' to your wall" or "Klint uploaded a photo of his cat to icanhazcheezburger."

So if we can convince these major players to implement ActivityStreams as a syndication format...

You could pull stuff from all around the web into Couchappspora?

Exactly. Jeremie Miller's Locker Project is really exciting. I had a chance to have dinner with Jeremie and the rest of his team at Sing.ly a few weeks ago and they got me really excited about Locker and Telehash and I got them really excited about CouchDB and Ostatus. It was a productive meeting.

At a high level I am promoting the idea that data should be able to flow freely. I'm actually getting mentored by Stewart Brand this year, which is one of the most exciting things to ever happen to me. He famously stated "information wants to be free," and I think that idea gets more and more important as we move forward. Especially when governments start shutting off the Internet.

To state it even more concisely, I want users to have the option to be totally private and "off the grid." Right now it is a binary system. You're on Facebook or not. But I want to introduce the option to host your data anywhere if you are motivated to do so.

I would like to see Linux nerds running social networks from their cellphones and when they turn their phones off they become invisible online.

I think that most people don't care about their privacy. A small percentage does care a lot and it's important to facilitate the rights of all users.

I want to see Couchappspora or other related technologies be combined with mesh networking hardware in situations like Egypt or Iran where you run a social network server on your cell phone and it will automatically 'hop' your data around the crowd to find the one guy who has internet access.

I would love to see something like that. I've been covering mesh networks a lot lately.

Couch.io/CouchOne/CouchBase are working on getting Couch running everywhere, which is really exciting in that respect. You can run it on Android now and iOS and in browser via HTML5 are in the pipeline.

Is it possible to run Couchappsora on Android then?

Yep, although I don't have an Android.

What advice would you give people just starting out with Couch?

The hardest thing to 'grok' with couch is how to get data in and out, the workflow of pushing and pulling. There are some great developer tools that make it really easy, but like any other command line utilities they can sometimes be a pain to install since the entire API is implemented in HTTP. It is hard to wrap your head around sometimes, but you can literally do anything in couch over HTTP, which is one of it's strongest points.

Back to part one.

I recently talked to him about his Code for America fellowship, how he got started programming, CouchDB and much more. Tune in next week for part two of the interview.

Sponsor

Klint Finley: Can you start by telling us a little big about who you are and what you do?

Max Ogden: Sure. I'm Max Ogden, originally from Oregon, most recently Portland. Now I'm talking to you over IRC from Harvard's guest wi-fi and I technically live in San Francisco, although I'm in Boston for the month of February on assignment for Code for America where I'm a fellow this year.

I'm working out of Boston City Hall on a software project intended to give more educational resources to low income populations in the city of Boston. But outside of Code for America I've previously worked at some startups in Portland. But my real passion lies in the communication and networking space. More recently I've been getting involved in different privacy-centric open source projects.

How did you become a Code for America fellow?

I was recruited by Tim O'Reilly. He worked his charismatic magic on me. The mayor of Portland, Sam Adams, was presenting me with an award for doing open source and open data work in Portland and Tim was at the awards ceremony. He said "We'd like to see this happen in more cities than just Portland." That was a pretty exciting proposition.

How did you get started with Web development, and programming in general?

In high school I started taking classes, I actually got hooked by playing Starcraft back in 1999. The custom level editor has elements of event-driven asynchronous programming and I started geeking out on custom Starcraft maps. And then realized that programming is just Starcraft without the space aliens.

My programming career was largely self-directed, but then I got a great job at a marketing research company under some really talented engineers. It was a really good mentoring environment. Agile processes, pair programming, test driven development. A lot of experimentation with newer technologies and a lot of freedom to innovate. But the nature of the work was building human interfaces to technology.

The company's purpose was to create ways for consumers to express qualitative information about consumer goods, a la focus groups, but using consumer technology as the capture mechanism. So I have a background in helping people be more human using technology. Which is an interesting field/challenge.

Did you go to college for computer science or did you go straight into the job market after high school?

I skipped college altogether actually and went straight into startup land.

You do a lot of work with CouchDB, notably GeoCouch and Couchappsora. What attracted you to Couch?

I got interested in CouchDB when Portland started releasing a lot of government generated data sets. Because Portland is so laid back, I would get off of work at 5pm and then have the rest of the evening to hop around town and code at cafes and bars with peers. Cost of rent is low so you can afford to not take your work home with you.

So the Mayor's office started releasing raw data and it was in the native GIS format that the government created it in, which requires certain proprietary software in order to work with. A lot of major cities have $50,000 + annual licenses for these software suites. The barrier of entry there is high. So I tried decoding a single data set to see what the workflow was like and nine hours later I had a list of 2,900 bicycle parking racks in JSON format

So the problem arose as: how do I host the other approximately 100 geographic datasets in the same nice JSON format for other developers to work with, so that they don't have to go through all of the trouble that I originally had to go through to work with the data. So I was shopping around for databases and GeoCouch was brand new at the time and was the most elegant and simple solution that I've found since.

Basically I can dump a bunch government data into it and with little to no special configuration it automatically becomes a publicly accessible API. I guess the biggest "feature" of Couch.

The relevance to my software philosophy is the idea that Couch is by default open and replicable. So any data that is in Couch becomes fully replicable by anyone.

Right now we have the notion of a blog presented via HTML and then a separate syndicatable feed, usually via RSS. Couch is basically a more robust pub/sub mechanism that would replace RSS in that model. So for open government data it is really enticing because anyone can replicate the actual API itself, not just consume the API via one-off queries.

Can you talk a little more about GeoCouch and what it is?

GeoCouch is just a light layer on top of Couch. There are a couple notable CouchDB "plugins." One is GeoCouch, the other is CouchDB-Lucene. GeoCouch lets you quickly look up spatial objects. So if you have, for instance, a list of all of the fire hydrants in Boston - there happens to be 13,000 - and you wanted to show a firefighter the three fire hydrants on the block that he is standing on you can filter those three out of the list of 13,000 in a very efficient manner. On the order of milliseconds.

CouchDB-Lucene is similar in nature, except instead of spatial queries it lets you do full text indexing. The normal out of the box Couch has a more familiar B-Tree index, which is a pretty general purpose indexing engine for lists of data. But spatial queries and full text indexes are more specialized around big lists of geolocated objects or large bodies of freeform text. The general workflow is: You put data into a database ("folder," "bucket") and then you hook up a Web application to pull that data out and display it. So I usually do a lot of the logic on the client side in JavaScript.

The nice side effect of that strict separation between data and presentation is that every site built that way automatically has a replicable API. So you can use the Couch "protocol" to subscribe to anyone's API.

When Diaspora got released they tightly coupled the data layer and the presentation layer so it was a big Rails app that had a lot of complex dependencies and fancy front end-user interfaces and an opaque backend that didn't adhere to any notable syndicatable protocols. The OStatus stack was missing, for instance. OStatus is a set of six or seven different formats like RSS for creating decentralized subscription based social networks. But OStatus is front-end agnostic. OStatus doesn't advocate for any specific user interface, it expects to you built your front end in new and interesting ways.

On some level I was bored with Facebook because every user is exposed to the same interface. It was too standardized. And on another level Facebook's centralized data ownership was morally questionable. And then Diaspora came along and didn't seem to promote any data standards but instead tried to make its front end the selling point. Which is great, but you need to pursue both the standards in the backend and the innovation in the user experience at least that's the way I look at it.

So I started hacking with RWW resident hacker Tyler Gillies on Couchappspora, which used the Couch protocol to replicate data but used Diaspora's front end.

To be continued...

As a developer of web applications, Marco Arment is the kind of guy who builds things fast, gets it out there and iterates quickly. I asked him: is that pretty much his philosophy, or is there more to it?

Sponsor

Arment replied that there are two different approaches he takes. For his browser-based web apps, he said, "the features are very, very simple. At least, they end up being very simple once I've handed them out. And so the web apps are intentionally kept very basic."

But for the iPhone and Kindle platform devices that Instapaper has, in addition to the web site, they are "much more complicated."

Arment told us that he has to do a conservative release cycle with these apps. "Especially with the App Store's approval process," he said, "I have to do a much more conservative release cycle on that."

If Instapaper has to issue a bug in a release on the iPhone, he told us, he typically has to wait 10 days before the fix is approved and pushed out.

"That process has gotten better over time, by a lot actually. Much better than the way it used to be. But, there's still a delay there. And so I have to be very, very careful that I don't ship bugs to customers in the iPhone app. I'm much more conservative, the release cycle is much longer and there's much more testing involved before I release. I'm generally very conservative about it."

Arment concluded by noting that "on the [Instapaper] web site, I'll just do something very quickly and throw it out there and see what happens. Then adjust it and iterate until it's ready."

Let us know your thoughts on product development for web apps - browser-based or native apps - in the comments.

Photo credit: Lucius Kwok

Just as interesting to watch is how new startups will design for the changing usage patterns of TV consumers. Clicker is one such company. Co-founded by ex-Ask.com CEO Jim Lanzone, Clicker aims to be the TV Guide of Internet TV. I spoke with Lanzone to find out how the idea came about and to hear his thoughts on how Internet TV is evolving.

Sponsor

How Clicker Was Conceived

RWW: How did you come up with the idea for Clicker and what was the inspiration for it?

Jim Lanzone: Every Monday, Dave Goldberg [at the time an entrepreneur-in-residence at Benchmark] and I would go to Starbucks on Sand Hill Road and brainstorm together. He pointed out to me that in the 70s, TV Guide was worth more than the big three networks combined. We agreed that television was going to be online and when that happened, finding what to watch or figuring out what you want to watch was a lot more like a search and navigation problem than a schedule grid - you know, a calendar - which was what TV Guide has been over the last 50 years.

So we just started brainstorming [on that] for about a month. And then, I got very, very serious about it. Bill Gurley at Benchmark and I then started talking more seriously about it. Dave was not ready to jump in at that point, maybe that early. And so the more I talked about it with them, the more serious I got about wanting to start a company around it. I was looking at all kinds of other things to do, other public companies to go to or take over somebody else's start-up. This [Clicker] was definitely the riskiest, but it was the one that I felt the most passionate about. Also I loved the idea of being able to get some of the members of my old product and technology teams back together to tackle a new problem.

Building the Product

RWW: Tell us about how you started the company and the beginnings of the product.

Jim Lanzone: I raised money from Bill at Benchmark and then from Jeff Yang at Redpoint. Bill found a small little company in Los Angeles that had been putting together a very early-stage version of what I wanted to build. But mostly doing it manually, you know - not with technology. The company was called 'Modern Feed.' And so we actually acquired them as part of the funding. We subsumed them into our company.

I started hiring some of the main people from the Ask.com technology team [which Lanzone used to lead]. That was the foundation of the Clicker team. It really all came together in January of '09, and then we launched into beta at TechCrunch 50 in September of '09, and then out of beta just two months later at Bill Mallow's Conference at UTV in November.

The first year was spent building the kernel of the company; the real asset that we have that we can deploy in many different ways, which is the database of shows, episodes, movies, music videos, and live events on the web. And the technologies that maintain and update that on automated basis [and] organized in a structured data fashion.

The goal is [to build] the ultimate programming guide for the next generation of TV, which is about navigation and discovery. So it's not just finding TV shows, it's also how you decide what to watch.

Next Page: Who's Watching Internet TV? Plus the Second Screen vs Set-Top Boxes debate...

I caught up with GetGlue founder and CEO Alex Iskold to discuss the evolution of the product since its inception. It's changed from an under-used geeky Firefox browser add-on, to a mainstream service where hundreds of thousands of people check-in to Mad Men and other popular entertainment shows. How has GetGlue made this transition? One word, by getting emotional.

Sponsor

What's more, the changes have been good for GetGlue. It has experienced strong growth this year. Iskold told me that "in the month of August alone we saw over 8 million ratings and check-ins." That's about 300,000 ratings and check-ins every day. GetGlue currently has over 600,000 users and is, according to Iskold, riding "an upward trend in the social entertainment market."

People Get Emotional About Entertainment

GetGlue changed its branding and launched a new website, GetGlue.com, last November. It changed almost overnight from a geeky browser add-on called Blue Organizer to a destination website called GetGlue. Mobile applications followed soon after.

"Once we launched the website," Iskold told me, "it made a world of a difference and ever since [we've had] exponential growth that continues to increase."

It wasn't until the re-launch that Iskold and company realized that their core users are emotional about the things that they're watching on TV and the things they're consuming.

"It was because we kind of stood back and said, what we need to do is create something that will be a fit for entertainment. The big insight was that it needs to be emotional. Our users are really emotional about GetGlue and about their entertainment - so that strikes the chord with them. That was a big turn around for us."

"When you you read a book," Iskold said (knowing that I am a book lover), "it's an emotional experience, right? Likewise, other people are very passionate about different forms of entertainment. Somebody may like True Blood, somebody may like Mad Men, somebody may be into classical music. Humans are attached to entertainment and entertainment drives our emotions. Everybody wants to talk about entertainment and essentially it's a form of self-expression. Which books you like, which movies you like, which shows you watch - it's self-expression and something that we'd like to discuss and tell each other about."

From Browser Add-On to Destination Website

It was fairly obvious to industry observers last year that the Blue Organizer browser add-on wasn't quite working - a few of us at ReadWriteWeb found it hard to keep using the product after the initial trial period. Despite the issues, the November re-branding and move to a different delivery platform was a drastic change. I asked Alex Iskold what drove the company to that? He replied that "a process for every start-up that ultimately succeeds is a process of iteration."

"I think it starts with the idea. The idea has always been the same, which is the vision of connecting people around entertainment that they love, around things they love. I mean you know that since day one, we were fascinated with the idea of why should we be separated by different websites? If you are a NetFlix customer and I am a Blockbuster customer, why can't we share each other's tastes and why can't we see each other's comments?

So this original idea is at the heart of the service, which is that we're connecting people around entertainment. But the delivery, as you said, just wasn't right. The browser add-on model was interesting and innovative. But the problem was it wasn't generating emotions and it wasn't resonating with people. I mean we had over two million downloads of the add-on, but it just wasn't the delivery [mechanism] that people ultimately wanted."

Iskold told me that the company had a bunch of internal conversations about how to move beyond the browser add-on. Eventually one of their investors told the team that they "cannot be this closed network." He encouraged GetGlue to be "part of the open web" and launch a destination website.

Usage Patterns

Seeing as the product changed so much, how do people use GetGlue nowadays?

"Well, we have a bunch of mobile apps," replied Iskold. "We have an iPhone app and we recently launched an iPad app. We have an Android app. We also have a mobile website. Most of the usage for checking in happens through the [mobile] apps. So it's a natural thing. You're sitting on the couch, you're watching Mad Men, you want to check-in and tell your friends what you're doing. Or you're at work listening to music and you can check-in."

In terms of browsing information on GetGlue, Iskold said that "most of the ratings are happening through getglue.com [i.e. the website], about 80% of the ratings." He suggested that use cases for browsing GetGlue are when the user is bored or nostalgic.

"I want to go in check out what my friends are doing but also read a bunch of lists - action films that I've seen, books that I've read. So you're just basically being nostalgic [about] entertainment and browsing through peoples' profiles. We see most of the ratings coming through this way."

I wondered whether a lot of the new mainstream audience has been coming in via Twitter and Facebook? Surprisingly, Iskold said that the opposite is more generally the case.

"People are sharing from GetGlue to Facebook and Twitter a lot," he noted. "They're sharing at a rate of three shares per minute to each network. On weekends and evenings, we're actually seeing over ten shares per minute to each of these networks."

I asked if the current audience contributes content (ratings, check-ins) at the same rate as the early adopter users that the browser add-on had? Iskold pointed to checking in and rating as two very common actions.

"Everybody rates. Ratings are huge for us. Checking in is a really light weight thing. Hey, so you checked in to Mad Men. But do you actually like the show? If so, then what other shows do you like? That's how you build your taste profile and then you get suggestions. Based on the suggestions, you check-in again. So the whole flow is basically when users get in they start checking in. It's really light weight. Then, over the next couple of weeks, they will build up their taste profile, with thousands of data points. And we will then be able to give them intelligent recommendations for what else to consume. And then because of the check-ins, they stay engaged. They can also get rewards from big brands and that's pretty exciting for people."

Partnering With Entertainment Brands

Another reason for GetGlue's recent momentum is the partnerships it has inked with blockbuster entertainment companies like HBO, Showtime, Fox, PBS, Universal Pictures and "dozens of other brands." At the end of last week, GetGlue announced its first major news network partnership - with MSNBC. It also obtained "a very exciting set of rewards" from ABC (the makers of Mad Men, a show that is obviously popular on GetGlue), Discovery HDTV and a couple of other networks.

I was curious to know how a partnership with an entertainment company usually works?

"There are various ways that partners promote GetGlue," commented Iskold. "The most basic one is like you said: they're promoting us through their Twitter accounts, through their Facebook accounts and sending their fans to us. Secondly, and you can see this on HBO for example, they have [GetGlue] units on their homepage. [For example] Sony Pictures has some units on their homepage which basically say: 'check-in with GetGlue to unlock exclusive rewards from us.'"

Iskold said that GetGlue will soon announce "partner check-ins" - which will be "essentially another way for partners to promote us."

"It's actually a really interesting use case. Most of the TV networks have full episodes online and they want people to come to their site to watch them - because they can run ads on those episodes. So it's economical for them. Imagine a widget on Showtime's Dexter web page, where you can be watching the show and at the same time check in and say "I'm watching Dexter on Showtime." The kicker is that the link will be shared to Facebook and Twitter, which will point [back] to Showtime's page. So for the networks, it's really interesting because it's a simple and viral mechanism to get people to spread the message, via fans you have watching on your web site."

In Part 2 of this interview, we explore Alex Iskold's product development philosophy and some of the trends that he's tracking.

Let us know in the comments if you use GetGlue or a similar product. Do you "check in" to entertainment things?

Disclosure: Alex Iskold used to be a feature writer for ReadWriteWeb, back in the old browser add-on days!

A small startup company called InfoChimps released for sale yesterday three very large sets of data extracted from 500 million Twitter messages. Included in the offering are the senders and recipients of 1 billion @ messages, Retweets and Favorites. We wrote in-depth about the release late last night. This morning we interviewed Iain Dodsworth, creator of the most popular Twitter client, TweetDeck, about the value he might find in that data and the direction he's aiming to take TweetDeck in the future.

Sponsor

Dodsworth: Straight off the bat - an archive of tweets could form the basis of a profiler and that's very interesting. Sentiment analysis (which I am ALL over) requires that kind of base corpus.

RWW: InfoChimps isn't releasing full text yet, but they would do a custom slice if you wanted it.

Dodsworth: It's the historical element that a large number of services are missing and where they will fall flat - analysis based on the last few hundreds tweets is almost pointless.

RWW: I'm curious what "a profiler" might mean to you and what this data could help make possible in those terms.

Dodsworth: For me a true profiler would be akin to the holy grail - we would analyse who a person converses with, who RTs them the most, essentially all interactions. Then we would track activity metrics (how many tweets sent, replies) and then we would analyse language patterns (usage of certain words) to ascertain how they express themselves and pinpoint sentiment. Off the top of my head this could lead to elements of intention prediction and I'm steering TweetDeck to have this kind of very very basic Artificial Intelligence at its heart.

I'm currently researching intent predicition inside high frequency trading systems and it's fascinating and could directly relate to TweetDeck and social media systems/services in general.

[Dodsworth's background is in developing for financial services, at places like Prudential Financial and PricewaterhouseCoopers.]

RWW: What would intention prediction look like in this context? On twitter?

Dodsworth: At its most basic if TweetDeck could predict what the user was probably about to require next, based on current activity, then it could start to collate that data in the background - cross twitter/facebook/linkedin data for example. I'm looking at it right now from a cross-service data gathering perspective where our servers do the gathering and hopefully get around the issues of API limits for example.

This is based on future functionality we're mapping out now which is a lot more complex than looking at someone's profile or seeing how many RTs one of your tweets has.

I'm thinking the scope is full social graph rather than just twitter/facebook.

RWW: I guess I'm having a hard time imagining what "what the user was probably about to require next, based on current activity, then TweetDeck could start to collate that data in the background - cross twitter/facebook/linkedin data for example" might look like. Like, if I'm looking at a person's profile, I'd probably like to see their LinkedIn data?

Dodsworth: Good example...or see how a certain person you're tweeting with right now stacks up against "similar" people you've spoken to - a box could pop up mid-conversation and give you a tonne of metrics on this person. How full of [crap] are they? Are they a social media guru? Would you be wise to tell this person anything sensitive? Based on previous language patterns, is the person you're tweeting with right now probably lying? A bit out there but possible in theory.

Since her unplanned departure from an L.A.-based startup in 2008, Berrent has traveled through eight countries, documenting and publicizing the struggles of those in developing areas through her blog posts, tweets, images, videos, and her own presence at events at home and abroad. From post-Katrina New Orleans to a trash dump in Manila to a monastery in Burma, read on for her story of trying to achieve social good through social media.

Sponsor

RWW: "Social media for social good" has become the catchphrase du jour, it seems. What does it actually mean; how much can social media users affect social change, and how?

I am a strong believer in the idea that the things you do online are meant to facilitate your offline interactions. People are so fast to click a button, and that can be great. Retweeting, forwarding, and Facebook walls are great engagements. But what's more difficult is the donate button. That's the big hurdle and disconnect. I'm trying to provide these inspirational opportunities in timeboxed campaigns. Social media is slowly catching on, but there's a lot of noise. Standing out is hard; it's important to have an offline component.

Berrent was visibly disturbed by what she witnessed at this Manila trash dump, where she saw shoeless children running through piles of debris.

RWW: Tell me about your experiences with Kiva borrowers. What kinds of people and enterprises have you seen? In your opinion, does microlending have a measurable impact on struggling local economies?

Kiva is really unique. It has a lot of power users - more than any nonprofit I've ever seen. One man has made a thousand loans. It's individual stories, and people really connect. You get updates on that person, and people say it's their favorite email of the month. As a microlending company, Kiva is one spoke in the larger wheel of microfinance. On a global scale, it has a very big impact.

Typically, when you go to a village or province, certain industries are prevalent. In a fishing community, maybe the borrower bought a fishnet or a fishing boat. In an area with a lot of bamboo, it's going to be crafts. I worked in eleven branch offices. I met over 40 different female borrowers individually and over 250 in my time there.

I can see that the money Kiva provides makes a difference. Microfinance is a very slow process, and there are gems and sparks of people who break through the poverty cycle. When you see villages changing, it's really something. It's like watching grass grow, but it's really beautiful grass.

This woman is a pig farmer and a recipient of funds from a Kiva-affiliated organization.

RWW: Now you're working on a seven-day, seven-city tour to raise awareness and funds for malaria prevention through bed nets. Where did this idea come from?

It's a city-by-city competition on who can raise the most money for malaria nets, but also an opportunity for anyone to donate who wants to get involved. The tour starts this Saturday night in New York City and continues for the next seven days in Miami, New Orleans, Chicago, Seattle, San Francisco, and ends in Los Angeles on Friday...

I'd just finished Kiva training, and I was going to the Philippines for three months. And all I could think was, "When I come back, I'm going to be thirty." I've honed in a lot on my direction - using the Internet to help people. And what if I could use this opportunity to give back, involving people in different parts of the country - something really ambitious?

I wanted it to be about saving lives. I wanted to say, "I saved this many lives on my birthday." I've done a lot of work in HIV and AIDS; I looked into that and polio and malaria, and that's what stuck with me. The campaign has no administrative fees. One hundred percent of the funds go to malaria... in rural northern Ghana. Providing malaria nets will really be a part of saving lives there.

Berrent met this monk in Burma and spent the afternoon pagoda-hopping with him.

RWW: What needs or gaps do you see in philanthropic efforts online?

I think it's not having a strategy to begin with, not knowing the tools in your toolbox before you start. There's a lot to be said for jumping in and having fun, but nonprofits don't have the resources to play around online. They think it's about getting interns and getting followers and fans without figuring out why a medium is important and how to make it successful for them.

RWW: What's one surprise - good or bad - that you've come across since you started working with Kiva? What did you not expect from this experience, and what did you learn?

I learned that it's much more complicated than the website makes it seem. There's an entire division devoted to foreign exchange currency. The operational cost analysis, the challenges of technology in the developing world, the processes of remittance - it's incredibly complex. There are regional specialists. On the site, you can make a loan in five clicks, but a lot of machinery comes together to make it that way.

RWW: What's next for you? Is there more globe-trotting in your immediate future? How do you think the web will continue to be part of your life and career?

One of the best parts of this past year has been that I've gone through long periods where I didn't have Internet access. That's brought me a heightened and renewed sense of my purpose in the world and my authentic desire to make the world a better place. I'd like to be able to continue to support campaigns - even for-profit ventures - that I believe in, and I think social business is a wonderful intersection of the two.

I want to explore avenues with online and offline components, while continuing to blog and tell stories I'm passionate about.

Follow Berrent's next adventures on Twitter or at her blog.

And all this is just the tip of the iceburg that is Sloane Berrent's fascinating story. For a fuller look at her travels and timeline, check out this list of her nine favorite posts on her blog, The Causemopolitan, covering humanitarianism, her work in New Orleans, the phenomenon of serendipity in international travel, and much more.

Many thanks to Sloane Berrent for the use of her videos and images as well as for sharing her story with us and our readers.

Tim O'Reilly is a publisher of technical books, the organizer of a series of conferences on diverse topics, an investor in web startup companies and smart electrical grid technologies. He's credited with shepharding the term Web 2.0 into public consciousness and he regularly uses his extensive influence to call on technologists to "do something worthy," especially in the face of ecological and political crisis. Now he's brokering meetings of Obama administration officials and bleeding-edge geeks.

Sponsor

"What I learned when I went to Washington," O'Reilly told me by phone as he drove down a California highway earlier this month, "was how much the dialogue is determined by the companies that go there." O'Reilly is a man in the habit of helping determine dialogue around important issues and the opportunity the Obama administration offers to change government is no different. "[Google CEO] Eric Schmidt told me - 'tell a big story - talk to people and then share what you've learned.'"

O'Reilly is talking to people, but he's helping people talk to eachother as well. He's introducing officials like Vivek Kundra, the new CIO of the Federal government, and Federal CTO Aneesh Chopra to ground-breaking hackers like geek rennaisance man Chris Messina and YCombinator founder Paul Graham. He's bringing together geospatial visionaries and the government officials that provide them the GPS data they work with.

"What I've learned from all these conversations," O'Reilly says,"is about government as a platform. It's not just social media use by government, or government using wikis. No, it's something more profound. How do you think like a platform provider? We've moved our government from a lean vehicle for collective action, and over the last 200 years it has become so strong that it's now 40% of GDP. I want to go back to the original vision of the role of government: a convener of things that we as individuals and companies can't do alone. Standard setting, pilot programs; government providing enabling technologies for citizens to serve themselves.

"This morning we did a call with the White House and some geohackers, talking about what's wrong with government geodata now and how could it be fixed. The government people said we need to translate this into real projects that will appeal to politicians. 'If you fix this kind of geodata then we'll be able to provide this service - street safety, education attainment, public policy objectives,' was what they wanted to hear from the hackers. It's really about social innovation, building better tools for us as a nation to use technology to focus on real problems."

Healthcare, education and innovation policy are the three sectors O'Reilly says have the most momentum when it comes to government as platform.

"The old model," O'Reilly argues, "said we'll build services ourselves or we'll make deals with a few prefered providers that we'll then offer to our customers. This is very similar to what we saw recently in the cell phone market. Rather than providing all the apps themselves, Apple provided a platform and said to developers 'go build on it.' That's where I think the government is trying to go. Instead of offering a website, here's an API [application programming interface]. Can we spark innovation against what we're doing? It's not about picking a provider or partner and then your conduit to the private sector is them, instead its about evangelizing your platform so far more people develop on top of it."

"There are absolutely other companies coming to Washington and saying otherwise, to stick with old model," he says,"but there's an opportunity for government to say if people want to build services on this then we need the data we make public to be granular and timely. We should not be publishing updates once a month. Real time, local, responsive to users - those are new thinking for government. It's just like the 90's when government was discovering websites, now they are discovering web services and we're saying this is what they need to look like."

That conversation will become very public when O'Reilly hosts the Gov 2.0 conference next month. The lineup of geeks and people from the government is already intruiging and O'Reilley has said that some of the holes in the schedule are placeholders for very high-profile speakers who haven't yet sent final confirmation.

"Vivek [Kundra, US CIO] says he wants to make working for government sexy," O'Reilly says. "It's a huge part of our economy and there's a lot of opportunity for entreprenuers. Why are we letting beltway bandits get away with overchanging government to do work? We're missing opportunities to get our best thinking into government planning."

Making work for the government sexy is going to be a very big challenge. If there's a person and a paradigm that just might be able to do it, though, Tim O'Reilly and this vision of "government as platform" might be the right combination.

Photo: O'Reilly at Web 2.0 Expo Berlin, by Adam Tinworth

In Part 2 we discuss: how previously reticent search engines like Google and Yahoo have begun to participate in the Semantic Web in 2009, user interfaces for browsing and using data, what Tim Berners-Lee thinks of new computational engine Wolfram Alpha, how e-commerce vendors are moving into the Linked Data world, and finally how the Internet of Things intersects with the Semantic Web.

Sponsor

Semantic Web and Search Engines Like Google, Yahoo

RWW: You've been talking about the Semantic Web for many years now. Generally the view is that Semantic Web is great in theory, but we're still not seeing a large number of commercial web apps that use RDF (we've seen a number of scientific or academic ones). However we have begun to see some traction with RDFa (embedding RDF metadata into XHTML Web content), for example Google's Rich Snippets and Yahoo's SearchMonkey. Has the takeup of RDFa taken you by surprise?

TBL: Not really, but the takeup by the search engines is interesting. In a way I was happy to see that, it was a milestone for those things to come out of the search engines. The search engines had typically not been keen on the Semantic Web - maybe you could argue that their business is making order out of chaos, and they're actually happy with the chaos. And if you provide them with the order, they don't immediately see the use of it.

Also I think there was misunderstanding in the search engine industry that the Semantic Web meant metadata, and metadata meant keywords, and keywords don't work because people lie. Because traditionally in information retrieval systems, keywords haven't proven up to the task of finding stuff on the Web. One of the reasons is that people lie, the other is that they can't be bothered to enter keywords. So keywords have gotten a bad reputation, then metadata in general was tarred with this 'keywords don't work' brush. Because a lot of Semantic Web data included metadata, then people thought that with Semantic Web data -- again, that people will lie and won't have the time to produce it.

Google rich snippets example; image credit: Matt Cutts

Now I think there's a realization that when you're putting data online, that people are motivated NOT to lie. For example when your band is going to produce its next album, or when your band is going to play next downtown, you're motivated to put that information up there on the Semantic Web. There's an awful lot of cases when actually data is really important to people; and it's on the web anyway. So I think it's great that some of the search engine companies are starting to read RDFa.

Does this mean that they [search engines] will start to absorb the whole RDF data model? If they do, then they will be able to start pulling all of the linked data cloud in.

Will they know what to do with it? Because when it's data in a very organized form, I think some people have been misunderstanding the Semantic Web as being something that tries to make a better search engine - i.e. when you type something into a little box. But of course the great thing about the Semantic Web is that you can query it, you can ask a complicated query of the Semantic Web, like a SQL query (we call it a SPARQL query), and that's such a different thing to be able to do. It really doesn't compare to a search engine.

You've got search for text phrases on one side (which is a useful tool) and querying of the data on the other. I think that those things will connect together a lot.

So I think people will search using a search text engine, and find a webpage. On the front of the webpage they'll find a link to some data, then they'll browse with a data browser, then they'll find a pattern which is really interesting, then they'll make their data system go and find all the things which are like that pattern (which is actually doing a query, but they'll not realize it), then they'll be in data mode with tables and doing statistical analysis, and in that statistical analysis they'll find an interesting object which has a home page, and they'll click on that, and go to a homepage and be back on the Web again.

So the web of linked data and the web of documents actually connect in both directions, with links.

User Interfaces for Semantic Content

RWW: At the recent SemTech conference, Tom Tague of Thomson Reuters' Calais project suggested that user interfaces for semantic content are key in getting more take-up. With that in mind, I wonder if you've seen some great interfaces or designs for semantic applications in recent months - if so which ones and why did they impress you?

TBL: I think that whole area is very exciting at the moment. The only piece of hacking I've done over the past few years has been on a thing called the Tabulator [a data browser and editor], which is addressing exactly that. Partly because I wanted to be able to look at this data. And now there are lots of different ways that people need to be able to look at data. You need to be able to browse through it piece by piece, exploring the world of data. You need to be able to look for patterns of particular things that have happened. Because this is data, we need to be able to use all of the power that traditionally we've used for data. When I've pulled in my chosen data set, using a query, I want to be able to do [things like] maps, graphs, analysis, and statistical stuff.

W3C Tabulator, a data browser/editor; Image credit: wiwiss.fu-berlin.de

So when you talk about user interfaces for this, it's really very very broad. Yes I think it's important. There's also the distinction we can make between the generic interfaces and the specific interfaces.

There will always be specific interfaces; for example if you're looking at calendar data, there's nothing else like a calendar that understands weeks, months and years. If you're looking at a genome, it's good to have a genetics-specific user interface.

However you also need to be able to connect that data, through generic interfaces. So if my genome data was taken during an experiment which happened over a particular period, I need to be able to look at that in the calendar - so I can connect the genetics to the calendar.

So one of the things I hope to see is domain-specific things for various different domains, and the generic user interfaces. And hopefully the generic interfaces will be able to tie together all of the domains.

Next Page: Wolfram Alpha; e-Commerce and Linked Data