That was the message Vivek Kundra, the first federal Chief Information Officer in the Obama administration, brought to the Online News Association conference in Boston this morning.

On data.gov, there is information on a huge range of government activity but there hasn’t been a big pickup, he said. Journalists need to adapt themselves to tell the stories buried in digital data. Some news organizations are innovating, he added. But others are ignoring change at their peril.

“There is a battle for the soul of the future of journalism,” he said. “Do you want to be Amazon or Barnes and Noble?”

The data miners will be the ones telling some of the best stories, he said. He added that reporters need to access the incredible volume of information governments collect about everyday life.

“If you look at what ‘s happening in the world today, everything is being instrumented” — from the times buses are running to wait times in hospitals, he said.

An emerging role for journalists is taking this data and separating noise from signal.

He said in his work in the Obama government, he tried to push departments to put their information in the cloud.  He identified about US$20 billion in services that could move to cloud but  he faced tough resistance from the “old guard,” which were protective of existing — hidden from the public — data storage and management practices.

Governments, he said, should embrace private cloud services providers for data because it’s cheaper and government can’t compete for the expertise of top-notch security engineers.

She cited seven reasons for optimism in journalism today:

1) “Conditions are favorable for online paywalls.” The New York Times launched its metered access strategy this week, betting that loyal readers will pay. “Scale is still important but brand is back.”

2) “Local is still up for grabs.” Hyper-local experiments such as the Texas Tribune and EveryBlock are showing that people are interested in local.

3) Twitter’s growing power as a newsgathering tool”. This is evident even more so in the last few weeks with political change in North Africa.

4) “Apps are the holy grail of engagement.” NPR app users are NPR loyalists. Let’s look more closely at the behavior of these app users

5) “The web is not dead.” New NPR users come from the web via npr.org.

6) “Legacy news organizations are finally ready to be disruptors.” Platform agnosticism is “nonsense.” Yes, news organization have to everywhere. But the idea they should put the same content everywhere doesn’t work.

7) “Digital natives have come of age & they care about journalism.” Media literacy is so important for these young people.

Jenkins, a journalism and cinematic arts professor at the University of Southern California, makes a blunt statement about the relevance of news in the social media age: “If it doesn’t spread, it’s dead.”

Obvious? Perhaps. His point is that news organizations need to better understand how — and why — people take content and re-package it for others. He likens the transaction to a bottle of wine you buy at a store then give to your dinner host. You might put the wine into a decorative bag and tell the cook a story about how you thought of her when you chose that label and vintage.

We bought it as a commodity, we give it as a gift, and the moment of transformation comes when we remove the price tag. We need to better understand the same transformation as consumers take content from commercial sites and circulate it via Twitter or Facebook to their communities.

When people spread content to their social networks, they edit it and add comments to frame it for a specific social purpose. Others might re-mix the content and mash it up — think the Donald Duck and Glenn Beck mashup and, in Canada, Rick Mercer’s Photo Challenge. Sure, these are lighthearted examples. But as Jenkins points out, people make sharing decisions based on the “social or sentimental value of the content.”

I’m reminded of the New York Times’ excellent interactive last week “Budget Puzzle: You Fix the Budget.” It didn’t provide the tools for a mashup, per se. But it promoted participation — a social experience that people could share with their friends, something people apparently did on Twitter more than 11,000 times.

Jenkins argues news organizations have to be able to meet people in their conversations: ”Journalists need to know how they fit into those circuits.”

However, according to Mathew Ingram, even on new platforms such as the iPad, news organizations are lagging in promoting sharable content. Publishers, he argues, seem to be “hoping that you will forget all about the Internet and social media and all of those irritating things that get in between you and the consumption of their wonderful content.”

Jenkins refers to a “constant tension” in the news business between “meter access” and “spreadability.” We can see it in the New York Times’ persistent plan to erect a metered paywall in early 2011, even while being fully cognizant of the power of social media. Jenkins would seem to be skeptical of the Times’ chances in restraining the social nature of the Internet:

News sites which prevent the sharing of such content amongst readers may look like ways to protect the commercial interest of that content, but in fact, they kill it, destroying its value as a cultural resource within networked communities, and insuring that the public will look elsewhere for news that can be spread.”

What to do for the rest of us? A good starting point might be Rohit Bhargava’s 5 New Rules Of Social Media Optimization, which offers tools for getting your content more frequently included social media conversations.

There are two elements worth noting:

1. It sets out a standard for publishing information gleaned from social media that news staff may not be able to verify. In “exceptional circumstances” governed by “timeliness or if it is in the public interest,” editorial staff may publish user-generated content such as tweets, Facebook posts, YouTube videos or photos. The kicker is that “we are clear with the audience about what we know.” This is an important acknowledgement that the CBC will be visible when important events emerge from social media in real-time, without compromising the news organization’s commitment to verification. It suggests that events can happen so quickly in social media that journalists sometimes need to make their process of verification public — a conclusion contained in a set of guidelines I co-authored for the Canadian Association of Journalists on “Re-Tweeting or Re-Posting Information Found In Social Media.”
2. It makes a statement about personal use of social media, acknowledging the danger that “personal opinions on controversial subjects” can “erode the trust of our audience.” True enough. However, it states that personal social media activity must conform to the CBC’s social media principles, whose guiding rule is not to publish anything in social media that wouldn’t go on the air or the news site. This is simply too restrictive and doesn’t acknowledge how people actually use social media — especially in their personal lives. Applying the “publishing” standard to social media use has the effect of hammering down whimsy, spontaneity and experimentation, in short: personality — the key driver of “social” in social media.

Here’s the issue in a nutshell: the very first statement of social media principles declares that Twitter, Facebook or flickr “can be useful tools for gathering information, as well as disseminating it.” What about sharing, engaging and receiving feedback? “Dissemination” sounds like 20th-century broadcasting. “Gathering” sounds like “taking.”

According to Esther Enkin, executive editor of CBC News, this document took “nearly two years of work.” In some ways, the social media section feels like it was authored in 2008. It lacks practical guidelines for the kind of interacting it is encouraging its reporters to take on.

In all, there is much to admire here and Enkin’s commitment to making this a “living document” offers the promise of adapting the document as social media practices evolve.

If you’re a journalist over 35, an employer likely told you not to post campaign signs on your lawn, attend public rallies or sport bumper stickers. The advice was meant to portray you — and your employer — as independent and without bias.

But online media challenge that perspective. Social media advocates argue that trying to separate one’s personal and professional lives online is a lost cause. Journalist and community engagement advocate Steve Buttry argues: “Most of the Twitter world mixes business with pleasure. Building walls means you won’t understand how Twitter works.”

Others argue that revealing one’s assumptions and values publicly is a badge of integrity. According to media researcher and consultant David Weinberger, “transparency is the new objectivity.”

Can the perception of objectivity be achieved in the age of social media?

• What guidelines should journalists follow in maintaining personal blogs?
• Should they care who they friend or follow on Facebook and Twitter?
• What about joining social media groups or accepting badges for participation?

What do you think?

The Guardian was the first news organization to really harness the power of crowdsourced data with its MP expenses database.

Now it has the lofty goal of distributing its technology worldwide to expose wrongdoing.

The man who has the job of opening up The Guardian‘s data technology told an audience at the Mesh technology conference in Toronto this morning: “The plan is to make the API (Application Programming Interface) open source.”

“We’re also exploring the possibility of making it a data clearinghouse,” in essence, allowing other people to upload their own data.

Chris Thorpe, The Guardian’s Developer Advocate for its Open Platform, set out the grand goal of  helping the oppressed and the impoverished worldwide. “We want to heal the world,” he said. “Openness is the friend of good.”

In June 2009, The Guardian hired a programmer for a week and built a portal to distribute more than 400,000 government documents of MPs’ expenses. The project prompted thousands of people to join The Guardian’s reporting team in pouring through claims and highlighting ones for further investigation. It was a huge step forward in newsgathering.

The Guardian API aims “to shine light on corruption,” Thorpe said. ”Let’s be a world liberal voice.”

The audience pressed Thorpe to state The Guardian’s stance on the current trend toward news paywalls. To that, he said, ”The people putting up paywalls will really reduce their influence in the world.”

“We’re seeing good advertising revenue from being open,” he added, but refused to talk numbers.

He said The Guardian is starting to realize “We have to behave more like a newspaper” by increasing audience engagement with more in-depth content.

“What we’re finding is the more pages people view, the more people are likely to click on adverts,” he said.

The Halifax Chronicle Herald signalled today it’s going to beef up its commenting system to improve the level of discourse on its site. Good to hear. Much of what passes for comment on news websites continues to be the lowest possible level of name-calling and uninformed bravado.

The Herald’s director of news content Dan Leger states:

“So we’re working on ways to end abuse, partly through registration and software to encourage use of real names and identify abuse. It won’t be perfect, but perhaps we can temper the over-the-top attacks.”

The issue of weak commenting systems was highlighted nationally last Wednesday when Halifax weekly The Coast said it would surrender the IP addresses of commenters following a Nova Scotia Supreme Court order. It’s the first time a news site has done that in Canada. Two senior fire officials at the Halifax Regional Municipality had sought to unmask the identities of these commenters, alleging they defamed them by painting their behaviour as racist and incompetent.

The move resulted in many people online and on local radio imploring news organizations to adopt a policy requiring real names for commenters as a means of raising the quality of online conversation.

They wanted to hear from someone like Howard Owens, who argues a real names policy is practical and enforceable at his New York State news site The Batavian.

I check public databases for names that match in the zip code provided. If no match, the user is asked to provide either by fax, e-mail or in person a copy of a picture ID.

Owens, himself, calls his policy “a ‘best effort’ practice,” siding with the news executives Richard Pérez-Peña cited in his New York Times article last week who “say that merely making the demand for a name and an e-mail address would weed out much of the most offensive commentary.” He may be right.

But while this may work at smaller news organizations, it is likely to be unworkable at sites that get tens or hundreds of new registrations a day. Pérez-Peña says as much in the Times story, which details the growing unease of news organizations with anonymous comments.

Real names strikes me as an offline solution to an online problem.

I argued in an interview on CBC Radio’s Mainstreet Nova Scotia last week that the big problem with the comments areas on most news stories is they seem to be abandoned by the news site itself. The result is the so-called “broken windows” problem — it becomes a wasteland vacated by moderates who have been long-since shouted down by blow-hards.

Author/blogger Scott Rosenberg states: ”Show me a newspaper website without a comments host or moderation plan and I’ll show you a nasty flamepit that no unenforceable ‘use your real name’ policy can save.”

I’m much more intrigued by Gawker’s approach. It implemented a more robust software system last year that gave its staff and community a way to promote the “funniest, thoughtful, intelligent, well-argued” comments. It divided its community into two tiers:

• a small community of “starred” commenters “who have proven themselves to be engaged, intelligent, humorous, fair-minded, thoughtful, rational.” These people can promote “well-written, thought out, intelligent and/or otherwise notable comment” below stories.
• The rest, whose comments will be obscured behind a “Show all comments” link

A key aspect is that the discussion is guided by an engaged community with an interest in creating a place people want to be. In addition, the community is fluid, with new commenters rising to become stars, and people who abuse their star power falling to become relative nobodies.

Gawker Media CTO Tom Plunkett suggested in this graph last week, that the site recovered — and thrived —  from an initial drop in comments that resulted from stripping people’s status based on the number of followers they had — and making many comments more difficult to view. He concludes: “purging commenter accounts is not a solution for the out-of-control commenter community. Nor is a large moderation staff.”

Will this completely stop defamatory comments? Probably not. Editor moderation is likely the only way to do that. But it’s a bottom-up — not top-down— prescription that seems a better fit with the medium.

It live blogged 8 events this past summer during Nova Scotia’s provincial election. Seven were political events — party platform releases or debates — the other was a discussion of the Herald’s use of social media. The Herald used CoverItLive, the leading (free) live-blogging service based in Toronto.

Rick Conrad, editor of thechronicleherald.ca, calls it a “vital” means of engaging readers and its effect is evident in the quality of conversation: “You get a more defined audience – almost a more educated one – because it’s something they are interested in.” He acknowledges the events attracted the occasional “crank” but “99% of the people were there for the right reasons and wanted to have a real discussion.”

Some tips from the Herald’s experience:

• Promote, promote, promote: People need advance notice to schedule a live blog into their day. Conrad promoted them on the site, in print, and on Twitter. The result was some success in expanding the site’s audience: “For the social media blog, I got the distinct sense it attracted people who wouldn’t normally think of picking us up every day or coming to us online.”
• It takes two: If you’re covering an event, you generally need at least two people – one to report and the other to engage the audience. Says Conrad: “With the platform releases, [the audience] would come with a lot interesting — and tough — questions like “What does this mean?” When you’re listening to what the leader is saying and trying to go through the documents, it’s difficult to make a snap analysis.” Conrad suggests bringing a reporter along, if you can. He concedes it’s difficult because a reporter frequently has to file a story as well. But he adds, “Readers really like to talk to a person directly involved in the story.”
• Let the audience lead: When to drive the conversation and when not to? It depends on the topic, says Conrad. If the audience is getting information from other sources it often works best to let them lead the conversation. “With the election night blog, we essentially took the approach that ‘this is their night’ and we would pop in with the occasional comment on the results … Whereas with the platform releases, I found there was more interaction with me because I was their eyes and ears.”
• … but keep it civil: Conrad says he approved the vast majority of comments, but rejected a few: “For me, the reason for doing the live blogs for the election was to give as many people as possible the chance to comment.” However, a few were “nasty” and wildly speculative. “We didn’t think that was appropriate. You want [the conversation] to be substantial and you don’t want it to spiral down.”
• Use it to assess your own efforts: People are especially engaged when they feel they can make a difference. The Herald received almost two comments for every person who attended the session in which it asked how it could make better use of social media. While Conrad acknowledges the difficulty in drawing conclusions from raw numbers, he says the tone of the conversation was spirited. “For me, the people who were involved in that blog were really into it.”
• Keep it available for replay: The number of people who replayed the Herald’s live blog the day following election night was almost twice the number who participated live. Some people don’t have time to follow a live stream — or aren’t available when it’s scheduled. But many are interested in seeing which topics resonated with the audience.

Conrad says he would like to live blog a provincial budget release and he hopes to experiment with the tool at a concert or a live arts event.

Google Fast Flip interface for mobile.

So this is it.

In April, Sharon Waxman reported a conversation with Google CEO Eric Schmidt in which he told her:

“In about six months, the company will roll out a system that will bring high-quality news content to users without them actively looking for it” … one in which users “will be automatically served the kind of news that interests them” … and which Google “will be able to sell premium ads against premium content.”

Google Fast Flip appears to be all that.

The company claims the visual news finding tool “personalizes the experience for you, by taking cues from selections you make to show you more content from sources, topics and journalists that you seem to like.”

Make no mistake — this is an application built from the ground up for mobile. (In addition to being available in Google Labs on the web, Google offers it as a mobile site for iPhone and Android devices.)

Reading stories from multiple sources isn’t easy to do on a mobile device right now. Neither is search. Done right, personalized, visual news could truly be a killer app.

As Scott Karp says, it’s “a new UI (user interface) for news.” Further, it’s Google re-asserting its claim to the browser, not the app, as the viewing space for mobile content.

Martin Nisenholtz, senior vice president of digital operations at the New York Times told Ken Doctor, “We’re trying to find out how users relate to a visual interface.” There’s certainly much to learn — and there are quirks with Fast Flip. But, the interface, with a firm nod to Apple’s Cover-Flow, is an excellent fit for mobile devices.

That’s great for consumers. And, as Nisenholtz points out, it’s the brand presence news publishers have wanted. But it also gives Google the upper hand in yet another news viewing space.

Colour-coded lines show traffic congestion in Google Maps for Mobile.

“It’s stop and go on the bridge.” “Watch out for that deer on the 102.”

The voices are a staple of local radio — listeners calling in to report traffic conditions. On-air updates build audiences, involve listeners and bring advertisers to morning and afternoon shows.

But Google believes it can deliver traffic reports faster and with more accuracy by harnessing the power of mobile users.

Google announced today it was bringing the power of crowd-sourced data to its Traffic layer in Google Maps, currently available only in the U.S.

Users of Google Maps for Mobile have to allow Google to see their location first. But when they do, they contribute traffic data anonymously and effortlessly as they travel. Their GPS-enabled mobile phone contributes their location and aggregates it with others to form a live picture of traffic in the area. Its announcement states:

When we combine your speed with the speed of other phones on the road, across thousands of phones moving around a city at any given time, we can get a pretty good picture of live traffic conditions. We continuously combine this data and send it back to you for free in the Google Maps traffic layers.

Google also announced it was expanding its Traffic layer to cover all U.S. highways and arterial roads.

Google has the leading mobile map app. While it’s competing with standalone GPS providers such as TomTom and Garmin in providing driving assistance, the division between smartphones and standalone devices is already starting to fall. TomTom released its turn-by-turn maps as an iPhone app 10 days ago — the first company to do so.

Google’s move is yet another grab at services traditionally provided by news media — services that bring revenues to support local newsrooms. While newspapers have had their audiences slashed by aggregators of news and classified listings, radio has remained relatively unscathed by advances in online technology.

The move puts radio producers on notice that they too need to find other ways of strengthening listener interaction.