JISC PoWR

Preservation of Web Resources: a JISC-funded project [Archived Blog]

Some Use Cases For Preserving Twitter Posts

Posted by Brian Kelly on June 26th, 2009

I’ve come across two schools of thought regarding the preservation of Twitter posts (tweets). From Twitter fans, it’s a really useful form of informal conversation, with a particular strength being the way in which messages quickly disappear. It’s like having conversations in your favourite bar – and we wouldn’t want such conversations preserved, so why would we wish to preserve tweets? Sceptics of Twitter might regard Twitter as inane babble in a bar and would agree that it’s pointless to preserve it.

However this week I have come across a couple of examples which illustrate why it might be useful to preserve tweets.

Preservation of Twitter data for research purposes

I recently described how Twitterers Subvert[ed] Daily Mail’s Racist Poll. For me that provided a fascinating example of how Twitter can be used by activists to support a viral campaign (in this case to undermine the racist sentiments in the question “Should the NHS allow gipsies to jump the queue?“). But why did this poll succeed in attract a large turnout (with the poll being undermined with around 94% voting yes before the Daily Mail removed the poll) whereas a subsequent poll on Should immigrants be forced to respect British culture? failed to arouse much interest?

Since many of the tweets and retweets provided a link to “http://bit.ly/w4b6Q” we can look at the statistics provided by the bit.ly short URL service which tells us that 48,796 clicked on the link.

Statistics for clicks to Daily Mail pollAs shown in the histogram as might be expected the vast majority of clicks to the poll took place on 19 June 2009, the day the poll was the hot topic.

But who created the interest in the Twitterverse originally? Was it, as I speculated in my post, the UK-based psychologist community? Or where there a number of starting points? How was the message communicated? Who where the ‘influencers’ (to use the terminology from a recent blog post by Martin Weller) whose tweets were felt to contain links worth following? Who were the ‘hubs’ who were responsible for communicated the message to a wide audience? And was Twitter actually the main driver or were their other viral mechanisms in operation as Josh suggested, who went on to add “we must keep perspective and not neglect other areas or over-celebrate Twitter as a comms tool!“.

In a recent post by Andy Powell on “Influence, connections and outputs” Andy  suggested that we want “some sensible measure of scholarly impact on the social Web” and that we need to understand”what do we want to measure; what we can measure and how can bring these two things close enough together to create something useful”. Although I’m not suggesting that the Twitter responses to the Daily Mail poll have any scholarly impact, I do think that this could provide us with an opportunity to understand how ideas (whether scholarly, political or simply marking) are communicated and transmitted by Twitter. But if we wish to make use of this particular example, we will need to preserve the Twitter stream – and we will need to preserve the structured data (the Twitterer ID, the time, the IDs of those being retweeted and the numbers of followers) in order to carry out any rich analysis.

Preservation of an organisation’s memory

As recently described on the This is Bath blog the Bus station Twitter project ends – today. The bathcsc Twitter account and accompanying blog, were set up to keep users of the First bus service in Bath informed of news – including delays to bus services. Sadly First’s head office in Bath  have announced that the company “is having to make changes to its business structure. As part of making the company more cost efficient, the customer services function for Bath Bus Station will now be handled by a specialist, area-wide centre based at Exeter“.

This news has not gone down well with many of the fans of this innovation use of social networking tools (and I should add that, as a daily bus user, I personally found the Twitter service very useful) as can be gauged from the comments on the blog to the announcement of the demise of the service and on the Bus station Twitter project ends article.

In the article a spokeswoman said that the company was hoping to learn lessons from the nine-month experiment:

The Bath Twitter trial attracted an enormous amount of interest, both inside and outside the organisation. We are now looking at the lessons we have learnt, and benefits associated with the system, with a view to developing a national social media approach to customer services for the future.

But is the company in a position to preserve the tweets? Will the company be in a position to appreciate the ways in which Twitter can be used effectively if this record is lost? The danger, of course, is that a new media markeing agency wil sometime in the future advise on how Twitter could be used – and fail to learn from the successes of the initial trial, perhaps even making as big a mistake as Habitat have done recently.

What To Do? When To Do It?

Are these illustrations of when preservation of Twitter tweets reasonable? If so, can you think of other examples which we need to consider?

And if there is a case to be made for preserving tweets, what tools should be used? I am aware of the Print Your Twitter service, but this just creates a PDF file of tweets matching the search criteria.  To summarise what I think is needed:

  •  Ability to create a structured repository of tweets from a Twitter ID
  •  Ability to create a structured repository of tweets matching a search term

Anyone know of software which can be used to do this? I’m aware that one could simply use Twitter to search for “http://bit.ly/w4b6Q” or “@bathcsc” but this doesn’t provide a persistent and reusable data store.

10 Responses to “Some Use Cases For Preserving Twitter Posts”

  1. Josie Fraser Says:

    Been struggling with the archiving issue for a while, mainly in order to capture outputs from events. The best solution I’ve found so far is to set up a Friendfeed room, with keyword or hashtag twitter search. Ideally do this in advance.

  2. Brian Kelly Says:

    Hi Josie
    The use case for preserving tweets for an event is another good example. I mentioned this is a post on Preserving Tweets: The CILIP2 Experience – but should have included this case in this post.

  3. Joss Winn Says:

    I pull all my Tweets and any other social web activity into my WordPress blog using the Lifestream plugin. http://wordpress.org/extend/plugins/lifestream/ See here: http://blog.josswinn.org/stream/ All activity is ‘archived’ in the WordPress MySQL database and also available as an RSS feed to syndicate elsewhere: http://blog.josswinn.org/feed/?feed=lifestream-feed If you want to also make the stream searchable, you can publish your daily activity as a blog post and it will then be searchable using the normal WordPress search box.

    I’ve got to say though, I’ve no interest in the preservation of my own tweets. I do have an interest in archiving Google Reader shared links and Delicious bookmarks.

  4. Jim Richardson Says:

    See the 24 March 2009 post on Twitter ephemerality by @shifted for an earlier discussion of use-cases and methods for tweet preservation.

    There’s a wealth of material associated with conferences worth preserving. Beyond that, and with greater impact, the history of #iranelection and potentially further public events will be more difficult to write if a Twitter archive is not preserved.

  5. Richard M. Davis Says:

    Picking up on Joss’s comment, a WordPress blog is indeed a very effective store for Tweets, and the idea also occurs within our ArchivePress project. I haven’t looked at the Lifestream plugin yet, but I have used FeedWordPress to pull in Tweets from RSS feeds to WordPress extremely effectively (I discovered a few weeks ago that I had been inadvertently saving all my Tweets in a forgotten WordPress instance.)

    An institutional archive of anything put out by newsfeed – tweet, blog post, wiki update… – and based on event-driven updating, rather than arbitrary crawling/backup intervals – sounds like a very worthwhile system and resource to me; and we will be looking into this for ArchivePress.

  6. AJ Cann Says:

    http://wthashtag.com/ is an interesting alternative to a FriendFeed group for archiving tweets.

  7. Mark Bockenstedt Says:

    As AJ just pointed out, What the Hashtag is a useful way of archiving tweets with the caveat that they contain hashtags. Our unique transcript export feature allows you to query our tweet cache for a hashtag over a given timeframe and view (and save or print) a list of everyone who used that hashtag. This is especially popular for chat-style hashtags so people who missed the event can go back and recap what was said without having to page back and forth.

    It’s one of our most popular features so I’d say that archiving tweets is a pretty standard practice. It’s also much easier than setting up a FriendFeed room/group/imaginary friend :) .

  8. JISC-PoWR » Blog Archive » Tools For Preserving Twitter Posts Says:

    [...] (RSS) « Some Use Cases For Preserving Twitter Posts Preservation Policies for Content Hosted on Third Party Blogs [...]

  9. Charles Bernardo Says:

    Just get some free Twitter apps here and use as you need to get more followers and don’t tweet things that you don’t want to live with forever is always good advice, many free links here at
    http://www.thetwitterstore.com

    And lighten up, no one will care 50 years from now if you tweeted you were drunk or what you were eating.

  10. Jake Gomez Says:

    i always update my Twitter and i love to twitter my daily activities to my friends and loved ones. i also maintain a personal blog for entries which requires more detail.