JISC PoWR

Preservation of Web Resources: a JISC-funded project [Archived Blog]

Tools For Preserving Twitter Posts

Posted by Brian Kelly on July 14th, 2009

I recently described some Some Use Cases For Preserving Twitter Posts including preservation of an organisation’s digital memory and preservation of data for subsequent data mining. The post, however, failed to include perhaps the most obvious example: preservation of Twitter posts (tweets) related to an event.

In response to that post a number of solutions for preserving tweets were suggested including FriendFeed, the WordPress Lifestream plugin and What the Hashtag. In addition following a tweet I posted I received details of the Tweetdoc service.

With this year’s Institutional Web Management Workshop (IWMW 2009) rapidly approaching it is timely to decide the tools we’ll be using to preserve the discussions associated with this event. We began keeping a record of the amplification of the IWMW event back in 2005 when an IRC channel was provided for use by the small numbers of participants who had a laptop and WiFi connectivity at the event. The IWMW 2005 event took place during the 7/7 bombings in London and a record of the awareness of what was having can be gleaned from the archive of the IRC discussions.

This year we will once again be making use of Twitter and will be capturing tweets which contain the event hashtag #iwmw2009.  The tools we are currently intending to use are What the Hashtag and Tweetdoc service.

Use of wthashtags service to view Twitter posts about #iwmw2009Use of What the Hashtag to views tweets containing the #iwmw2009 tag is illustrated.

As can be seen the output can be access as an RSS feed. In addition the data can be viewed as an HTML resource, and a data range can also be supplied.

We intend to capture this tweets about the IWMW 2009 event after the event is over, and store the data on the UKOLN Web site, in order to avoid dependencies on the What the Hashtag service itself.

We will also explore other services, such as Tweetdoc – although in this case as the data is only available as a PDF resource, it is not well-suited to provide data for analysis by other services.

Are there any other services we should be looking at? And what functionality might be desirable for a more generic service for preserving tweet? Any thoughts?


Note: This blog post has been written to support a poster which will be provided for the Missing links: the enduring web conference. The poster, which has been produced by Marieke Guy and Brian Kelly, UKOLN, is entitled “Preservation Policies and Approaches for Use of Web 2.0 Services“. A series of blog posts published on this blog provide more detailed information of the content summarised in the poster.

8 Responses to “Tools For Preserving Twitter Posts”

  1. Ingmar Says:

    I really like the archivist. It saves tweets in an xml-file.

  2. Chris Rusbridge Says:

    I’ve saved tweets for a few events by the crude approach of doing a Twitter search and cut/pasting into a Word file. It’s a pain as you have to repeat for each page of results. Unfortunately if you do it a few days after the event, you only get approximate timings, and of course it’s in reverse chronological order.

    Surely it’s not beyond the wit of your audience to come up with a better solution (;-)?

  3. PeteJ Says:

    Your storing tagged Twitter posts outside of the Twitter system raises some interesting questions of data ownership, I think.

    The Twitter ToS say:

    “Your profile and materials uploaded remain yours. You can remove your profile at any time by deleting your account. This will also remove any text and images you have stored in the system.”

    Given this, I think, strictly speaking, you probably need to make it clear to anyone using an IWMW hashtag on Twitter that their tagged posts will be stored outside of the Twitter system, and even if they delete their Twitter account, you will retain copies of their content.

    I suppose you may also need to consider the possibility that someone may request the deletion of stored posts? Given the clause above, it seems to me they would be quite within their rights to do so. Using a hashtag on Twitter provides a means of retrieving content, but it shouldn’t change people’s rights over their content.

    I think this is an example of where making use of an existing service like Twitter with its own ToS differs from the case where you set up your own service for the event and you get to set the terms and conditions, and people get to accept them (or not) as they enrol for that service.

  4. Brian Kelly Says:

    Hi Pete
    Thanks for your comments – you have raised some interesting issues. We have already
    published details of how we intend to use Twitter. I have updated this to say that we will try to delete tweets if the owner wishes this do be done. However, as you know, this may be problematic if the tweets are processed by a variety of tools. I know that Eduserv pull tweets into the Coveritlive services as you did for the
    JISC Data Management Infrastructure Briefing Day. I’d be interested to know how you would handle a request to delete such Twitter posts.

  5. PeteJ Says:

    :-)

    Right now I don’t know how “we” would handle it, and I don’t use CIL enough to know what the options for editing are.

    But (imo) the same points I made above apply, and “we” should be prepared for that contingency.

  6. Jim Till Says:

    may be useful for preserving transcripts containing a series of tweets, but what about the preservation of individual tweets? So far, doesn’t seem to be able to archive tweets, but does archive the URLs for tweets. An example: the URL has been archibed at .

  7. Jim Till Says:

    How to preserve of individual tweets?

    WebCite provides a way of archiving URLs. However, it doesn’t
    doesn’t seem to work for tweets. It does work for the FriendFeed URLs for tweets.

  8. Jim Till Says:

    My apologies for messing up these comments. –Jim