An Archived blog page is now available giving further information on the archiving of the blog. It includes blog statistics for future reference.

The JISC PoWR team would like to say thank you to all our readers. Most of the team members are involved in new digital/Web preservation work so this won’t be the last you hear from us!

It is now therefore timely to officially announce that we do not intend to publish any new posts on the blog after a couple of post which provide a summary of how this blog was used.  A week or so after the final posts have been published we will switch off comments on the blog – so that we will no longer have to spend time in checking for spam comments.

The blog itself, and all posts and comments, will remain available for the indefinite future – by which we mean that we will seek to provide access for a period of at least 3 years from now.

The summary posts we intend to provide will contain details about the blog such as:

• Number of posts and comments
• Details of contributors
• Details of blog theme and plugins used
• Details of type and version of software used

If you have any suggestions for any other information it would be useful to provide and record please do let us know.

We intend to use the closing of the blog as a case study which will be documented as part of the JISC Beginner’s Guide to Digital Preservation. The Beginner’s Guide will eventually be available online but the process of creating the guide is being documented in the JISC Beginner’s Guide to Digital Preservation blog.

This Guide uses similar content to PoWR: The Preservation of Web Resources Handbook but in a way which provides a practical guide to web preservation, particularly for web and records managers. The chapters are set out in a logical sequence and answer the questions which might be raised when web preservation is being seriously considered by an institution. These are:

• What is preservation?
• What are web resources?
• Why do I have to preserve them?
• What is a web preservation programme?
• How do I decide what to preserve?
• How do I capture them?
• Who should be involved?
• What approaches should I take?
• What policies need to be developed?

Each chapter concludes with a set of actions and one chapter lists the tasks which must be carried out, and the timings of these tasks, if an institution is to develop and maintain a web preservation programme. In addition points made in the Guide are illustrated with a number of case studies.

The guide was edited by Susan Farrell who has used her knowledge and expertise in the management of large-scale institutional Web services in writing the document.

The Guide can be downloaded (in PDF format) from the JISC PoWR Web site. The Guide is also hosted on JISCPress service which provides a commenting and annotation capability. It has been published on the Lulu.com print-on-demand service where it can be bought for £2.82 plus postage and packing.

If you want to discuss the Guide on Twitter you should use the #jiscpowr tag.

Till recently JISC Involve was running on an old version of WordPress (1.2.5). Earlier this month the JISC Digital Communications Team upgraded their server to the latest version of WordPress (2.9.2) and then migrated all the JISC Involve’s blogs over to the new installation.

Although all blog posts, comments, attachments, user accounts, permissions and customisations were supposed to move over easily JISC Involve users were encouraged to back-up the content of drafts etc. ‘just in case’.

Unfortunately there were some technical problems migrating the content and as a consequence the original theme was lost and URLs now redirect.

Luckily the JISC PoWR team were able to locate the original theme and reinstall it.

However the process has made them aware of the need to record details of the technical components and architecture of the blog. This information can be critical in a migration process and when ‘closing down’ a blog.

The JISC PoWR team will ensure that such information is routinely recorded.

Is there any other information that is important for preservation or migration purposes?

It will look at reasons why JISC projects might want to preserve their deliverables, will introduce mainstream terminology and processes and offer clearcut solutions. The guide will also offer lists of references and resources, a checklist of issues users will need to think about and a number of case-studies by which they will be able to benchmark themselves against.

A number of the discussions initiated on the JISC PoWR blog (such as preservation of Web 2.0 services including blogs and wikis) will be taken forward on the new project.

A project blog has recently been launched at http://blogs.ukoln.ac.uk/jisc-bgdp/

The announcement was fittingly made on Twitter.

Yesterday the Library of Congress blog published a list of FAQs abouut the approach they will be taking.

The FAQ explains:

• Why is it important to preserve the Twitter archive?
  It sees Twitter is part of the historical record of communication, news reporting, and social trends – all of which complement the Library’s existing cultural heritage collections.
• What is in the Archive?
  Public information. Not private account information or deleted tweets.
• What does the Library plan to do with the archive?
  Its aims are preserving access to the archive for the long term and making data available to researchers.

The Blue Ribbon Task Force on Sustainable Digital Preservation and Access (BRTF-SDPA), a new international initiative funded by JISC and other organisations, has recently released its report entitled Sustainable Economics for a Digital Planet: Ensuring Long-Term Access to Digital Information. Its report examines the complicated and diverse issues from an economic standpoint. It identifies the problems intrinsic to all preserved digital materials, and proposes domain-specific actions that address the challenges to sustainability. The report focuses its inquiry on materials of long-term public interest in content domains with diverse preservation profiles, namely scholarly discourse, commercially owned cultural content and collectively produced Web content.

JISC is organising a free one-day symposium in London on 6 May 2010 where the Blue Ribbon task force will present its final report and invite responses from the BBC, the Natural History Museum, the British Library, European Bioinformatics Institute and the European Commission. Further information is available.

---

Storing Information in the Cloud

The Department of Information Studies at Aberystwyth University is currently running a small research project funded by the Society of Archivists examining operational, legal and security issues relating to the storage of information in the cloud for access. This is a topic which is likely to be of interest to those involved in website preservation work, as there has been increased interest in cloud services to support institutional activities. Although the project does not address preservation issues directly, the outputs of the project should be of interest to information professionals involved in digital preservation. We already had interest from various institutions in a follow-up project on preservation issues and are hoping to continue research in that in the near future – so watch this space!

What is the Aim of the Project?

Our aim is to generate debate and to highlight some of the issues surrounding the storage of information in a virtual environment. We have already seen many organisations outsourcing email and data storage to cloud providers such as Google and Amazon for cost and efficiency reasons. Cloud computing can have financial and operational advantages such as reduced deployment cost, increased storage capabilities and scalability. However, cloud computing raises quite a few security and compliance issues that need to be addressed when outsourcing information storage to third parties.

We, therefore, aim to develop a toolkit designed to enable information professionals whose organisations are about to deploy information into the cloud to ask the right questions and identify the right strategies for ensuring that information is kept securely, accessible and in line with relevant legislation. Even though preservation will not feature prominently in the toolkit, it is understood that preservation questions are an integral part of assessing how to manage the information life-cycle in the cloud and need to be addressed right at the start of setting up information management services and procedures.

What did the Survey tell us?

As a first step, we conducted an online survey aimed rather narrowly at information management professionals as the main stakeholders in information security and governance via Listservs and professional bodies’ members lists. Given the limited chosen audience we had a good response rate and gathered interesting insights into what professionals think and do about storing information in the cloud:

• The overwhelming majority of people who completed the survey worked in the public sector.
• Roughly 30% of participants said that their organisations are already using cloud computing and another 40% claim that their organisations are interested in cloud computing but have no active plans as yet.
• Most organisations use or intend to use software-as-a-service and deployment into a private cloud as cloud computing models.
• Data storage, email and standard office applications were named as the main IT services deployed into the cloud.
• There appeared to be no single outstanding driver for cloud computing – reducing cost, scalability and flexibility were the most popular by a small margin.
• Similarly, concerns about storing information in the cloud appeared to be evenly spread with concerns about the retrieval/destruction of data when terminating the cloud service, loss of control over data and data protection at the top of the issues list.
• Preservation and retention management were singled out for areas of further research. And demand in further guidance on operational, security and legal aspects in the form of best practice guidance was identified by the majority of participants

What is Next?

We will run a workshop in Manchester on 21 May 2010 in order to work through some cloud storage scenarios and investigate further issues and approaches to ensuring the secure storage of information in the cloud. Bookings will open soon and you can find more information about the workshops at http://www.dis.aber.ac.uk/en/news/cloudworkshop.asp.

Following the analysis of the results of both the survey and the workshop, a toolkit and report will be made available by the Society of Archivists in the autumn of this year.

If you have an interesting case study or would like to find out more, please contact me (email: nis@aber.ac.uk).

A list of a number of the sites which have been ‘mothballed’ is given in the article. Some of the sites are for programmes that have ceased broadcasting (eg. On The Record) and others are for events which are now over (e.g. Politics ‘97).

I was particularly interested to read about the BBC policies regarding the decommissioning of such Web sites. The article provide a link to the BBC’s policy which describes that inactive pages are left online for reference as “We don’t want to delete pages which users may have bookmarked or linked to in other ways. In general, our policy is only to remove pages where the information provided has become so outdated that it may lead to actual harm or damage.”

With the promises of large cuts for public sector organisations in the offing after the general election I suspect that we will find Web sites in many higher education origanisation being decommissioned.  But will  content be simply deleted, will the content be left ‘as is’ or will a more manged approach to such decommissioning take place? 

I feel there will be a renewed interest in the decommisioning of Web sites.  I hope the JISC PoWR’s Handbook on the Preservation of Web Resources will be of interest to organisations which find themselves  in this position.

Kevin has been Head of Digital Archives at the University of London Computer Centre (ULCC) since 1997, during which time his multi-disciplinary group has provided services related to the preservation and reusability of digital resources on behalf of other organisations, as well as conducting research, development and training.

The group has operated the National Digital Archive of Datasets for The National Archives of the UK for over twelve years, delivering customised digital repository services to a range of organisations.

As a member of the JISC’s Infrastructure and Resources Committee, the Advisory Council for ERPANET, plus several advisory boards for data and archives projects and services, Kevin has contributed widely to the research information community.

Kevin has been an active member of the JISC PoWR project and written many blog posts sharing his expertise.

The DCC has just begun its third phase of work makes the following comment on it’s Web site (A new phase, a new perspective, a new Director):

“As a firm and trusted proponent of the DCC we look forward to his energetic leadership in this new phase of our evolution.”

At JISC PoWR we offer Kevin our congratulations and wish him all the best in his new role.

The site was unveiled earlier this week by the Minister for Culture and Tourism, the Rt Hon Margaret Hodge MBE MP, and Chief Executive of the British Library, Dame Lynne Brindley, this project demonstrates the importance and value of the nation’s digital memory.

Websites included in the UK Web Archive include:

• The Credit Crunch – initiated in July 2008, this collection contains records of high-street victims of the recession – including Woolworths and Zavvi.
• Antony Gormley’s ‘One & Other’ Trafalgar Square Fourth Plinth Project – involving 2,400 participants and streamed live by Sky Arts over the web to an audience of millions, this site will no longer exist online from March 2010.
• 2010 General Election – work has started to preserve the websites of MPs such as Derek Wyatt, who will be retiring at the next election, creating a permanent record of his time as a Member of Parliament.

This important research resource has been developed in partnership with the National Library of Wales, JISC and the Wellcome Library, as well as technology partners such as IBM.

British Library Chief Executive, Dame Lynne Brindley said:

“Since 2004 the British Library has led the UK Web Archive in its mission to archive a record of the major cultural and social issues being discussed online. Throughout the project the Library has worked directly with copyright holders to capture and preserve over 6,000 carefully selected websites, helping to avoid the creation of a ‘digital black hole’ in the nation’s memory.

“Limited by the existing legal position, at the current rate it will be feasible to collect just 1% of all free UK websites by 2011. We hope the current DCMS consultation will enact the 2003 Legal Deposit Libraries Act and extend theprovision of legal deposit through regulationto cover freely available UK websites, providingregular snapshots ofthe free UK web domainforthebenefit of future research.”

Further details are available from the British Library.

KRDS2

The Keeping Research Data Safe 2 project commenced on 31 March 2009 and will complete in December 2009. The project will identify and analyse sources of long-lived data and develop longitudinal data on associated preservation costs and benefits. It is believed that these outcomes will be critical to developing preservation costing tools and cost benefit analyses for justifying and sustaining major investments in repositories and data curation.

The Survey

The survey was carried out between between September and November 2009 to identify key research data collections with information on preservation costs and related issues. 13 survey responses were received: 11 of these were from UK-based collections, and 2 were from mainland Europe. The responses covered a broad area of research including the arts and humanities, social sciences, and physical and biological sciences and research data archives or cultural heritage collections.

The survey questionnaire sought to identify cost information available for the main KRDS2 activities in the Pre-Archive and Archive phases. Information for some activities is very high (archival storage cost information is available in 100% of the responses). Other more infrequent activities such as disposal (and perhaps also preservation planning) are less well represented. Knowledge of acquisition costs is also relatively low (46%).

Further information is available from the KRDS2 project Web page.

Mr Birdling’s letter brought a rapid response from David Thomas of the UK National Archives who was keen to reassure readers that central government websites were being archived, even without the legislation which prompted Mr Birding’s original letter. (That story refers to the changes to Legal Deposit regulations which would permit the British Library and other UK copyright libraries to capture UK content without the permission of rights owners.)

But earlier examples of non-specialist concern with preserving web content exist. One of my favourite examples comes from the Usenet group uk.railway whose contributors include a fair number of rail enthusiasts (“trainspotters” if you’re feeling unkind.) Privatisation of the UK railway network means that we have a plethora of train operating companies, or TOCs, each of whom operate their own web site, much as the great companies of old such as LNER might have done if the web had existed then. The difference is that now these companies come and go every few years when the government puts operating contracts out for re-tender. Railway ephemera such as promotional leaflets and timetables are a key part of the print collections at places such as the National Railway Museum. “What happens to TOC web sites when franchises change?” wondered one poster to uk.railway back in 2007. The Internet Archive has certainly captured some material, but it isn’t the same as a collection controlled by an institution such as the NRM. I wasn’t able to give a very positive answer to their question. I don’t believe the National Railway Museum are yet able to capture websites as part of their collection, and it’s not clear that any of the members of UKWAC see TOC sites as falling within their collecting policy.

And herein lies a lesson. Rail enthusiasts are incredibly effective at preserving railway heritage, both through their own efforts and through influencing others. They include many people with an enviable range of technical abilities. They ensured that special legislation was passed to ensure the preservation of railway heritage after privatisation. Not content with simply preserving heritage, some of them set about recreating it through building an entirely new steam locomotive. But their combined efforts have not yet (so far as I know) ensured that past railway web sites have been preserved. If they can’t manage it without institutional help, what hope is there for the rest of us ?

5th International Digital Curation Conference

“Moving to Multi-Scale Science: Managing Complexity and Diversity” | 2-4 December 2009

The IDCC is an established annual event reaching out to individuals, organisations and institutions across all disciplines and domains involved in curating data for e-science and e-research.

The DCC will be hosting a workshop programme on 2 December followed by a Pre-Conference Drinks Reception at the Natural History Museum. The main conference will open on 3 December with a keynote speech from Professor Douglas Kell, Chief Executive of the Biotechnology & Biological Sciences Research Council (BBSRC). Other key speakers will include: Professor Ed Seidal, National Science Foundation; Cliff Lynch, Coalition for Networked Information; Timo Hannay, Nature Publishing Group. The first day of the conference will incorporate an interactive afternoon for posters and demos, followed by a Symposium entitled “Citizen Science: Data Challenges” led by Richard Cable, BBC Lab UK.

The second day will be made up of peer-reviewed papers in themed sessions covering Disciplinary and Institutional Challenges, Practitioner Experience, Metadata, Software Preservation & Managing Risk.

Places are limited so please register now.

Registration to close on 20 November 2009

We discussed Yahoo’s announcement that the Geocities service was to be shut down some time ago in a post entitled ““Seething With Anger” at the Demise of Geocities“. What I find interesting in the article is the information that “… there’s the real effort, by the Archive Team, who have been trying to archive as many Geocities pages and sites as they could“.

I’d not come across the Archive Team wiki before. They describe themselves as a “project composed of volunteers, currently coordinated by Jason Scott” which invites.

• Writers, who can create clear essays and instructions for archivists and concerned parties.
• People with Lots of Hosted Disk Space who have a proper hosted webserver and fat pipe, who are willing (when asked) to consider hosting mirrored dead sites or archives.
• People who love setting up torrents who can do the same as the mirror folks, but do so hosting torrents.
• OCD-rich individuals who want to download things who will respond to our alerts and call outs and download entire sites or diagnose ways to get at obfuscated data.

The wiki home page informs us that “This website is intended to be an offloading point and information depot for a number of archiving projects, all related to saving websites or data that is in danger of being lost. Besides serving as a hub for team-based pulling down and mirroring of data, this site will provide advice on managing your own data and rescuing it from the brink of destruction.”

Hmm. I wonder how effective a volunteer organisation is likely to me? My initial thoughts were fairly sceptical, but other volunteer-led initiatives, such as Wikipedia, do seem to be successful. What are your thoughts?

Obsolescence, deterioration of physical storage media or withdrawal of institutional support: just what will prove to be the greatest threat to the materials we digitise today? This seminar projects one hundred years into the future and attempts to predict the future ‘preservability’ of what we digitise today. This seminar will examine changing user demands and inevitable developments in technology.

After a brief opening from Dave Kilbey of JISC Digital Media the scene setting introduction was given by Dr William Kilbride, Executive Director of the Digital Preservation Coalition.

The Preservation Landscape

As well as the more conventional look at the key issues (the volumes of data available, the complexities and complicated requirements of this data teamed with rising public expecations) William gave a really interesting talk on the path of literacy. He demonstrated through the Stroop interferance test how once we can read and write we tend to process this information quicker that image information. The result is a that literate cultures tend to be hegemonic through discursive power. His point was that the consequences of our work are not inevitable or neutral: digitisation is a social practice that can be used for good and for ill. After this slight aside William ran us through some of the main challenges which include obsolescence of technologies, correct configuration of hardware, software and operators, and the need for a constantly managed service. He ended with a few ‘answers’ from a survey of recent JISC digitisation projects. When asked how long their resources were to be available answers varied from “perpetuity” to “forever or three years”. He concluded that digital preservation is possible but our legacy will be what we make of it and cannot be taken for granted.

The Camera Raw format and preservation

Nigel Goldsmith, a photographer working for JISC Digital Media gave a quick run through of the possibilities of using Raw camera format. Raw offers the photographer greater control over the processing of their images, however this flexibility comes at a price. Raw is a proprietary format which requires specialist applications to view. Nigel’s suggestion was to archive raw but to keep it along side another format, possibly tiff or Jpeg2000.

Preservation Metadata Initiatives and Standards

After coffee Getaneh Alemu from the Humanities Computing Department, the University of Portsmouth gave us a whirlwind tour of state-of-the-art metadata standards and how metadata can help ensure the integrity, identity and authenticity of digital documents. His overview included a look at OAIS, NLA PANDORA, CEDARS, NEDLIB, LMER, PREMIS, and METS metadata initiatives and standards. He concluded that at the moment preservation metadata formats tend to have element naming issues that descriptive metadata initiatives don’t tend to have.

The challenges of archiving computer games and other multipart digital interactives

After lunch Tom Woolley from the National Media Museum talked about some of the digital media preservation issues they are tackling on-site at the museum. The museum is involved in a number of initiatives that aim to let visitors ‘have a go’ at old games and old internet environments. The tricky dilema is giving users a taster of old games in a cost effective way, actually using original kit (like ZX Spectrums) would have a heavy cost attatched. The key is often emulation. The museum also try to capture the context of games by capturing fan information, discussion forums, FAQs etc. Tom was followed by James Newman from Bath Spa University who works with Tom on the National Video Game Archive.

James talked about one of the biggest challenges of video game archiving: supersession. Within the gaming world there is a tendency to be always looking for the ‘next big game’ which has resulted in an environment where games creators don’t value old games. Although there is a niche market for retro games, gaming is an area where the experience is almost completely associated with the technology, making archiving very difficult.

The importance of collaboration

Simon Tanner, director of King’s Digital Consultancy Services focused on institutional preservation and the importance of collaboration in sustainability. He started off by saying that one of the biggest challenges is that we may run out of the minerals to make microchips. He later played on the climate issue again by saying that he currently saw digital preservation as sitting in the same space as climate change: people viewed it as potentially a terrible thing (the loss of digital objects) but currently it does not impact on individuals, so it remains low on the priority list. Simon pointed out that sustainability of resources was becoming a mandate but remains an unfunded mandate. The way to deal with this was through the ecology of collaboration – within your institution and out side.

A Poisoned Chalice? Accepting Responsibility for Sustainable Access

The day concluded with a talk from Neil Grindley, JISC Programme Manager for Digital Preservation. Neil pointed out ath ensuring that an organisation’s digital assets are safe, secure and accessible for the long term should (in theory) be an interesting, responsible and useful role for anyone in an organisation to accept. The critical importance of digital assets, the ubiquity of digital methods and the need for people in all walks of life to have effective means to refer to persistent sources of data reinforce this notion. How is it then that long-term asset management, information lifecycle management, data curation, digital preservation (call it what you will) is often regarded as a peripheral specialist activity that it is difficult to resource, complex to carry out, and delivers benefits that are, at best, simply an insurance policy rather than an activity that adds value to an organisation? Neil’s presentation examined the importance of defining clear roles for those involved with digital preservation and considered the importance of associating this professional activity with strategic and tactical frameworks. He advocated the need for allocation of responsibility and internal preservation policies. JISC has spent 6 million in the digital preservation arena between 2005 and 2009, yet there is still work to be done. He concluded by pointing out the need for human judgement when deciding what to keep and predicted that in the future digital preservation will be integrated with administration departments, have better tools and will take more terms from the cultural heritage area.

After Neil’s talk there was a panel session and time for questions, unfortunately I had to leave to make the difficult drive home through rush hour traffic!

The day was an interesting one, although the talks were a real mixed bag they all offered constructive steps forward to make today’s digital media collection something that we may be able to access and use 100 years on.

Harvard WAX includes themed collections on Women’s Voices and Constitutional Revision in Japan, but of particular interest to us in PoWR is their A-Sites collection: the semi-annual captures of selected Harvard websites. “The Harvard University Archives is charged with collecting and preserving the historical records of the University,” state the curators, recognising their formal archival function in this regard. “Much of the information collected for centuries in paper form now resides on University web sites.”

Helen Hockx-Yu of the British Library met with the WAX team in May 2009. “I was impressed with many of the features of the system,” she said, “not just the user and web curator interfaces but also some of the architectural decisions. WAX is a service offered by the Library to all Harvard departments and colleges. In exchange for a fee, the Departments use the system to build their collections. The academics may not be involved with the actual crawling of websites, but spend time QAing and curating the websites, and can to some extent decide how the archive targets appear in the Access Tool. The QAed sites are submitted directly into Harvard’s institutional repository.”

It is very encouraging to read of this participatory dimension to the project, indicating how success depends on the active involvement of the creators of the resources. Already 48 Harvard websites have been put into the collection, representing Departments, Committees, Schools, Libraries, Museums, and educational programmes.

The delivery of the resources has many good features also; there’s an unobtrusive header element which lets the user know they’re looking at an archived instance (instead of the live website). There’s a link explaining why the site was added to the collection, and contextual information about the wider collection. Another useful link allows researchers, scholars and other users to cite the resource; it’s good to see this automated feature integrated directly within the site. The Terms of Use page addresses a lot of current concerns about republishing web resources, and strikes just the right balance between protecting the interests of Harvard and providing a service to its users. Like a good OAIS-compliant repository, they are perfectly clear about who their designated user community are.

Best of all, they provide a working full-text search engine for the entire collection, something that many other web archive collections have been struggling to achieve.

The collection is tightly scoped, and takes account of ongoing developments for born-digital materials: “Collection managers, working in the online environment, must continue to acquire the content that they have always collected physically. With blogs supplanting diaries, e-mail supplanting traditional correspondence, and HTML materials supplanting many forms of print collateral, collection managers have grown increasingly concerned about potential gaps in the documentation of our cultural heritage.” The project has clear ownership (it is supported by the University Library’s central infrastructure), and it built its way up from a pilot project in less than three years. Their success was partially due to having a clear brief from the outset, and through collaboration with three University partners. What Harvard have done chimes in with many of the recommendations and suggestions made in the PoWR Handbook, particularly Chapters 5 (Selection), 16 (Responsibility for preservation of web resources) and 19 (How can you effect change?)

There are many aspects of this project which UK Institutions could observe, and perhaps learn something from. It shows that it is both possible and practical to embed website collection and preservation within an Institution.

It is hoped that this study will generate some stimulating debate on this matter and lead to a greater understanding of the current and future state of the Records Management profession in the UK where digital and Web preservation may be key.

Please access the survey at the following link.

The survey closes at midnight on 5th September.

This year’s event will be hosted by California Digital Library (CDL) at Mission Bay Conference Center in San Francisco on October 5th and 6th, 2009.

UK presentations include Maureen Pennock on ArchivePress, David Giaretta on significant properties in OAIS and Adam Farquar on (Planets) metadata.

“A year ago my academic web site disappeared. And those who made it go away probably ignored that such a thing could happen.“

The article goes on to describe how last year Richard “found out that the School of Business had redesigned their web site. And in the process they just ignored my research. About ten years worth of virtually daily updates were gone That included most of the manuscripts for my published work. The same thing happened to lecture notes, powerpoint slides, course documentations, useful links, etc. It had all disappeared from the Web!“.

Richard did have some good news to report: “Courtesy of the Internet Archive you can still find most of my academic stuff on the Web through their Wayback machine.” although Richard did wonder why he had to rely on the Internet Archive (“a 501(c)(3) non-profit that was founded to build an Internet library”) – after all, wouldn’t you expect your institutional library to provide this service?

Richard’s losses of his digital resources have continued – a blog he set up at Stockholm University was deleted after he left the institution – although, again a copy is archived on the Internet Archive.

Richard’s experiences have left him disillusioned with the attitudes towards the digital preservation of scholarly resources. He concludes by recommending that academics take responsibility themselves for preserving their resources:

Meanwhile, for those of you who publish stuff on the Web while working with an organisation, including universities. Try to put your content where you control it. Most likely you will move between work places, temporary assignments, and soforth. If you want your stuff to be preserved, it is your responsability to make sure it is.

But how easy will this be for the typical academic? Richard doubts whether “the issues I bring forward today are heavily discussed among university chancellors, political leaders, educational policy makers, and scientific philosophers.“  But surely we need to ensure that this debate takes place. And, in today’s economic climate, that debate needs to include discussions of the costs of digital preservation (disk storage may be cheap but management of content is not).

Richard’s tale is based on his experiences as an academic in Sweden. Is the situation different in the UK, I wonder?  Judging by Stuart Smith’s lament that “Mummy I lost my MP3!“, which I summarised in a post on “Disappearing Resources On Institutional Web Sites” in December 2008 it would seem that we have similar experiences in the UK higher education sector. Does anyone have any positive experiences to share?

The statistic much banded about (for Web pages not pieces of string!) is 44 days, believed to originate in an article by Brewster Kahle (of Internet Archive fame) published in 1997 and titled Preserving the Internet. Brewster’s original quote is specifically about URLs, “…estimates put the average lifetime for a URL at 44 days.”

Whether this figure still stands today is a matter currently being discussed on the CURATORS@LIST.NETPRESERVE.ORG list after a query from Abigail Grotke of the Library of Congress.

Abbie offered up the 44 day statistic and pointed out that on the Digital Preservation Web site they have a graphic that discusses Web volatility stating “44% of the sites available on the internet in 1998 had vanished one year later“.

The other figure often cited is 75 days from a Michael Day’s report Collecting and preserving the world wide web.

“The dynamic nature of the Web means that pages and whole sites are continually evolving, meaning that pages are frequently changed or deleted. Alexa Internet once estimated that Web pages disappear after an average time of 75 days. (Lawrence, et al.,2001, p. 30).“

Another figure sometimes suggested is 100 days, this seems to come from Rick Weiss article for the The Washington Post, Washington, DC, 24 November 2003, On the Web, Research Work Proves Ephemeral -  no longer available.

So what is the average lifespan of a Web page today? Is it getting shorter or longer? The Internet Archive now gives 44 -75 days as its ball park figure. I’d have to hazard a guess that with the rise in use of Web 2.0 technologies the Web is actually getting more transient by the day.

Is this OK?

Maybe if it’s just a tweet you sent your friend, however if it’s something more substantial that’s disapearing then it’s a real worry.

Twitter proved very popular during this annual event for institutional Web managers, with over 1,500 Twitter posts (tweets) being published during the last week of July. Further statistical information is provided in a post on Evidence on Use of Twitter for Live Blogging.

We suggested that a two character code (P1 to P8)  could be used to identify each plenary session and that using this as a hashtag in conjunction with the event’s hashtag (#iwmw2009) would enable the tweets about a particular talk to be easily identified and, in theory, this data migrated to a managed environment.

As an example you can search for the tweets related to:

• Derek Law’s opening plenary talk on  Headlights on Dark Roads (P1) using Tweetzi, Twazzup or Twitter Search
• The Mashups Round the Edges workshop session by Mike Ellis and Tony Hirst (A8) using Tweetzi, Twazzup or Twitter Search

We have recently used The Archivist desktop application to create a local copy of the tweets for the plenary  talks at the conference, and these have been made available on the IWMW 2009 Web site from the individual pages for the plenary talks (e.g. see the page for Derek Law’s opening plenary talk). The pages also contain a summary of the number of Twitter posts which were found using the tool.

One reason for wishing to do this is to provide an answer to the speaker who may ask “I Wonder What They Thought About My Session?“.

The Rambling Librarian post also picks up on the important “implication … that digital preservation will be even more critical. Imagine all the collaborative efforts gone when the server crashes. Or power fails.”

Absolutely! And let’s ensure that the digital preservation aspects are being considered right at the start of any development activities rather than being ignored by those focussing on the new possibilities which this technology can provide.

Hmm, I wonder if there are any funding possibilities available for exploring the preservation aspects of Google Wave?

If anyone can successfully preserve the Web it was this lot!

Session 1: Setting the scene

Keynote: Adrian Brown, Assistant Clerk of the Records, Parliamentary Archives

The opening keynote was given by Adrian Brown from the Parliamentary Archives who did adept job of setting the scene and the challenges ahead. His talk centred around the key jigsaw pieces of selection, capture, storage, access and preservation. He pointed out that the Web used to be primarily publisher of fixed content and it is the new types of content (discussions on wikis, Twitter, blogs etc.) that offer the biggest challenge. Parliament themselves are using many new forms of information, and although permission is not always an issue capturing content on other sites often is. You may own the content but someone else often owns the presentation of that content. Adrian also emphasised the need for us to co-ordinate selection and be as explicit about what we are not going to capture as what we are going to capture (the unpublishing button on facebook being a classic example of how difficult it can be to go back!) Another major challenge is that of temporal cohesion – the rate of capture of content is often far slower than the rate of change of content.

He concluded with a Peter Murray Rust quote that stresses the need for us to “Create and release herds of cows, not preserve hamburgers in a deepfreeze.”

Web Archive and Citation Repository in One: DACHS: Hanno Lecher, Librarian, Sinological Library, Leiden

HannoLecher talked about the citation problems he has encountered while working on the digital archive for Chinese studies, a resource that captures and archives relevant resources as primary source for later research. His work has indicated the possible need for citation repositories. Some optins might include verifying

URL references, evaluating the reliability of online resources, use of PURLs, avoidance ofobscure URLs and even the extreme banning of online resources as reference. As such approaches are not always practical (or appropriate) he use of applications like Snagit, Zotero or WebCite.

The future of researching the past of the Internet: Eric T. Meyer, Research Fellow, Oxford Internet Institute, Oxford University

Eric Meyer works primarily looking at social behaviour and the Internet and as part of the Transatlantic Digitisation Project spent some time building an archive collection of World War 1 and 2 Internet resources. The process gave the team incite into the feasibility of  creation of such an archive. Lessons were also learnt on the need for involvement of domain experts (webometrics, SNA, eResearch, etc.) and the desire to move from snapshots to more continuous data. Eric concluded by stressing that what researchers really want is a global way to access archives, not a national one.

After the morning coffee break the workshop moved from the general to the specific.

Session 2: Creation, capture & collection

An overview of Web Archiving Tools by Helen Hockx-Yu, Web Archiving Programme Manager, The British Library

Helen Hockx-Yu provided a very practical overview of current tools, covering Heritrix, NetArchiveSuite, Web Curator Tool and PANDAS. She also discussed archival formats such as ARC and WARC, which is highly desirable from a long term archival standpoint. Helen concluded a brief discussion on the limitations and challenges harvesters present from issues with rendering and dealing with ‘bad’ content to reliance on open source tools that are still very much evolving

Context and content: Delivering Coordinated UK Web Archive to User Communities: Cathy Smith, Collections Strategy Manager, The National Archives

Cathy Smith and the UKWAC team have recently been working on study entitled Delivering coordinated UK web archives looking at what audiences should web archives anticipate and what does this mean for selection, ingest and preservation and what do researchers want from archives. The conclusions include the notion of a single view to an archive, though this does not necessarily mean a single repository. Cathy explained that “hey just want the car to work – they don’t want to see what’s under the bonnet”. In response to the study the NA have recently begun automated whole domain harvesting – shallow as well as selective harvesting and they are looking at allowing all archiver to be searched together – a collection.

Capture and Continuity: Broken links and the UK Central Government Web Presence: Amanda Spencer and Tom Storrar from the National Archives.

Amanda Spencer and Tom Storrar were set with the task of sorting out the problem of broken links on government Web sites. 60% of URLs quoted in Hansard between 1997-2006 are broken leaving the public unable to find information and with little faith in the authority of government Web sites. The Web Continuity project has begun to create an archive of all Central Government Web space. Some thinks they are trying out include XML site maps, using redirects instead of 404 messages and capturing all government Web sites before they are closed.

In the questions and discussion session the panel were asked what advice they would give to Web managers in how they can allow archiving of their sites to be more effective. The suggestions tackled content (following accessibility guidelines, W3C standards, making code valid, adding in hard links to files next to streaming content, having transcripts of audio files) and communication (enabling a dialogue between Web archivists and Web creation people.)It was noted that there is a fine line between encouraging good practice and stifling innovation and that at times communication with content creators is a luxury. Responsible harvesting also means explaining why you are doing it to those involved.

The discussion then moved to other possible points of capture such as using Google Cache or asking ISP providers.

Session 3: Issues and approaches to long term preservation of web archives

After lunch the programme moved on from the specific problems of now to problems of the future.

Diamonds in the Rough: Capturing and Preserving Online Content from Blogs: Richard Davis, Project Manager, University of London Computing Centre (ULCC)

Richard Davis, a member of the JISC PoWR team, started off by emphasising the importance of blogs today. This has lead him and colleague Maureen Pennock to consider the need for their preservation, particularly at institutional level, resulting in the ArchivePress project. Richard pointed out that traditional Web archiving tools can be difficult to configure and use: a ‘hammer to crack a nut’. A comment by Chris Rusbridge that ”blogs represent an area where the content is primary and design secondary” suggested that using blog feeds might be key. The feed could be a point at which to gather blog content for archiving. The test bed for ArchivePress will include blogs from the DCC, Lincoln University and UKOLN. In the future ArchivePress could possibly be used to harvest Twitter.

Beyond Harvest: Long Term Preservation of the UK Web Archive: Maureen Pennock, Web Archive Preservation Project Manager, The British Library

Maureen Pennock started her talk by giving some general background to UK Web Archive. Now the archive has been established the next task is preserving the archive, an ongoing activity. The main undertakings are documenting System Dependencies, consideration of containers and metadata standards (following a review, WARC is the preferred format for preservation, but they are also using a combination of features from METS and PREMIS), preservation workflow and defining the preservation strategy. The future involves the consideration of many new areas, for example should we preserve computer viruses.

From Web Page to Living Web Archive: Thomas Risse, Senior researcher, L3S Research Center

Thomas Risse introduced the LiWA (Living Web Archives) project, an EU FP7 research project. LiWA has been looking at building tools to address the specific challenges of the current generation Web, e.g. link extraction from dynamic pages.

Emulating access to the web 1.0: Jeffrey van der Hoeven, Koninklijke Bibliotheek, The Netherlands

Jeffrey van der Hoeven started off with a whirlwind history of the Web browser. From the WorldWideWeb browser in 1991, which was more like a document viewer, followed by Mosaic and Netscape, and onto now with Firefox and Google Chrome. Today’s browser do many things, from rendering Web sites, bookmarking, caching, authentication and providing RSS feeds. They have changed from viewers to all round workplaces – Google Wave is an extreme example of this. The browser is currently the centreal application of the computer and there is a need for a sustainable solution to render Web sites. At the Koninklijke Bibliotheek they have been recently working on the Dioscuri project, a durable emulator. They have also been working with the planets project using Global Remote Access to Emulation (GRATE) and on the Keep Emulation Environments Portable (KEEP) project.

After another coffee break we moved onto the final discussions

Session 4: Discussion and next steps

What we want with web-archives; will we win? Kevin Ashley, Head of Digital Archives, University London Computer Centre (ULCC) Kevin Ashley

Kevin Ashley, also part of the JISC PoWR team, kicked off his talk by introducing us to the future of the Web, Web 8.0. Although Kevin was just joking with us he explained that when the next major change in the Web happens, we will need a much better metaphor than ‘Web 3.0′: it is the future that we need to think about. We need to be asking the key question as to what we want from Web archives, e.g. potential use cases. Preserving the Web is not just about content. Sometimes it will be useful to look at Web content in aggregate, e.g. for visualisation, linguistic analysis, etc. Right now many academics are doing own Web archiving – content and presentation or both. However we need to not take just a document centred view of data but also consider the web of data and data about the web. A lot of future interest will not in Web content, but in the context of use.

The day concluded with a round table discussion with many speakers from the day. Some of the topics discussed were the permissions challenge (primarily with regard to UKWAC and similar initiatives). Other countries and initiatives have favour a more ‘just do it’ approach.

The day left most people enthused and positive about the future for Web archiving.

A conference report will follow on the DPC Web site and will be sent out via the dpc-discussion and digital-preservation JISCmail lists.

All presentations are now available in PDF Format from the DPC Web site.

WAC09 was used as the Twitter tag for the event. You can see the tweets at http://hashtags.org/tag/wac09/messages.

The event was sponsored by the Digital Preservation Coalition (DPC) and the Joint Information Systems Committee (JISC) and the six partners of the UK Web Archiving Consortium (British Library, National Library of Wales, JISC, Wellcome Library, The National Archives and the National Library of Scotland).

The JISC PoWR team presented two posters, one provided a general overview of the project and the other discussed Preservation Policies and Approaches for Use of Social Web Services.

The content of the second poster has been mentioned in a number of previous blog posts:

• Preservation Policies for Content Hosted on Third Party BlogsTools For Preserving Twitter Posts
• Some Use Cases For Preserving Twitter Posts
• Preserving Tweets: The CILIP2 Experience
• Missing links: the enduring web

Although the poster was submitted to the British Library it was unfortunately mislaid, so here is the actual content.

Click on the image for a larger version.

The approach taken by the JISC PoWR project ( and described in more detail in the JISC PoWR handbook – see particularly chapter 13) has been to recognise that there are legitimate reasons to make use of such services and to look at ways in which content hosted on such services can be managed and curated in the long term.

The need to do this is of relevance to UKOLN which provides a number of blogs on externally-hosted services including the UK Web Focus blog, provided by Brian Kelly and the Rambling of a Remote Worker blog, provided by Marieke Guy.

The first stage is to define and publicise a policy covering the long-term access to the content of these two blogs, including what will happen if either of the authors leaves UKOLN.

Policies posted on the UK Web Focus and Ramblings of a Remote Worker blogs state that:

• A rich copy of the contents of the blog will be made available to UKOLN (my host organisation) if I leave. Note that this may not include the full content if there are complications concerning third party content (e.g. guest blog posts, embedded objects, etc.), technical difficulties, etc.
• Since the blog reflects personal views I reserve the rights to continue providing the blog if I leave UKOLN. If this happens I will remove any UKOLN branding from the blog.

These two simple statements can help, we feel, in ensuring that the content can be managed if the blog authors leave (or if they fall ill, go crazy or die!). The statements seek to avoid uncertainties regarding what can be done with the content.  The second statement also clarifies that if the authors were to leave, they may wish to continue using the blog.

It may be argued that since both blogs make their content available under a Creative Commons licence this already grants the host institution, along with anyone else, the rights to preserve the content. This may be true, but there is no harm in making this explicit, we feel.

Would it not be legitimate for organisations to expect its employees to make similar statements which clarify the ownership of content hosted on Web 2.0 services and created as a normal course of one’s duties?

In response to that post a number of solutions for preserving tweets were suggested including FriendFeed, the WordPress Lifestream plugin and What the Hashtag. In addition following a tweet I posted I received details of the Tweetdoc service.

With this year’s Institutional Web Management Workshop (IWMW 2009) rapidly approaching it is timely to decide the tools we’ll be using to preserve the discussions associated with this event. We began keeping a record of the amplification of the IWMW event back in 2005 when an IRC channel was provided for use by the small numbers of participants who had a laptop and WiFi connectivity at the event. The IWMW 2005 event took place during the 7/7 bombings in London and a record of the awareness of what was having can be gleaned from the archive of the IRC discussions.

This year we will once again be making use of Twitter and will be capturing tweets which contain the event hashtag #iwmw2009.  The tools we are currently intending to use are What the Hashtag and Tweetdoc service.

Use of What the Hashtag to views tweets containing the #iwmw2009 tag is illustrated.

As can be seen the output can be access as an RSS feed. In addition the data can be viewed as an HTML resource, and a data range can also be supplied.

We intend to capture this tweets about the IWMW 2009 event after the event is over, and store the data on the UKOLN Web site, in order to avoid dependencies on the What the Hashtag service itself.

We will also explore other services, such as Tweetdoc – although in this case as the data is only available as a PDF resource, it is not well-suited to provide data for analysis by other services.

Are there any other services we should be looking at? And what functionality might be desirable for a more generic service for preserving tweet? Any thoughts?

However this week I have come across a couple of examples which illustrate why it might be useful to preserve tweets.

Preservation of Twitter data for research purposes

I recently described how Twitterers Subvert[ed] Daily Mail’s Racist Poll. For me that provided a fascinating example of how Twitter can be used by activists to support a viral campaign (in this case to undermine the racist sentiments in the question “Should the NHS allow gipsies to jump the queue?“). But why did this poll succeed in attract a large turnout (with the poll being undermined with around 94% voting yes before the Daily Mail removed the poll) whereas a subsequent poll on Should immigrants be forced to respect British culture? failed to arouse much interest?

Since many of the tweets and retweets provided a link to “http://bit.ly/w4b6Q” we can look at the statistics provided by the bit.ly short URL service which tells us that 48,796 clicked on the link.

As shown in the histogram as might be expected the vast majority of clicks to the poll took place on 19 June 2009, the day the poll was the hot topic.

But who created the interest in the Twitterverse originally? Was it, as I speculated in my post, the UK-based psychologist community? Or where there a number of starting points? How was the message communicated? Who where the ‘influencers’ (to use the terminology from a recent blog post by Martin Weller) whose tweets were felt to contain links worth following? Who were the ‘hubs’ who were responsible for communicated the message to a wide audience? And was Twitter actually the main driver or were their other viral mechanisms in operation as Josh suggested, who went on to add “we must keep perspective and not neglect other areas or over-celebrate Twitter as a comms tool!“.

In a recent post by Andy Powell on “Influence, connections and outputs” Andy  suggested that we want “some sensible measure of scholarly impact on the social Web” and that we need to understand”what do we want to measure; what we can measure and how can bring these two things close enough together to create something useful”. Although I’m not suggesting that the Twitter responses to the Daily Mail poll have any scholarly impact, I do think that this could provide us with an opportunity to understand how ideas (whether scholarly, political or simply marking) are communicated and transmitted by Twitter. But if we wish to make use of this particular example, we will need to preserve the Twitter stream – and we will need to preserve the structured data (the Twitterer ID, the time, the IDs of those being retweeted and the numbers of followers) in order to carry out any rich analysis.

Preservation of an organisation’s memory

As recently described on the This is Bath blog the Bus station Twitter project ends – today. The bathcsc Twitter account and accompanying blog, were set up to keep users of the First bus service in Bath informed of news – including delays to bus services. Sadly First’s head office in Bath  have announced that the company “is having to make changes to its business structure. As part of making the company more cost efficient, the customer services function for Bath Bus Station will now be handled by a specialist, area-wide centre based at Exeter“.

This news has not gone down well with many of the fans of this innovation use of social networking tools (and I should add that, as a daily bus user, I personally found the Twitter service very useful) as can be gauged from the comments on the blog to the announcement of the demise of the service and on the Bus station Twitter project ends article.

In the article a spokeswoman said that the company was hoping to learn lessons from the nine-month experiment:

The Bath Twitter trial attracted an enormous amount of interest, both inside and outside the organisation. We are now looking at the lessons we have learnt, and benefits associated with the system, with a view to developing a national social media approach to customer services for the future.

But is the company in a position to preserve the tweets? Will the company be in a position to appreciate the ways in which Twitter can be used effectively if this record is lost? The danger, of course, is that a new media markeing agency wil sometime in the future advise on how Twitter could be used – and fail to learn from the successes of the initial trial, perhaps even making as big a mistake as Habitat have done recently.

What To Do? When To Do It?

Are these illustrations of when preservation of Twitter tweets reasonable? If so, can you think of other examples which we need to consider?

And if there is a case to be made for preserving tweets, what tools should be used? I am aware of the Print Your Twitter service, but this just creates a PDF file of tweets matching the search criteria.  To summarise what I think is needed:

•  Ability to create a structured repository of tweets from a Twitter ID
•  Ability to create a structured repository of tweets matching a search term

Anyone know of software which can be used to do this? I’m aware that one could simply use Twitter to search for “http://bit.ly/w4b6Q” or “@bathcsc” but this doesn’t provide a persistent and reusable data store.

Yet, as we discovered in JISC-PoWR, few institutions have truly incorporated web archiving into their overall records and asset-management systems, let alone recognised the specific value of blog content (or even of using blogging to replace traditional approaches to reporting and minuting). Perhaps it just seems too complicated. For those that want to, the only tools that seem to be readily available are specialised tools – like Web Curator Tool and PANDAS – that utilise crawlers like Heritrix and HTTrack to copy websites by harvesting the HTML framework, and following hyperlinks to gather further embedded or linked content. The result might typically be a bunch of ARC/WARC files (a file format specifically designed to encapsulate the results of web crawls), containing snapshots of the browser-oriented rendering of web resources. For many web resources, especially static pages, this is sufficient. When it comes to blogs, though, the archived results seem a bit too static – as I noted in an earlier JISC-PoWR post.

Treating blogs only as web pages overlooks the fact that they are derived from rich, dynamic data sources, and are usually databases themselves. An archive of blogs should allow us to do exactly the same kind of selection as on a live blog: selecting posts by author, date, category, tag. And since a blog is structured data, isn’t the underlying data a more appropriate target for long-term preservation, rather than endless, often duplicate copies of just one particular view of that data?

So what if, instead, the archiving tool were a bit of software already in use, or at least widely used, supported and understood? And Open Source, naturally.

This is the premise behind ArchivePress, a new JISC-funded project being undertaken by ULCC and the British Library. It is a ‘proof-of-concept’ project to progressively explore the implications and possibilities of using newsfeeds and blogging software – WordPress, of course – to capture and archive blog content dynamically, as it happens. ArchivePress will demonstrate the use of feed-reading aggregation to populate the database automatically with posts, comments and embedded content. The result will be a working model of a WordPress installation, with extra plugins, which can be easily setup by any institution to harvest content from blogs they have an interest in. We’ll continue our association with UKOLN, who, along with Lincoln University and the Digital Curation Centre, have agreed to let us use some of their blogs in our development and testing.

In some respects there seems nothing terribly new to anyone already adept with blogs, newsfeeds and newsreaders – except that this appears to be the first attempt to exploit them to create accessible, managed collections of blog posts, with the potential to meet the more exacting requirements of archives and records management, such as reliability and authenticity. Even organisations that have a single mandated blog platform may wish to consider this approach to preserving their blog content. ArchivePress might also be of value to other blog-based activities, from local-history projects, to school and college blogs.

ArchivePress has its own website and blog, that will build a cumulative picture of its results and the issues it encounters over the next 6 months. It wouldn’t have been possible without JISC-PoWR, and we hope it will complement that work. Please check it out and add it to your feed reader. We have a great team who will be contributing, including Maureen Pennock (ex-UKOLN, now at British Library) and Ed Pinsent (UKWAC and JISC-PoWR) – and we even plan to squeeze some guest posts out of web preservation alumni. I’ll also be talking about ArchivePress at the Missing Links workshop at the British Library in July.

Use of the tag was very popular and a number of blog posts have been written about its success including CILIP: More Popular Than Swine Flu!, Twittering Librarians, Twitter – better than a conference,  and CILIP 2.0 – Open Session.

Why?

Some might argue that the significant use of the tag could turn out to be a defining moment in CILIP’s movement towards engagement with Web 2.0. Those with an eye for preservation might argue that if this is the case we will want to preserve the tweets that embody this ‘crucial moment’.

How?

Luckily help was on hand in the form of Dave Pattern, systems manager at the University of Huddersfield Library. After the event Dave provided a good example of rapid software development (or in his words “a quick and dirty hack“!) when he quickly wrote a piece of software to harvest the tweets and make available a transcript of them. Dave used the Twitter search API to fetch the results (in JSON format) of the #cilip2 tag every 60 seconds. He then had to dedupe the results from the previous seaches. Once he’d got all the tweets he wrote a quick Perl script to process them and generate some HTML.

He also provided a Wordle visualisation of the content of the tweets.

This now means that the CILIP community continue to have access to:

• A record of the discussions, annotated with the time of posting.
• A Wordle cloud which provides a visual summary of the topics.

Software such as this could potentially be used by others to preserve tweets. For those interested Dave is happy to share the code.

If does however raise a number of questions: Should you keep a record of Twitter posts? If so, how would you go about doing it? Where should the responsibility lay? Should it be something event organisers need to consider? Is there a role for commercial companies to play or will this type of preservation just fall at the feet of interested participants (those keen on mashup creation)?

Is there scope for some sort of tool to be developed in this area? Should funding councils in specific areas be funding the creation of a more managed service? Should the tools be more generalised?

Some suggestions are already provided in the Handbook and on the JISC PoWR blog:

• Permission to capture Twitter
• Twitter Groups and Twitter Problems
• Preservation of your Tweets

This blog post provides the background for one of two case studies which will be displayed on a poster on Preservation Policies and Approaches for Use of Social Web Services to be presented at the Missing links: the enduring web workshop to be held on Tuesday 21st July 2009 at the at the British Library Conference Centre, London.

Richard Davis, ULCC, will be giving a presentation on Diamonds in the Rough: Capturing and Preserving Online Content from Blogs. Other members of the team will be presenting posters on the JISC-PoWR Project and on Preservation Policies and Approaches for Use of Web 2.0 Services.

In next few posts we’ll describe in more detail what we’ll be covering. Online registration is still open and closes on Friday 10th July 2009. We hope to see you there…

The study aims to:

• Investigate how UK Web archives are delivered to users now, and how they might be delivered in the future
• Define the long-term historical and research value of online content in the UK
• Look at different organisations that collect Web archives, and their interests

The study will run until late July 2009, and the results will be published on The National Archives and UK Web Archiving Consortium Web sites in August 2009.

We’ll published details on the availability of the study once it is published.

Not for everyone, I suspect, but I’m pleased to see a diversity of approaches being taken to explaining digital preservation concepts. And making it available on YouTube means that the animation can be easily used in a wide variety of contexts, such as being embedded in this blog post. What do you think?

Although in an update to his post Jeremy does admit that “no data has been destroyed yet; no links have rotted” and that his “toys-from-pram-throwage may yet prove to be completely unfounded” Jeremy is right to raise concerns regarding the recent announcement that “Yahoo [is] to shut down GeoCities“.

Some people, as illustrated by JR Raphael’s article in PC World entitled “So Long, GeoCities: We Forgot You Still Existed” are not losing any sleep over GeoCities demise whilst others, such as the Online Lunchpail blog feel that “the demise of GeoCities … proves my point that the U.S. government never should have approved the takeover of GeoCities by Yahoo!“.

From my perspective I feel that the concerns raised by Jeremy Keith (who, it should be pointed out, is a professional Web developers) will become more widely appreciated as ordinary Web users, who might have used the first generation of public-facing Web-hosting services such as GeoCities for their initial simple Web development activities, realise that their may be sentimental attachments to one’s early work – just as I regret having lost my scrap book from primary school (I remember writing “When I grow up I want to be a Beatle, sing ‘She loves you, yer, yer, yer’ and earn £100 a week“). And what of the social historians – have we lost our cultural memories of the initial take-up of the Web outside of the universities and business sector?

In a blog post by Jason Scott on the ASCII  “weblog of computer history, punditry and trivia” Jason describes the efforts being made to preserve content published on GeoCities. But Jason admits that

“I can’t do this alone. I’m going to be pulling data from these twitching, blood-in-mouth websites for weeks, in the background. I could use help, even if we end up being redundant. More is better. We’re in #archiveteam on EFnet. Stop by. Bring bandwidth and disks. Help me save Geocities. Not because we love it. We hate it. But if you only save the things you love, your archive is a very poor reflection indeed.”

What is to be done? Should the digital preservation for the general public’s digital heritage (as opposed to an institutional digital heritage) be left to volunteers? Or will future generations regard us as having failed in our responsibilities as previous generations failed to preserve the built environment and left us with the soulless shopping centres and high-rise building which were developed during the 1960s?

The PANDORA Librarian at the State Library of Queensland has just asked the question about permission. “Do we have a blanket permission for Twitter http://twitter.com/? That is, if we find a Twitter site we want to gather, and we get permission as well from say a politician or whoever has content on that site, can we go ahead and gather?”

The National Library of Australia, who have been involved in a library-centric web archiving programme for many years, replied with the following advice:

“We received advice from Twitter, when asking to archive the Prime Minister’s twitter page, which gives a clear indication that they have no problem with archiving,” said the NLA’s Manager of Web Archiving. “I have also been in contact with another Twitter staffer who also gave permission. So I think it is safe to assume that we may archive selectively from Twitter, depending upon receiving the permission of the Twitteree. However in the case of the PM, we didn’t seek that explicit permission as we have blanket permission, from his Party and the PM’s dept.”

And that advice from Twitter.com confirms this:

“Archiving the Prime Minister’s twitter updates is fine, since he owns the copyright to the messages that he posted. According to our Terms of Service:

Copyright (What’s Yours is Yours)

1. We claim no intellectual property rights over the material you provide to the Twitter service. Your profile and materials uploaded remain yours. You can remove your profile at any time by deleting your account. This will also remove any text and images you have stored in the system.

2. We encourage users to contribute their creations to the public domain or consider progressive licensing terms.

This is potentially a useful little development. It may send archivists in the direction of checking the terms of service of other third-party hosting organisations, such as Google Docs or SlideShare, to establish what’s permissible and what isn’t. If copyright and ownership issues are sidestepped or aren’t even mentioned, this could be a cause for concern.

In the meantime, presumably we must also give credit to our Australian colleague for devising the neologism “Twitteree”.

The archived material includes blogs (such as blogs for Bush). Currently permission is necessary for offsite access for researchers. All archived Web sites are available to researchers onsite at the Library of Congress.

Metadata

At the Library of Congress they are currently providing metadata for individual Web sites through brief records using the MODS schema. There is a MARC collection level record (for the collection itself) with a link to an entry/overview page for each collection that links to search and browse functions with MODS metadata for each individual Web site that was collected.

An overview of their metadata approach (at the collection and item levels) is available. They are also in the process of developing more formal descriptive metadata profiles for their digital content and have developed one for the Library of Congress Web archives.

For a list of publicly available Library of Congress Web archives and access to each, see the Library of Congress Web Archives site.

More information on activities at the Library of Congress are given in a Powerpoint presentation given at the Digital Library Federation 2008 Fall Forum.

Papers and posters are invited which address long term preservation issues in relation to Web content. Abstracts of not more than 300 words should be sent to the conference organisers by 8^th May 2009.

Coincidentally I recently received the following email from a service I subscribe to:

Our previous request to you to provide a new owner for the  list has not produced a response.  Therefore, we assume the list is no longer useful and aim to close it in one week’s time.
We would be happy to provide a zipped copy of the archives and any files on deletion of the list, should they be required.

In this case it appears that the service has been little used for over a year. And yet what if useful information is still available on the service? Is a week’s notice enough for users of the service to consider the implications of this decision, identify appropriate solutions and then implement them? And let’s not forget that this email was sent outside of term time when researchers could be away.

The email did not make it clear if data was to be deleted, the service was to continue to be made available in a read-only mode or the interface to the data hidden – all possible solutions if it is felt necessary for a little-used service to be withdrawn.

There’s still a need to establish the best practices when Web-based interfaces to services are to be removed, I feel. And such issues do not just affect the third party services outside of our community.

We will provide information on the detailed programme and how to register when this information is published.

Firstly, we don’t want to gather absolutely everything that’s presented as a web page in the wiki, since the wiki contains not only the user-input content but also a large number of automatically generated pages (versioning, indexing, admin and login forms, etc). This stems from the underlying assumption about doing digital preservation, mainly that it costs money to capture and store digital content, and it goes on costing money to keep on storing it. (Managing this could be seen as good housekeeping. The British Library Life and Life2 projects have devised ingenious and elaborate formulae for costing digital preservation, taking all the factors into account to enable you to figure out if you can really afford to do it.) In my case, there are two pressing concerns: (a) I don’t want to waste time and resource in the shared gather queue while Web Curator Tool gathers hundreds of pages from DigiRep, and (b) I don’t want to commit the JISC to paying for expensive server space, storing a bloated gather which they don’t really want.

Secondly, the above assumptions have led to me making a form of selection decision, i.e. to exclude from capture those parts of the wiki I don’t want to preserve. The parts I don’t want are the edit history and the discussion pages. The reason I don’t want them is because UKWAC users, the target audience for the archived copy – or the designated user community, as OAIS calls it – probably don’t want to see them either. All they will want is to look at the finished content, the abiding record of what it was that DigiRep actually did.

This selection aspect led to Maureen Pennock’s reply, which is a very valid point – there are some instances where people would want to look at the edit history. Who wrote what, when…and why did it change? If that change-history is retrievable from the wiki, should we not archive it? My thinking is that yes, it is valuable, but only to a certain audience. I would think the change history is massively important to the current owner-operators of DigiRep, and that as its administrators they would certainly want to access that data. But then I put on my Institutional records management hat, and start to ask them how long they really want to have access to that change history, and whether they really need to commit the Institution to its long-term (or even permanent) preservation. Indeed, could their access requirement be satisfied merely by allowing the wiki (presuming it is reasonably secure, backed-up etc.) to go on operating the way it is, as a self-documenting collaborative editing tool?

All of the above raises some interesting questions which you may want to consider if undertaking to archive a wiki in your own Institution. Who needs it, how long for, do we need to keep every bit of it, and if not then which bits can we exclude? Note that they are principally questions of policy and decision-making, and don’t involve a technology-driven solution; the technology comes in later, when you want to implement the decisions.

Since it is blogging applications that provide this stucture and behaviour (usually from simple database tables of Posts, Comments, Users, etc), perhaps we should consider making blogging software behave more like an archive. How difficult would that be? Do we need to hire a developer?

One interesting thing about WordPress is the number of uses its simple blog model has been put to. Under-the-hood it is based on a remarkably simple data base schema of about 10 tables and a suite of PHP scripts, functions and libraries that provide the interface to that data. Its huge user-base has contributed a wide variety of themes and additional functions. It can be turned into a Twitter-like microblog (P2 and Prologue) or a fully-fledged social network (WordPress MU, Buddypress).

Another possibility exploited by a 3rd-party plugin is that of using WordPress as an aggregating blog, collecting posts automatically via RSS from other blogs: this seems like a promising basis for starting to develop an archive of blogs, in a blog.

The plugin in question is called FeedWordPress. It uses the Links feature of WordPress as the basis of a list of feeds which it checks regularly, importing new content when it finds it, as Posts within WordPress.

I installed FeedWordPress a while ago on ULCC’s DA Blog, and set it up to import all of the ULCC-contributed posts to JISC-PoWR, i.e. those by Ed Pinsent, Kevin Ashley and myself. I did this because I felt that these contributions warrant being part of ULCC’s insitutional record of its activities, and that DA Blog was the best to place to address this, as things stand.

JISC-PoWR also runs on WordPress, therefore I knew that, thanks to WordPress’s REST-like interface and Cool URIs, it is easy not only to select an individual author’s posts (/author/kevinashley) but also the RSS feed thereof (/author/kevinashley/feed). This, for each of the three author accounts, was all I needed to start setting up FeedWordPress in DA Blog to take an automatic copy each time any of us contributed to JISC-PoWR.The “author” on the original post has been mapped to an author in DA Blog, so posts are automatically (and correctly) attributed. The import also preserves, in custom fields, a considerable amount of contextual information about the posts in their original location.

In many cases, I’ve kept the imported post private in DA Blog. “Introductory” posts for the JISC-PoWR project blog, for example: as editor of DA Blog, I didn’t feel we needed to trouble our readers there with them; nevertheless they are stored in the blog database, as part of “the record” of our activities.

This is, admittedly, a very small-scale test of this approach, but the kind of system I’ve described is unquestionably a rudimentary blog archive, that can be set up relatively easily using WordPress and FeedWordPress – no coding necessary. Content is then searchable, sortable, exportable (SQL, RSS, etc). (Note, by the way, what happens when you use the Search box on the JISC-PoWR blog copy in UKWAC: this won’t happen with this approach!)

For organisations with many staff blogging on diverse public platforms this would be one approach to ensuring that these activities are recorded and preserved. UKOLN, for example, manages its own blog farm, while Brian and Marieke have blogs at WordPress.com (as well as contributing to this one), and Paul Walk appears to manage his own blog and web space. This kind of arrangement is not uncommon, nor the problem of how an institution get a grasp on material in all these different locations (it’s been at the heart of many JISC-PoWR workshop discussions). A single, central, self-hosted, aggregating blog, automatically harvesting the news feeds of all these blogs, might be a low-cost, quick-start approach to securing data in The Cloud, and safeguarding the corporate memory.

There are more issues to address. What of comments or embedded images? Can it handle Twitter tweets as well as blog posts? Does it scale? What of look-and-feel, individual themes, etc? Now we start needing some more robust tests and decisions, maybe even a developer or two to build a dedicated suite of “ArchivePress” plugins. But thanks to the power and Open-ness of  WordPress, and the endless creativity of its many users, we have a promising and viable short-term solution, and a compelling place to start further exploration.

In the session, entitled “Who should preserve the web?” a panel will

“Outline the key issues with archiving and preserving the web and will describe practical ways of approaching these issues. Looking at the international picture and the role of major consortia working in this area, the session will also offer practical advice from the JISC Preservation of Web Resources (PoWR) project on the institutional benefits of preserving web resources, what tools and processes are needed, and how a records management approach may be appropriate.”

If you are attending the conference we hope you will attend the session and participate in the discussions. If you are attending one of the other parallel sessions you can meet the UKOLN members of the  JISC PoWR team at the UKOLN staff. And if you haven’t bookeda place at the conference (which is now fully subscribed) feel free to participate in the discussions on the online forum.

In order to help you spot the poster at what is likely to be a very busy event we’ve included an image of the poster in this post (which is also available on Slideshare, if you’ve like to see more details of the content of poster).

If you have an interest in the preservation of Web resources, feel free to come along to the UKOLN stand and chat to myself or Marieke Guy, UKOLN’s team members for the JISC PoWR project.

LIWA – Living Web Archives – began in early 2008, but as with many EU projects, its startup phase involved a lot of internal activity without much of a public face. As a result we didn’t pick up on its work in the JISC-PoWR handbook, but I’m sure we’ll rectify this omission in any future revisions.

To pick one example of LIWA’s areas of interest, it intends to develop tools which make it easier to take a temporal view of web archives and to maintain temporal consistency. Temporal consistency – or rather its absence – will be familiar to anyone who has spent time exploring sites in the Internet Archive, where different pages, or even portions of the same page (such as images) will have been archived on different days. This can lead to occasional surprises when navigating through archived content, with links taking one to pages that don’t have the expected content.

LIWA’s partner’s include Hanzo, a UK-based web archive services company that we covered briefly in the handbook; I hope we can explore their potential value to UK HE in the future.

The preamble sets the context for the paper:

Blogs, it seems, are everywhere these days, but what about the next day (and the next and the next …). Opinions vary on whether or not blogs merit preservation beyond the actions of a blog’s respective authors. This briefing paper does not contribute to that dialogue. Rather, it provides an overview of issues to be considered by organizations planning blog preservation programs. Blogs are the product of a network of players, including blog authors, service providers, and readers. Discussed here are some key attributes of blogs, and the characteristics and behaviors of these players, which may impact preservation activities.

During the JISC PoWR project we recognised that despite blogs initially being commonly characterised as ephemeral (as commented on in the DPE paper) their increasing importance and role in both the research context and in our cultural history is becoming apparent, and like other Web resources their preservation is a matter that needs to be addressed, somehow.

The PoWR blog has a number of interesting posts on the preservation of blogs including:

• Legal scholarship recognises long-term value of blogs
• Student Blogs
• Auricle: The Case Of The Disappearing E-learning Blog

There is a also a section on preservation of blogs in the JISC PoWR handbook.

The Archivists and Records Managers Twitter Group is up and running. You can register at http://twittgroups.com/group/archives.

I’m sure there will be lots of interesting posts.

The preservation of Twitter posts (tweets) has again been discussed in the blogosphere. Maureen Pennock commented in her post entitled ‘Making retrospective sense of cross media communications: a new archival challenge‘ that the increasing number of communication mechanisms presents a big problem for archivists.

She points out that “Some of our conversations are cross-media; they may start on Twitter, but they move to Facebook and then the blog. Capturing only one of those accounts means that only part of our conversation is captured. Okay, so you’re probably not interested in capturing our interactions in your archives. But you probably are interested in capturing interactions from important people (back to Stephen Fry and Obama again) and you will thus face the same issues.”

She then says “We all know the problems we’ve got in capturing and archiving emails. What of Twitter? How do you get Tweets out of the system and integrate them into a collection? What of Facebook data? And YouTube?”

It seems the Twitter challenge is becoming more real as it becomes increasingly mainstream.

This change has been accompanied by a new domain name – http://www.jiscdigitalmedia.ac.uk/ rather than http://www.tasi.ac.uk/.

Now the TASI service provided many useful resources on best practices for digitisation.  But what has happened to links to these resources? Will we get a 404 error message? Or, even worse, will we get a message saying the domain no longer exists?

The QA Focus briefing document on “Improving The Quality Of Digitised Images” contains a reference to a Digital Imaging Basics resource which was available at the URL <http://www.tasi.ac.uk/advice/using/basics.html>. Following the link takes you to the resource, which is now available at <http://www.jiscdigitalmedia.ac.uk/advice/using/basics.html>.

There seems to have been a simple mapping of resources from the TASI domain to the new JISC Digital Media domain. And as the original resource has ‘cool URIs’ (i.e. they had no dependencies on a specific technology (such as a CMS, Java server pages, etc.) it was technically not a difficult task to migrate the links to the new domain.

Well done TASI / JISC Digital Media. The challenge now is to see how long such redirects will continue to function.

The only way to archive digital information is to keep it moving. I call this movage instead of storage. Proper movage means transferring the material to current platforms on a regular basis — that is, before the old platform completely dies, and it becomes hard to do.

The reasons for this are the continual changes in the formats and degradation of the storage media. I think this relates to the ideas discussed previously on this blog about an emphasis on ongoing access to Web resources rather than the preservation of such resources. In the case of Web resources the need tends to arise from changes in the technologies used to deliver the Web services rather than the formats themselves.

But whether a new term needs to be created is questionable – after all, Kevin Kelly is simply describing the well-established concept of migration of formats. As described in a glossary entry on the DCC Web site:

Migration: A means of overcoming technical obsolescence by transferring digital resources from one hardware/software generation to the next. The purpose of migration is to preserve the intellectual content of digital objects and to retain the ability for clients to retrieve, display, and otherwise use them in the face of constantly changing technology.

Despite this reservation I still think it’s good to see a slightly different variant on the ideas which have been discussed on this blog reaching a new community.

Below is an outline of the ‘wiki’ situation at the University of Edinburgh:

At Edinburgh University our main effort to date has been making sure that wikis are retention scheduled, and considering what the ideal retention period for a wiki should be. As part of setting up any new wiki space the University records details such as space owner and proposed use, but due to the wide variety of uses it is difficult to specify a generic retention period. There is the option for the space owner to delete a wiki space; however the most likely scenario is that a space atrophies over time, the owner stops engaging, and it is therefore then up to the University to be proactive in identifying and pruning out dead spaces.

At present the service policy talks about a default retention period of 1 year, which is primarily to make space owners aware that if not used their space may be deleted. If we have anything that requires long term migration we would look into outward migration; either to a new system or to an archive.

I found it very encouraging to see this pro-active and practical-minded approach to the management of wikis. In many ways Edinburgh’s RM approach vindicates a lot of the RM advice which we have recommended in the PoWR Handbook; as we say early on, we must manage resources in order to preserve them. It is also encouraging that in Edinburgh’s case at least the wiki problem is considered primarily in terms of information and staff management, and not exclusively in terms of the technological solutions that might be applied.

In particular:

1) Edinburgh: “Make sure wikis are retention scheduled”.

• PoWR: “Deciding which aspects of your web resources to capture can be informed to a large extent by your Institutional drivers, and the agreed policies for retention and preservation.”  (p 22)

2) Edinburgh: “Consider the ideal retention period for a wiki”.

• PoWR: “The attraction of bringing a website in line with an established retention and disposal programme is that it will work to defined business rules and retention schedules to enable the efficient destruction of materials, and also enable the protection and maintenance of records that need to be kept for business reasons.”  (p 93)

3) Edinburgh: “Make space owners aware that if not used their space may be deleted”.

• PoWR: “Quite often in an academic context these applications rely on the individual to create and manage their own resources. A likely scenario is that the academic, staff member or student creates and manages his or her own external accounts in Flickr, Slideshare or WordPress.com; but they are not Institutional accounts. It is thus possible with Web 2.0 application for academics to conduct a significant amount of Institutional business outside of any known Institution network. The Institution either doesn’t know this activity is taking place, or ownership of the resources is not recognised officially. In such a scenario, it is likely the resources are at risk.”  (p 42)

4) Edinburgh: “The service policy talks about a default retention period.” This approach seems to incorporate rules as part of setting up any new wiki space, starting to manage the resource at the very beginning of the record’s lifecyle.

• PoWR: “If  we can apply a lifecycle model to web resources, they will be created, managed, stored and disposed of in a more efficient and consistent way; it can assist with the process of identifying what should and should not be retained, and why; and that in turn will help with making preservation decisions.” (p 34)

5) Edinburgh: “If we have anything that requires long term migration we would look into outward migration; either to a new system or to an archive.”

• PoWR: “Migration of resources is a form of preservation. Migration means moving resources from one operating system to another, or from one storage/management system to another. This may raise questions about emulation and performance. Can the resource be successfully extracted from its old system, and behave in an acceptable way in the new system?”  (p 33)

• “The usual aim of archival appraisal has been to identify and select records for permanent preservation. Quite often appraisal has taken place at the very end of the lifecycle process (although records managers intervene where possible at the beginning of the process, enabling records of importance to be identified early).”  (p 36)

 JISC and the Advisory Services have been looking at ways to be more agile and flexible to respond to the changing needs and demands of thefurther and higher education communities. The outcome of this review is to create a new company called JISC Services.

JISC infoNet, JISC Legal, JISC TechDis, Netskills, Procureweb and TASI are coming together to create JISC Services which will formally come into existence on 1 August 2009.

The aim of the new company is to create a more flexible and comprehensive source of advice, with increased opportunities for addressing new and changing needs across the community. This change is designed to ensure that our services continue to offer the internationally acclaimed advice for which they are renowned. Putting the further and higher education communities at the centre of what we do will be strengthened by working together as one company to deliver expertise and advice.

You will still be able to access all of the services you currently value via the usual channels and over the next few months the services will increasingly join together at events, on projects and in producing resources.

Find out more about the JISC Services at: http://www.jisc-services.ac.uk

I recently wrote about the closure of organisations and best practices for preserving the resources hosted on the organisational Web sites. This case is rather different – rather than closing down organisations JISC is building on the strengths of the advisory services and seeking to provide benefits to the user community by providing a more seamless interface (and remember, if the advisory services were regarding as failing to deliver a valuable service we might have expected the organisational changes to have provied an opportunity to close any lame ducks).

The challenge, from the perspective of Web site preservation, is to try to ensure that valuable resources are not lost in the merger process.  I feel that this change could provide valuable lessons for the wider community – the JISC Advisory Services, after all, won;t be the last organisations to be reorganised! And let’s hope that the lessons are based on a successful migration of the Web resources, and not lessons on what can go wrong!

We are lucky enough to start 2009 with a guest blog post from Dr James Currall, Director of Information Strategy, IT Services & HATII Senior Research Fellow, University of Glasgow.

James has been involved with the highly successful Glasgow MPhil (now MSc) course in Information Management and Preservation since it inception, in which he teaches about the transition from storage of information on physical to digital media, information security, the role of numbers as information and a variety of other topics including risk and information management as an investment. In this latter context he was the Project Director of the espida project which developed a sustainable business-focussed model for digital preservation. He gave a plenary talk on Web preservation at last year’s Institutional Web Management Workshop (IWMW 2008) entitled The Tangled Web is but a Fleeting Dream … but then again … which was very well received and is available to watch on Google Video.

And I’ll pass you over to James…

A few weeks back, I was involved in a discussion about the skills required by people involved in Digital Curation and much of that discussion was based around the DigCCurr Project which has a long list of skills, some of which are specific to Digital Curation, but many of which are rather of a more general nature. And this set me on a dangerous course – thinking ….What exactly is this ‘profession’ of digital curator that DigCCurr amongst others are trying to define?

Let us rewind to say the second half of the 16th century and let us suppose that you were charged by Mr Shakespeare’s publishers with curating ‘The Scottish Play’.  What would you have done?  What exactly is this ‘information object’?  Is is the fonts, the layout, the pagination, the language, the story, the stage directions or what?  In spite of the absence of the profession of ‘paper curator’ we have inherited a rich heritage.  Along the way, many items will have been lost – it was always thus and, in spite of the optimistic techno-determinism of some, it always will be EVEN IN THE DIGITAL AGE. I would argue that this is all good and necessary and whilst I would mourn the passing of Algol, Reverse Polish Notation, amplifiers based on thermionic valves or chunky discrete solid state components, vinyl records, reel to reel tape and other really splendid ideas that were IMHO much better than the ‘mass market equivalents’ that replaced them, we have to discard much of our baggage as we move on.

So what is this preservation activity all about?  Is it not about the preservation and curation of information not of digits?  During a session with my MSc students, We visited the Way Back Machine and had a look at the University of Glasgow Web site (you wondered when I would get on to the web didn’t you?).  The page that we selected at random was from 18th October 2000. As a web page it is rather uninteresting, when I looked at it today there was no style sheet, the graphics were all missing and it was generally rather uninspiring, but ….  what is interesting is the headline news story ‘Funeral of the First Minister, Donald Dewar’. For those of you firth of Scotland, Donald was a leading light in the establishment of devolution for Scotland and the first First Minister of the devolved administration in Scotland. He was a graduate of the University of Glasgow and his premature passing at the age of 63 was tragic.  The news story is about ‘administrative’ details of his funeral and the passage of his cortege past the University – details of importance in relation to the history of the University and perhaps of Scotland.  It is the information contained in the web pages that is of interest and importance, whilst the layout of the pages and such ‘technical’ details of passing interest as the ‘container’ for that information.

So with 2008 now ended let us bury the idea that the digital needs its own ghetto that we need to prepend everything with ‘digital’, be it: curation, preservation, art, culture, revolution, etc.  Digital artifacts are the currently ‘fashionable’ containers for information and whilst the term continues, the technologies underneath that are radically different at every turn and often require as much conversion one to another as a paper to magnetic disc conversion.  It is not the containers that are important but what they contain.  The Eastern concept of ‘Pointing at the Moon’ has something to say here.

If we come to regard preservation/curation as a finger pointing to the moon; we might come to mistake the finger for the moon and never see beyond it to the moon itself.

This short clip of Bruce Lee in ‘Enter the Dragon‘ (1974) captures something of this in a different context.

I am also reminded of the auditors in Terry Pratchett’s ‘Thief of Time’ who take a great painting and break it down into flakes of paint which they put in little piles of each colour and then spend time looking to see where the art has gone!  These auditors are described in the Wikipedia article for DiscWorld thus:

“The Auditors, cosmic bureaucrats who prefer a universe where electrons spin, rocks float in space and imagination is dead, represent the perils of handing yourself over to a completely materialist and deterministic vision of reality, devoid of the myths and stories that make us human.“

From http://en.wikipedia.org/wiki/Discworld#Elves_and_Auditors

In 2009 we need to see digital preservation and curation as ‘last year’s model’, of course we need to understand the importance of custody, metadata and identifiers, but above all we need to understand the centrality of the information in the artifacts that we are seeking to curate and preserve.  This piece is recognisably ‘Currall’ not because of a digital signature, not because it is on his web site and not because the owners of the JISC PoWR say it is – it is ‘Currall’ because of its recognisably iconoclast position, poor grammar and tortured logic – that is what needs to be preserved!

Information is the thing (even if that is hard and technology is relatively easy) – lose sight of that and the game is a bogey.

PS if you are interested in a rather more rigorous treatment of this topic you might like to access “Authenticity: a red herring?“ (doi:10.1016/j.jal.2008.09.004)

Various people at the University of Leeds were active in Web development activities back then. My role was in promoting its use (and I’ve discovered a copy of a special issue of the University Computing Service newsletter on the theme on online information services – in particular the Web – which is available on the Internet Archive). But in addition the Chemistry Department were, in conjunction with Imperial College, developing services which provided access to molecules on the Web; a colleague in the Computing Service provided access to the University Libraruy catalogue and Nikos Drakos, a researcher in the Computer Based Learning Unit, wrote the Latex2HTML conversion software (which was first announced in May 1993).

Fifteen years later my memories of our early involvement with the Web are beginning to fade. But as I knew this would happen I write a history of the various activities of colleagues at the University, which was published on the University”s Web site. Sadly, but perhaps inevitably, over time this resource was deleted, no doubt following a reorganisation of the Web site.

But this does not necessarily mean that the information is no longer available. As well as being an early adopter of the Web, the Computing Service had also had long standing involvement in digital preservation. And so the file should still be available on the University’s archive service. But although the bits and bytes may still be available, what are the processes needed for this resource to be retrieved?  Is this a service which the University offers? And is it a service which can be provided to a former member of staff, who left the University over 13 years ago?

As JISC PoWR project team members have commented previously, digital preservation isn’t just about the technical aspects of preservating bits in a format suitable for processing in the future – it’s also about the policies and the procedures.  And I think it’s time I send an email to my former colleagues to see ifthis resource can be retrieved.  I’ll provide details of my experiences in a future post.

The seminars will take place in Rome, and are free to attend. If you require further information please email:
<info@casparpreserves.eu>

Neil went on to add that “The PoWR handbook recognises that preservation is not an end in itself, but that it can complement an institution’s mission, whether that be improving the quality of research, conforming with national policy or avoiding the threat of legal action. It will evolve following the practical experience of its use to ensure it remains at the forefront of best practice advice for web preservation issues“.

The JISC PoWR project has been formally completed – but the interests of the project team (UKOLN and ULCC) in the area of preservation continues. We have agreed that we will continue to publish posts on this blog which are relevant to the area of the preservation of Web resources for a period of time- we will seek to publish at least 3 posts per month. Around Easter time we will review the status of this blog. As well as posts from members of the JISC PoWR project team we would also welcome guest blog posts from the community. So if you would like to write something about your interests in the area of Web site preservation please contact Marieke Guy (email M.Guy@ukoln.ac.uk).

But for now on behalf of the JISC PoWR team I’d like to wish everyone a happy and enjoyable Christmas.

…will build upon the fundamental assumption that blogs are an integral part of today’s legal scholarship.

and goes on to say:

This symposium will bring together academic bloggers, librarians, and experts in digital preservation …. Symposium participants will collectively develop innovative practices to ensure that valuable scholarship is not easily lost.

Join the conversation now by tagging items you think are relevant to this symposium with the del.icio.us tag FTLS2009.

It’s interesting to observe that this is an example of a community acting to preserve information of interest that is likely to be scattered over many institutions and none. (I suspect a fair amount of blogging in this area is done by practitioners who aren’t at an academic institution.) One of the concerns we identified in PoWR was that much material of this type was unlikely to be preserved as a result of institutional interests, unless one institution tried to bring materials like this into the remit of its special collections (and some have done this.)

The conference web site goes on to say:

This unique symposium will seek answers to the questions:

1. How can quality academic scholarship reliably be discovered?
2. How can future researchers be assured of perpetual access to the information currently available in blogs?
3. How can any researcher be confident that documents posted to blogs are genuine?

The symposium will include a working group break-out session to create a uniform standard for preservation of blogs, a document to be shared by bloggers and librarians alike.

That last goal of a uniform standard for blog preservation looks like a tall order and it will be interesting to see what emerges from this group, and what its wider relevance might be. But its a clear demonstration of the value of web material to some research communities, and their willingness to do something about it if their institutions can’t, or won’t, help them.

The message states :

MLA North East Websites after 12^th December, 2008

MLA North East over recent years has set up several websites which we have managed on behalf of the sector. This brief note is to inform you of the arrangements made for each of the sites.

www.mlanortheast.org.uk:  a holding page will refer visitors to MLA council site at www.mla.gov.uk All other content will be taken down at 4.00pm on Friday 12^th December, 2008.

www.thenortheast.com:  currently a portal to our sector’s on-line stores selling local studies material and other ephemera. The content will be taken down at 4.00pm on Friday 12^th December, 2008. The domain name is now owned  by One NorthEast.

www.archivesnortheast.com:  a portal to North East archives services, providing links to catalogues and paid-for  professional support in researching archives. This will continue under the auspices of the North East Regional Archives Council [NERAC] Contact Liz Rees liz.rees@twas.org.uk

www.wellinever.info: a portal to learning resources to teachers, pupils, parents and carers providing venue guides, information regarding learning visits and links to some of the sector’s regional on-line learning resources. This will continue under the auspices of Tyne & Wear Museums. Contact ian.thilthorpe@twmuseums.org.uk

www.primarysources.org.uk:  basic skills resources developed by primary teachers, working alongside learning professionals from six archives in the North East region and  designed for use schools.  These resources offer a fresh and engaging approach to teaching basic skills. This will continue under the auspices of Durham University. Contact andrew.preater@durham.ac.uk

www.discs-uk.info: DiSCS provides an online directory of information technology (IT) and digital services suppliers to work with the cultural and heritage sector. This site has transferred and is managed by The Collections Trust www.collectionstrust.org.uk

www.tomorrows-history.com: The regional local studies site for archives and record offices, libraries, museums, archaeology services, the region’s universities and commercial organisations.

Additionally, community groups have created one hundred local history projects. This will continue. The domain name is now owned by Newcastle City Council. The site is managed by Newcastle City Library Services. Contact Kath Cassidy kath.cassidy@newcastle.gov.uk

www.oralhistorynortheast.com: The site for oral history in the North East of England. Support for individuals and organisations undertaking oral history projects, to provide focus and support and a forum for the sharing of ideas and experience. This site is closed.

I think this demonstrates some good practices of what organisations which have set up or commissioned Web sites should do if they are forced to close, either due to changes in Government funding and policies (as is the case with the MLA Regional Agencies).

We can see that the Web site address and a brief summary of its purpose is provided, details of when the site ceases operation, contact details and, in a couple of cases, details of how the service is being continued by other organisations.

I know the implications of the demise of our organisation on the Web services we are providing isn’t something that we like to think about. But in a personal capacity once we reach a certain age and become aware of our resoponsibilities to others we due ten to make plans for what happens after we die, perhaps by making a will. So shouldn’t our organisations be making similar plans in case the oprganisation ceases to exist. And at a time of the credit crunch this is even more important than it used to be.

the Internet Archive will undertake a comprehensive crawl of the .gov domain.

The article concluded with a summary of the role of the Internet Archive:

The Internet Archive is a high-tech nonprofit, founded in 1996 by Brewster Kahle as an “Internet library” to provide universal and permanent access to digital information for educators, researchers, historians, and the general public. The Internet Archive captures, stores and provides access to born-digital and digitized content, and leads the development of Heritrix, the open-source archival web crawler, used to facilitate the collection of web data for this project.

What role might the Internet Archive have in the UK, I wonder?

The BBC News Web site has published an interesting article entitled Future in Bits asking how can the ever-changing Web be archived bearing in mind the dilema of the malleable nature of digital information.

The article draws attention to the fact that no UK-based commercial online newspapers are currently being archived.

David Stuart, a research fellow in Web 2.0 Technologies at the University of Wolverhampton is quoted as saying:

“The lack of an exhaustive archive of the UK web space not only risks the loss of information on web pages that are changed or taken down,” he said. “It also undermines the value of pages that link to them; the value of the web comes as much from the hyperlinks between pages as the contents of the web pages. This is especially true in the blogosphere, where so much of the content created by the public is built upon the foundations of traditional news stories“

Jessie Owen, digital continuity project manager at the National Archives explains that the key to archiving is preparation.

This is something the JISC PoWR handbook can offer help with.

This text has been mentioned at PoWR workshops, on the PoWR blog and on the JISC Information Environment Team blog. I can honestly say that it has had quite an impact on my thinking with regard to preservation and Web 2.0 resources, other members of the PoWR team may agree.

As I say in the conclusion:

This book offers up much food for thought. Bailey wants to wake up and shake his community. He wants to make them see that all is not well in the records management world and that if they don’t start moving with the times then they will be pushed out of the way. He contends there is a very real possibility that records management as we know it will cease to exist; it will be outsourced.

Go on, have a read.

JISC have announced the publication of a study on Digital Preservation Policies which can be downloaded in PDF format from the JISC Web site.

This study aims to provide an outline model for digital preservation policies and to analyse the role that digital preservation can play in supporting and delivering key strategies for Higher and Further Education Institutions. Although focussing on the UK Higher and Further Education sectors, the study draws widely on policy and implementations from other sectors and countries and will be of interest to those wishing to develop policy and justify investment in digital preservation within a wide range of institutions.

The study concludes “that for institutions digital preservation must be seen as “a means to an end” rather than an end in itself: any digital preservation policy must be framed in terms of the key business drivers and strategies of the institution.”

Two tools have been created in the study:

1) a model/framework for digital preservation policy and implementation clauses based on examination of existing digital preservation policies;

2) a series of mappings of digital preservation to other key institutional strategies in UK universities and colleges including Research, Teaching and Learning, Information, Libraries, and Records Management.

These tools are definitely worth taking a look at if you are embarking on a Web preservation strategy.

Version 1.0 is not substantially different from Version 0.3, released last month, but we have taken some recent feedback comments into account, added a useful and comprehensive Index to the Handbook, and made corrections arising from the proof-reading stage.

In the spirit of showing our commitment to the management of Web 2.0- type resources, we are also making the Handbook available on issuu and Scribd.

There was an interesting editorial by Siobhan Butterworth in Monday’s Guardian about ‘unpublishing’ – removal of content once placed on the Internet.

Siobhan explains:

Judging from the numbers of emails I get from people asking for material to be removed from the Guardian’s electronic archive, it seems that some people still don’t fully understand the implications of speaking to or even writing for a news organisation in the web age.

She goes on to argue that:

The web makes a lie of the old cliché that today’s newspaper pages are tomorrow’s fish and chip wrapping. Nowadays, as I’ve said before, the things you say about yourself in a newspaper are more like tattoos – they can be extremely difficult to get rid of.

It seems a good rule to set yourself when publishing content (or allowing content to be published about you) on the Web (and the same rule could apply to all emails sent) is: Are you happy for the whole world to see this?

The concepts that what you publish can be seen by all and that nothing truly disappears from the Web have slowly begun to embed themselves in our consciousness. This has been fuelled by a number of horror stories about employers accessing the Facebook (and Flickr and other socialnetworking sites…) accounts of perspective employees. A New York Magazine article from February this year quoted a teenager as saying “If I don’t delete it, I’m still gonna be there. My generation is going to have all this history; we can document anything so easily.” Many people do realise that the off-hand comments and inappropriate photos we blog or publish can come back to haunt us.

While in some ways this might seem to be the flip side of what JISC PoWR is about deletion is very much part of a preservation strategy.

So it pays to remember that:

• stuff can disappear, and quite often it is the really good stuff we wish we’d held on to.

but

• stuff that we wish would come out in the wash can stain for good.

So maybe we need to give some thought to how (and should things) be ‘unpublished’? What do people think?

The slides of the talk are now available from Slideshare and embedded below.

The presentation was well received and sparked a lot of interest particularly from delegates from US libraries. Donald Grose, Dean of Libraries from the University of North Texas, informed me that they have been preserving many of the US government Web sites for a number of years. See this related press release. Looking into Web resource preservation activity in other sectors and possibly other countries is definitely an area of interest for the future. Hopefully the JISC PoWR project will be able to talk more to Donald about his work in the future.

From the PoWR point of view, one thing I found useful was the summary statement from Iceland which suggests that the community is now starting to agree on three main approaches to selection – bulk/domain harvesting, thematic, and event-based. Sometimes I wonder if these approaches are too library-influenced (looking at websites primarily as publications, rather than as records, whereas in PoWR we are suggesting that HFE websites contain and function as a mixture of both), but it helped me sharpen up my thinking and I fed this idea back into the PoWR Handbook’s chapter on Selection.

Since they were all representing libraries, most of the presenters tended to focus on legal deposit (and attendant permission problems) as the biggest hurdle to gathering websites – and the one that eats up the most in terms of resources. Denmark have not resolved this completely, and although they have gathered lots of material from the web, they don’t make it widely available – only allowing controlled access for research or statistical purposes. France, on the other hand, have had legal deposit laws in place since 2006. Australia have found this such a problem (despite the strenuous efforts they have made to get the law changed) that it pretty much forced the decision to go the curatorial route. All permissions are negotiated, but the collections are shaped to a certain extent by community input.

With my archivist hat on, I sat up when we were told quite categorically that registration and cataloguing of web resources would not work, and that no-one should ever even attempt it. The BL admitted they were ‘not really collecting metadata’. If this is true, I immediately started to wonder why both PANDAS and Web Curator Tool (which I have used) have conventional Dublincore metadata elements built into their workflow? Don’t we anticipate cataloguing archived web resources in some way? I almost asked a question about this, until I reflected on the use of NutchWax and full-text indexing (which is probably acceptable until we can come up with some form of semantic tagging or automated metadata extraction for web resources).

Then I found myself reaching for the mic to ask a question about what I call ‘endangered resources’. We have already blogged about this on dablog, when my colleague Joanne Anthony raised a question about a web resource owned by a smallish institution which suddenly found itself with its funding removed. Was there anything such an Institution could do, I asked the panel, to preserve its website? And what were the pro-active steps being taken by these National Libraries to rescue or identify resources at risk? I know that UKWAC, for example, offers a public submission service on its website, although it is not very prominent or visible, nor is it quite clear what happens to requests for archiving once the form has been filled in. I received some interesting replies, including the amusing anecdote from France which suggests that their archival collections have been accessed by red-faced politicians who have accidentally deleted their own blogs. However, I still wasn’t quite sure what national initiatives exist to address what I perceive as a significant gap in the preservation of unrecognised (and therefore uncaptured) resources.

There was an entire session devoted to various web archiving initiatives on the second day, which at first sight one might think is of particular relevance (almost as much as Brian’s presentation, one might think.) I wasn’t at this session – it was one of those running in parallel tracks, and I was speaking in the other track – but Ed Pinsent was and will be writing at more length about it soon. But even without attending, I’m aware that many of the projects, operating as they do within their national domains in Australia or elsewhere, won’t have much role in helping save UK University web content (unless we move our domains to .edu.au – there’s a thought.) Even when the BL realises its long-term aim of harvesting across the entire UK web domain, it still will be selective in some ways about what it captures – about depth and frequency of harvests, and about the type of content. You won’t be able to depend on those institutions to capture what you want to be captured. So if these initiatives aren’t going to meet all our needs, do we need to do it ourselves ? The PoWR project thinks not, but that is one of the options institutions will need to examine. The work the IIPC is doing to develop harvesting and access tools will be of interest to those few institutions that feel able to operate these tools themselves – not something to be undertaken lightly.

Yet there was much of relevance at iPres2008. One recurring them, picked up at the outset by Lynne Brindley and in Steve Knight’s closing remarks, was that ‘digital preservation’ is not the term to be using in discussions with our institutions and the world, echoing remarks on the DCC blog which Brian later picked up on here. Steve prefers the phrase ‘permanent access’. which is indeed outcome-focussed. However, we’ve also said in PoWR that preservation isn’t always forever, so I would prefer something a little more all-embracing – ‘long-lived access’ might fit.

The sessions covering things like significant properties also touched on issues that PoWR is concerned with. When we decide to preserve something, what is it that we’re really trying to keep ? Most forms of preservation change the original object in some way, just as long-life milk isn’t the same as pasteurised, and neither are quite as tasty as fresh milk (or so I’ve been told.) This is clearly still a very difficult problem, and one that (to my mind) demonstrates that the digital preservation community hasn’t even developed a clear problem statement, much less a fully worked-out solution. So, in the meantime, we need to be pragmatic and do what seems best at the time. Always a good plan.

The slides of the talk (in which I focus primarily on preservation within a Web 2.0 environment) are now available and are embedded below.

There is also a video recording of the talk available (although I haven’t yet been able to upload the video to Google Video to allow it to be embedded in other Web pages, I’m afraid).

I should also add that Chris Rusbridge provided a comprehensive report on the conference. I was pleased to read Chris’s comments on my talk which he described as “a very entertaining talk, and well worth looking up“. He went on to describe me as “not a preservationist, but is a full-blown technogeek discussing the roles of the latest Web 2.0 technologies on his blog, in his role as UK Web Focus“. And this technogeek was particularly pleased to read that the JISC PoWR “project achieved a strong level of interaction through its several workshops“.

The slides and accompanying paper are available from the UKOLN Web site.

• European Archive, 16-17th of October 2008, Paris
  Two-day training session on Web Archiving. The training will cover all aspects of Web Archiving for librarians, archivists as well as technicians in charge of Web archiving. Special attention will be given to providing the necessary background on Internet technologies in general and Web publishing in particular to understand the media and requirements for its preservation.
  For further details have a look at the European Archive Web site

• Records Management in the digital age, 24th September 2008, London
  The theme of UNICOM’s networking and discussion evening is the challenge of information management and records management in the digital age and in the light of collaborative technologies, social networking and Web 2.0. It features three short presentations followed by interactive discussion and a drinks and networking session. The evening will be held at the Regency Hotel, London from 5pm – 7:30pm.
  For further details have a look at the UNICORM Web site.

• International Digital Curation Conference, 1st -3rd December 2008, Edinburgh
  The Digital Curation Centre is holding their 4th International Digital Curation Conference, which will comprise of a mix of peer-reviewed papers, invited presentations and international keynote speakers
  For further details see the DCC Web site.

The main presentations are now available for download:

• Presentation 1: Introduction to JISC PoWR (Kevin Ashley, ULCC).
  Presentation: [Slideshare] – [pres1.ppt PowerPoint file] – [pres1.pdf PDF]
• Presentation 2: Records Management vs. Web Management: Beyond the Stereotypes (Marieke Guy, UKOLN).
  Presentation: [Slideshare] – [pres2.ppt PowerPoint file] – [pres2.pdf PDF]
• Presentation 3: Web Preservation in a Web 2.0 Environment (Brian Kelly, UKOLN).
  Presentation: [Slideshare] – [pres3.ppt PowerPoint file] – [pres3.pdf PDF]
• Presentation 4: The JISC-PoWR Workshops – Inputs and Outcomes (Marieke Guy, UKOLN).
  Presentation: [Slideshare] – [pres4.ppt PowerPoint file] – [pres4.pdf PDF]
• Presentation 5: The JISC-PoWR Handbook – Explaining Web Preservation (Kevin Ashley, ULCC).
  Presentation: [Slideshare] – [pres5.ppt PowerPoint file] – [pres5.pdf PDF]
• Presentation 6: The JISC-PoWR Handbook – Identifying Web Issues (Richard Davis, ULCC).
  Presentation: [Slideshare] – [pres6.ppt PowerPoint file]- [pres6.pdf PDF]
• Presentation 7: The JISC-PoWR Handbook – Recommended Approaches (Ed Pinsent, ULCC).
  Presentation: [Slideshare] – [pres7.ppt Powerpoint file] – [pres7.pdf PDF]

An event report on the workshop written by Christopher Eddie, University of Oxford, is also now available in the Ariadne Web magazine.

It turns out that it’s not a separate product, nor a plug-in, but a built-in part of Adobe Acrobat. It was first available as a free add-on for Acrobat 4 in 1998 or 1999 , and I think it was then that I first saw this demonstrated at the PRO (as it then was) – hence my misunderstanding. Tools like this have their place, but (like all web preservation technologies) they also have their drawbacks. PDF’s print-oriented format isn’t a good match to some sites, much as some sites don’t look good when you try to print them. (In fact, I believe that Acrobat Web Capture effectively uses the browser’s print engine combined with PDF writer pseudo-printer to do its work, so there will be a close correlation.) But we’ll be covering this tool, along with others, in the handbook.

handbookdraft_10092008.pdf

Attached is an early draft of the PoWR Handbook. This release is timed to coincide with our third and final workshop in Manchester.  We hope to gather feedback from that workshop, and from any comments received via this blog, to feed into the final version of the Handbook.

You can also browse an Issuu.com version of the Report here.

Comments can also be sent direct to Ed Pinsent, if you would prefer not to comment here.

The challenges are significant, especially in terms of how to preserve Web resources. No doubt the institutional repository will play a role. Arguably, the absence of a solution to the preservation of Web resources leads to either retention or deletion, both of which carry risks. The workshop’s core message to practitioners was therefore to start building an internal network amongst relevant practitioners as advice and guidance emerge.

My thinking about this matter was certainly stimulated and I look forward to the next two workshops, and the handbook that will result. Web preservation is an issue which was always important but now grows increasingly urgent.

It is hoped that a trip report on the third workshop (for which bookings are currently still open) will be published in a future Ariadne.

The e-learning team at the University of Bath was one of the early adopters of blog technologies to provide a forum for reflecting on e-learning in a Web content.   The blog was set up by Derek Morrison when he was head of the e-learning unit. Derek had an interest in exploring the potential of new technologies, with one example of this being the series of podcast interviews he recording and made available on the blog back in 2005.  This included an interview with John Dale about the innovative blogging service developed at the University of Warwick (the first large scale student blogging service in the UK)  and, perhaps not as noteworthy, an interview with me on my reflections of the WWW 2005 conference.

The name of the e-learning team’s blog was Auricle, which has an advantage of being a very Google-friendly name, and a Google search for “Auricle Bath” finds links to the blog itself and various page which refer to the blog. Unfortunately it seems that the blog no longer exists – following a link to the blog’s home page gets a 404 error message:

The web address http://www.bath.ac.uk/dacs/cdntl/pMachine/morriblog.php was not found. It may have moved, or it may no longer be available.

How unfortunate – all that potentially valuable historical content giving views on the potential of the Web (including technologies such as blogs and podcasts) to enhance the quality of the student’s learning experiences now no longer available.  And how should the University of Bath feel about this loss of its intellectual endeavours and the role that the University had in being one of the early adopters of blogs by an e-learning team.

Why Did The Blog Disappear?

The URL for the Auricle blog provides an indication of some of the reasons for the disappearance of the blog: dacs refers to the Division of Access and Continuing Studies and cdntl to the Centre for the Development of New Technologies in Learning – but neither of these departments still exists.  Following staff departures and organisational changes, support for learning at the University of now provided by the Learning and Teaching Enhancement Office (LTEO)  with the e-Learning Team having responsibility for managing and supporting e-learning developments.

In addition to these organisational changes, the pMachine part of the blog’s domain name refers to the pMachine blog engine and morriblog clearly refers to  Derek Morrison, who left the University a number of years ago to support the HE Academy’s Pathfinder programme.

It is perhaps not surprising that following such changes and the influx of a large number of new staff in the e-Learning Team that the Auricle blog got lost somewhere along the way!

Can We Retrieve Any Of The Resources?

Is it possible to retrieve any of the blog posts and related resources? Is it possible to obtain any details about the blog, such as when it was launched, the number of posts published during its lifetime, how popular it was and, perhaps, the impact that the blog may have had?

Since the blog was public, as opposed to a blog which was restricted to members of the University of Bath, the contents of the blog have been indexed by Google. And using a combination of search terms, such as “Auricle Bath”, it is also possible to discover Web resources which cite the Auricle blog. This helped me to find a blog post on Stephen Downes’s blog on The Weblog as the Model for a New Type of Virtual Learning Environment? in which Stephen (a high profile Canadian e-learning guru)  clearly acknowledged the importance of Derek Morrison’s views on the potential of the blog as providing “the basis for a distributed, not centralised, information and learning object system“:

The author of Auricle nails it. “In the weblog, however, the announcements, articles, stories are the raison d’etre’ so much so that, not satisfied to present articles from one source, the weblog has the temerity, due to the adoption of the RSS standard, to receive syndicated stories from other sources and, in turn, offer it’s own portfolio of articles for use by others. For example, a blog supporting a programme or module could be the vehicle by which faculty post date and time-stamped short articles relevant to the course but which also link to related, but distributed, learning resources which are presented via RSS feeds. Such feeds can be static or dynamic so that updated RSS formatted information will be reflected in whatever application is displaying it, e.g. a la Auricle’s RSS Dispenser. Here then is the basis for a distributed, not centralised, information and learning object system.” (My emphasis) Derek Morrison, Auricle, February 27, 2004.

And the date of Stephen’s post (27 February 2004) indicates that the Auricle blog was available in early 2004.

With some further use of Google I discover that the Auricle podcast resources are still available on the University of Bath Web site – and I’m pleased that the MP3 file of my interview has not been lost. The RSS file also contains the publication dates, which show that the podcasts were published during 2005. We seem to have unearthed some further information about the Auricle blog.

Rediscovering The Blog!

It required a Google search for “Auricle Morrison” for me to discover that the Auricle blog is alive and well! It is now hosted at http://www.auricle.org/auriclewp/ (much better than the original URI!).  And as well as providing access to the original posts (although with a new look-and-feel, as the blog is now based on the WordPress blog software)  the blog is still active, with Derek using the blog to support his Pathfinder work at the HE Academy. As Chris Rusbridge pointed out on a post on “Digital Preservation” term considered harmful?” on the Digital Curation blog “phrases like “long term accessibility” or “usability over time” are better than the process-oriented phrase “digital preservation“. And here’s an example of how the Auricle blog has been preserved by continuing to still be used and accessible to its user community.

The Lessons

What are the implications of this case study for the wider community? And what lessons can be learnt?

We should be aware of the dangers of associating services with departmental names and specific technologies. This has been well documented, including Tim Berners-Lee’s article on “Cool URIs Don’t Change!” – although this is clearly easy to say, but more difficult to implement in practice.

I feel there is also a need for departments to audit their networked services and to document their policies regarding the sustainability of such services.  And such documented policies should be examined when departments change their names or there are significant changes in personnel.

And this case study provides an interesting example of a service which has been driven by an individual – Derek Morrison. As Derek clearly felt ownership to the Auricle blog, he was motivated to migrate the content of the blog to a new platform and, at a later date, to continue to contribute to the blog, although not as frequently as previously.  This probably saves the e-Learning Team at Bath from having to retrieve backup copies of the blog posts and provide an archived copy of the resource.  But who owns the blog?  And what would have happened if there had been an ownership dispute over the blog and the name of the blog?  These are questions which will be relevant to many academics and support who make use of blogs to support their professional activities – including myself and my UK Web Focus blog. But as the contents of my blog have a Creative Commons licence I would hope that there will minimise any legal barriers to the contents of my blog being migrated to a new environment should circumstances change.

Presentation. 1. Introduction to JISC-PoWR (Kevin Ashley, ULCC)
Presentation. 2. Records Management vs. Web Management (Marieke Guy, UKOLN)
Breakout Session: Web Preservation in your organisation
Presentation. 3. Web Preservation and Web 2.0 (Brian Kelly, UKOLN)
Presentation. 4. Legal issues (Jordan Hatcher, Opencontentlawyer)

LUNCH

Presentation. 5. The JISC-PoWR Workshops – Inputs and Outcomes (Marieke Guy, UKOLN)
Presentation. 6. The JISC-PoWR Handbook – Explaining Web Preservation (Kevin Ashley, ULCC)
Presentation. 7. The JISC-PoWR Handbook – Identifying Web Issues (Richard Davis)

COFFEE

Breakout Session: The next steps for Web Preservation in your organisation
Presentation. 8. The JISC-PoWR Handbook – Recommended Approaches (Ed Pinsent, ULCC)
Presentation. 9. Future possibilities
Final Thoughts

More information is available on the Workshop 3 page.

Places are still available. You can register using the Online Registration Form (note that this link takes you out of the JISC-PoWR blog to a Google Doc form).

1. You need to protect your institution
University Web sites contain evidence of institutional activity which is not recorded elsewhere and may be lost if the Web site is not archived or regular snapshots are taken. If you do not record certain information you are in danger of failing to comply with legal acts such as FOI and DPA, you may be breaking contractual and auditing obligations and put your institution at risk. This risk management approach has been taken to countless other digital resources (for example email – Curation of emails), it is only a matter of time before it is a standard approach to Web sites.

2. Starting a Web preservation programme will make you look like a ‘forward thinking’ university
You could be one of the first to start an official ‘Web preservation’ programme which will be great marketing fodder. (Remember the first UK Universities to offer blogs to students (Warwick), launch a YouTube channel and offer downloadable lectures using iTunes (University College London)? How about the first to get sued by a student for changing the course specification and having no record of the previous entry? Universities have already been sued over Web site accessibility, copyright of material on their site and allowing plagarism to take place.) Embedding Web preservation strategies will also help you think about the continuity of resources, dead links etc.

3. It could save you money
Web resources cost money to create and failing to repurpose and reuse them will waste money. Although Web preservation may have an initial cost, once the process has begun the savings can be great. Having a good strategy in place (which also should include selection and deletion where appropriate) will save both money and energy in the long run. Brian Kelly’s recent UK Web Focus Blog post on the environmental issues involved in digital preservation touches on this. As Owen Steven suggests in his comment it may make sense to link digital preservation to commercialism.

4. You have a responsibility to the people who use your resources
Students and staff may make serious choices based on Web site information and you have a responsibility to make sure a record is kept of this information.

5. You have a responsibility to the people who may need to use your resources in the future
Many of resources your institution publishes are unique and deleting them may mean that invaluable scholarly, cultural and scientific resources (heritage records) will be unavailable to future generations.

These reasons should give your senior management food for thought. These drivers and others will be expanded on in the JISC-PoWR handbook.

You can find out more on how to get started on a Web Preservation strategy by attending our upcoming workshop on Embedding Web Preservation Strategies Within Your Institution.

The idea of ‘heritage records’ is one that is often mentioned. Using Web sites as a ‘cultural snap shot’ has the potential to be a highly useful activity.

In his interesting and functional text Managing the Crowd: Rethinking Records Management for the Web 2.0 World Steve Bailey puts forward the point that deciding what will be important in the future is a tricky business. As he explains in the section on appraisal, retention and destruction: “The passage of time inevitably changes the filter through which we view our world and assess its priorities.”

Steve gives the example of the current plethora of Web sites that offer what we might call ‘quack’ remedies for medical problems. These sites may not seem to be of great interest right now but they may be invaluable to future historians who wish to demonstrate the distrust of the medical profession exhibited in 21st century western culture.

James Curral in his recent plenary talk at the recent Institutional Web Management Workshop used the example of blog posts made by soldiers out in Iraq and Afghanistan to demonstrate the irony of modern technology; these highly informative records could easily be lost while the diaries of World War II soldiers remain accessible.

Preservation mistakes have been made aplenty in the past. The destruction of much of the BBC’s flagship programmes in the 1970s has been well documented and in 2001 the BBC launched a a treasure hunt campaign to locate recordings of pre-1980 television or radio programmes. Ironically the Web site is no longer being updated, though it is still hosted on the BBc server.

So who can know what the future will bring? Which Web resources will we wish we had kept? Which student blog writer will go on to be a future prime minister or an infamous criminal? What bit of the terrabytes is the most important?

As Steve Bailey points out there is no crystal ball. It has always has been, and always will be, very difficult to predict what resources may prove to be valuable to future generations.

Although this offers little recompense for those making these choices, it does at least argue the case that we do need to preserve and we need to do so soon.

For information on how to reserve a place see the workshop 3 page.

Over the past few weeks I have become acutely aware that the term “digital preservation” may be becoming a problem.

Not quite what one might expect from Chris Rusbridge, director of the Digital Curation Centre (DCC)! And James Currall, who recently gave a plenary talk on Web site preservation issues at UKOLN’s IWMW 2008 event, appears to have been responsible for such heresy with his view that:

The digital preservation community has become very good at talking to itself and convincing ‘paid-up’ members of the value of preserving digital information, but the language used and the way that the discourse is constructed is unlikely to make much impact on either decision-makers or the creators of the digital information (academics, administrators, etc.).

But I have to say that I think that these views reflect the experiences we have had in the JISC PoWR project. Indeed Alison Wildish was quite open about this in her presentation at the first JISC PoWR workshop.

While we have to use “digital preservation” in appropriate contexts, including technical and other in-house discussions, and digital curation is appropriate in other contexts, terms that reflect the outcomes are more persuasive. The outcome of successful digital preservation is that digital resources remain accessible and usable over the long term.

and concludes by arguing that:

… outcome-related phrases like “long term accessibility” or “usability over time” are better than the process-oriented phrase “digital preservation”.

Amen to that! This reflects my views on the need to take a user-focussed approach to Web site development, with long term accessibility and usability simply mean that we need to think about the users in the future and not just those we have today.  And perhaps that’s the approach we have to take in order to ‘sell’ preservation to somewhat sceptical Web developers.

Should our slogan be “Web preservation is dead! Long live long term accessibility! Long live usability over time!” I wonder?

“The University would always have kept the print manifestation; what now to do with the web manifestation? One of the interesting changes they note over this time is the ‘rise of the logo’, and tracing changes in how the institution presents itself over time is also interesting.“

In a response to Lorcan’s post Tony Hirst referenced a blog post by Michael Nolan on the Edge Hill Web Services team blog in which Michael pointed out “one [example of interesting use of RSS] that caught my eye was the University of Warwick’s recent changes feed which allows you to subscribe to find out when the homepage changes. Better still, they have this for every page in their CMS.”

An example of this can be seen for the Research page on the University of Warck Web site. Although not nornmally visibile to most end users who visit this page, there is a link to an RSS feed of recent changes to the page. Using tools such as the Greasemonkey RSS Panel (available for Firefox) you can view the changes, as shown below.

In his comment on Lorcan’s blog Tony Hirst went on to suggest that ”A change feed, like on a wiki, could be one way (maybe) of facilitating 1st, 2nd or 3rd party web page archiving?“. I think Tony might be right. And maybe we are seeing the University of Warwick pioneering this approach, as the feed of recent changes seems to be provided by their in-house Sitebuilder 2 software, “the University’s web publishing tool“.

Perhaps when institutions are next procuring a CMS system they should be asking if vendors provide RSS feeds of changes to pages.  

This workshop provided an opportunity to review the outcomes of the first workshop, in which members of the JISC PoWR team and the 30+ participants identified some of the challenges to be faced in preserving content held on institutional Web services and explored some of the ways in which these challenges can be addressed. The slides for this review are available on Slideshare and are embedded below.

The main focus of the second workshop, however, was to look at the additional challenges which need to be addressed in a Web 2.0 context, when the content may be more dynamic, hosted by third party services and created by a wide range of users.

A PowerPoint presentation was used to initiate discussions based on a number of scenarios including use of blogs, wikis, Twitter, communications tools, social networks, ‘amplified events’ and use of third party repository services such as Slideshare – which is appropriate as this presentation is itself available on Slideshare and is embedded below.

This presentation doesn’t have any answers to these challenges – it was intended to initiate the debate at the workshop. Some of the approaches which may be relevant to the various scenarios have already been discussed on this blog including use of wikis, student blogs, use of Slideshare, instant messaging and Twitter and the wider set of discussions which took place at the workshop will feed into the final JISC PoWR handbook.

It is worth noting that this presentation was spotlighted on the Slideshare home page. This has helped to increase the visibility of the work of the JISC PoWR project: a week after the presentation hed been given there had been 713 views of the slides. It should also be noted that other Slideshare users had assigned various tags to the presentation (including data-portability, digital-preservation, sioc and preservation). As can be seen if you follow these links, we are beginning to see use of such social Web technologuies which can help users to discover related resources of interest to the digital preservation community.  This, to me, is a good example of the potential benefits which Web 2.0 can provide to those with n interest in the presevation of Web resources.

The page provides details of the Web usage statistics in the early years, with screen images shown of major changes to the home page from 1997 (unfortunately no screen images are available for the first three years of the service).

Information is provided on the people and groups responsible for the design, the changes which were made as new technologies became available, significant additional content that was added and details of awards which the site won.

This is an approach which I feel all institutions should consider taking.  And let’s start recording the history of those early years quickly, before the first generation of institutional Web managers start to retire, leave or forget the details of the institution’s Web history.

The Web 2.0 environment has a strong emphasis on communications between individuals and not just one-way publishing. This pattern of usage places additional challenges for institutions wishing to ensure that records are kept of the dialogue which takes place. And these challenges may well need to be addressed within the context of policies on the preservation of Web resources as increasingly digital communications technologies will have Web interfaces.

We will be publishing a series of posts looking at different aspects of Web 2.0. In this initial post we will provide a brief case study on use of instant messaging to support communications between two institutions. The case study will attempt to draw out some of the general policy issues which should be applicable more widely.

Use of IM for the QA Focus Project

This example describes the approaches taken to use of instant messaging to support communications between the project partners for the JISC-funded QA Focus project which was launched in January 2002. The project partners were UKOLN (based at the University of Bath) and, initially, ILRT, University of Bristol. However after the end of the first year of the project ILRT withdrew form the project and were replaced by AHDS, who were based in London.

In order to minimise the amount of travel and to help to provide closely integrated working across the project partners it was agreed to make use of instant messaging technologies. As well as enabling the team members to have speedy contact with each other it was also recognised that official project meetings could be held using the technology. It was appreciated that in this context there was a need to have a slightly formal protocol for managing the meetings, to compensate for the limitations of online meetings. And in addition to the best practices for managing the online meetings it was also agreed that a record of the transcript would be kept, and that this record would be copied across to the Intranet along with other formal documents.

After AHDS replaced by ILRT as project partners we decided to change our IM client from Yahoo Messenger to MSN Messenger. It was either during this change of IM tools or whilst making use of another IM client (I can’t recollect the exact details) that we noticed that different IM applications work in slightly different ways. This includes whether a transcript of dialogue is kept automatically and whether new participants to a group chat will see only new discussions or discussions which have taken place previously (which has the potential to cause embarrassments at the least).

The experiences we gained in use of IM led the project partners to develop a policy on use of IM (which covered issues such as the possible dangers of interruptions, as well as keeping records of formal meetings held on IM). The policy also clarified use of IM in an informal context, with their being no guarantee that records would be kept.

The policy stated that:

• IM software may be used for formal scheduled meetings. In such cases standard conventions for running meetings should be used. For example an agenda should be produced, actions clearly defined, changes of topics flagged and a record of the meeting kept.
• IM software may be used for direct communications between individual team members. For example it may be used for working on particular tasks, to clarify issues when working on collaborative tasks and to support team working. IM may be particularly suited for short term tasks for which no archive is needed and other team members need not be involved – for example, arranging a meeting place.
• Highly confidential information will not be sent using IM, due to the lack of strong encryption.

General Issues

The general issues arising from this case study include:

• The need to ensure that the users of the IM technologies and those involved in developing policies related to its use have a good understanding of how the technologies work together with an understanding of the differences between different IM systems.
• The need for simple documented policy statements

But what about the risks associated in making use of a third party service in this way? What will happen if, for example, the Slideshare’s business model is flawed and the company goes bankrupt? Rather than making use of a Web 2.0 service shouldn’t we be providing Slideshare’s functionality in-house?

I feel this is the wrong response: it would be similar to saying that we should not allow third party organisations to manage our savings – but we all have bank accounts. And, although we know from recent experiences in the UK that there can be risks when using banks, we don’t shut down our accounts when we became aware if incidents such as Northern Rock financial difficulties. Rather we assess the risks and then manage the risks (in the case of savings, this might be to limit one’s saving to a maximum of £35,000 with any single bank, as this amount is guaranteed by the Government).

In the case of Slideshare an in-house solution would not only be costly to replicate its functionality, but it would also be unlikely to provide the impact and popularity which Slideshare has.

The challenge then is to assess possible risks and to explore mechanisms for managing such risks. The approach I take is to look at the popularity of the service and its user community (an approach, incidentally, which has also been recommended when selecting open source software). The Techcrunch service can be useful if providing information on the financial background to many Web 2.0 companies and its information on Slideshare seems reassuring, with a post in May 2008 described how SlideShare had Secured $3M for Embeddable Presentations.

The risk management approach I have taken is to store a managed master copy of the slides on the UKOLN Web site and ensure that links to this resource are provided on Slideshare.  As can be seen from the image,  the URL is included  on the title slide and in the accompanying metadata. In addition the URL is also included in the footer of the hard copy printouts. I also provide a Creative Commons licence for the resource, which seeks to avoid any legal barriers to future curation of the resource and allow the resource to be downloaded from the Slideshare site.

This approach aims to ensure that the master resource is kept at a stable managed location, allows users to make a copy of the resource (if, for example, the Slideshare service suffers from performance or reliability problems) and allows uses to bookmark or cite the managed master version of the file.

In a Web 2.0 environment, however, other options become available. Rather than installing software locally, services which are available on the network can be used – and blogging services such as WordPress and Blogger are very popular blog hosting services.

What are the preservation aspects associated with the provision of a student blogging service? One might feel that the locally-installed application must be preferable, since management of the software and data is under the control of the institution. But what happens when students leave the institution? The normal policy in many institutions has been to delete student accounts and their data shortly after they leave. But is this desirable from the student’s perspective?  And what if they wish their data – their blog posts – to still be available after they leave the institution?

This is starting to happen, with the University of Warwick, which provided the first large-scale student blogging service a number of years ago. And as I wrote about a year ago, we are starting to see the first generation of student blog enthusiasts asking these questions. My post linked to a blog post hosted at the University of Warwick from a student (Jo Casey) who asked:

“In the middle of August I will be leaving Warwick (to be the new Corporate Communications Manager at the Open University). … But, given that I will have to migrate my blog, where is the best place to go?“.

Unfortunately Jo’s blog was been deleted after she left the University – I was fortunate to have captured her question on my blog.

In light of this particular example from an institution which pioneered use of students blogs, my question would be “Wouldn’t institutions be advised to recommend the use of mature hosted blogging services for members of the institution – such as students – who will normally only be at the institution for a short period?”

Would this be a desirable approach? What are the disadvantages? And could such problems be addressed?

Most electronic records management systems (ERMS) provide users with the ability to perform this declaration automatically. When they do so, the digital content they have created (e-mail, document or whatever) becomes ‘fixed’. UK Government have called this creating ‘locked down and secure’ records, a necessary step for ensuring their authenticity and reliability.

But ERM systems seem to work best with static documents; authors of reports, for example, understand that a good time to declare their report as a record is when the final approved version has been accepted. Yet one of the distinctive features of Web 2.0 content is that the information is very fluid, and often there is no obvious point at which to draw this line and fix content.

One example might be blog posts. These can receive comments from the moment they are posted and well into the future. Not only this but many bloggers go back and edit previous posts and delete comments. This matter was recently discussed on Brian Kelly’s UKWeb Focus blog. Phil Wilson asked:

“Brian, is there any reason you never modify or update your posts when you’ve made an error, and instead make users plough through the comments to see if anything you’ve said is wrong?” (UK Web Focus Blog)

Brian’s response was that he sometimes fixes typos and layout issues but is:

“reluctant to change the meaning of a published post, even (or perhaps especially) if I make mistakes. In part I don’t want to undermine the authority of any comments or the integrity of any threaded discussions.”

Brian is open about this in his blog policy stating that only in exceptional circumstances will postings and comments be deleted.

Concerns about censorship and bloggers deleting posts/comments were also recently made in responses to What is fair play in the blogo/commentosphere? on Nature’s Blog.

Assuming that blog posts are to be included within a records management programme or a preservation programme, the issues described above might cause problems for those attempting to preserve authentic and reliable Web resources.

One approach is to be explicit in your Web Resource Preservation strategy about when you freeze Web resources for preservation, and the implications of doing so.

Another approach might involve an agreed institutional policy such as Brian has, but with an additional form of wording that is explicit about the status of blog posts as records, including when and how they should be declared as records, and whose responsibility it is to do so. Should selected blog posts be declared as records by their owners into the ERMS? Or will they all be harvested by an automated capture programme, and if so, how frequently?

Any thoughts?

We’ve been using Wetpaint’s wiki feature as a collaborative space for our workshop feedback, and this suits us fine: once we have collated all the input for our project outputs, in a few weeks it’ll probably be no loss to us to delete the wiki, or just set it adrift among all the other jettisoned flotsam in cyberspace.

But what’s often given less serious consideration, in the excitement of using a third-party provider of wikis, blogs, Ning, etc., to get your collaborative hypertext project off the ground so quickly and easily – and without having to go cap or cheque in-hand to whoever guards your web space – is this key preservation issue: what happens when you want to get your painstakingly intricate web of hyperlinked pages out?

There are many good reasons why you might want to do this: you might want to migrate to another wiki system or CMS, as the shape and nature of your content evolves; or put it on a permanent, persistent footing by moving it into your own domain; you might simply want to back it up or take a snapshot; or you might want to pull out information for publication in a different form. When you had one or two pages, it might have seemed trivial; but what if you now have hundreds?

Unfortunately, just as exporting the information is often a secondary consideration for wiki content creators, so it also is for the wiki farm systems. The Wetpaint Wiki discussion boards indicate that an export feature was a long time in coming (and its absence quite a blocker to adoption by a number of serious would-be users). And what was eventually provided leaves a lot to be desired.

Wetpaint’s backup option “lets” you download your wiki content as a set of HTML files. Well, not really HTML files: text files with some embedded HTML-like markup. (Which version? Not declared.) Don’t expect to open these files locally in your browser and carry on surfing your wiki hypertext (even links between wiki pages need fixing). The export doesn’t include comment threads or old versions. Restoring it to your online wiki is not possible. But, for what it’s worth, you have at least salvaged some sort of raw content, that might be transformed into something like the wiki it came from, if hit with a bit of Perl script or similar.

I checked out Wikidot – another impressively-specced, free “wiki farm”. Wikidot’s backup option will deliver you a zip file containing each wiki page as a separate text file, containing your wiki markup as entered, as well as all uploaded file attachments. However, according to Wikidot support:

you can not restore from it automatically, it does not include all page revisions, only current (latest), it does not include forum discussion or page comments.

To reconstruct your wiki locally, you’ll, again, need some scripting, including using the Wikidot code libraries to reconvert its non-standard wiki-markup into standard HTML.

A third approach can be seen with a self-hosted copy of Mediawiki. Here you can select one or more pages by name, and have them exported as an XML file, which also contains revisions and assorted other metadata. Within the XML framework, the page text is stored as original wiki markup, raising the same conversion issues as with Wikidot. However, the XML file can be imported fairly easily into a different or blank instance of Mediawiki, recreating both hypertext and functionality more or less instantly.

In contrast to all these approaches, if you set a spidering engine like HTTrack or Wget to work “remotely harvesting” the site, you would get a working local copy of your wiki looking pretty much as it does on the web. This might be an attractive option if you simply want to preserve a record of what you created, a snapshot of how it looked on a certain date; or just in case a day should come when Wetpaint.com Inc., and the rest, no longer exist.

However, this will only result in something like a preservation copy – not a backup that can be easily restored to the wiki, and further edited – in the event, say, the wiki is hacked/cracked, or otherwise disfigured. For that kind of security, it may be enough to depend on regular backups of the underlying database, files and scripts: but you still ought to reassure yourself exactly what backup regime your host is operating, and whether they can restore them in a timely fashion. (Notwithstanding the versioning features of most wikis, using them to roll back a raft of abusive changes across a whole site is not usually a quick, easy or particularly enjoyable task.)

All this suggests some basic questions that one needs to ask when setting up a wiki for a project:

• How long do we need it for?
• Will it need preserving at intervals, or at a completion date?
• Is it more important to preserve its text content, or its complete look?
• Should we back it up? If so, what should we back up?
• Does the wiki provide backup features? If so, what does it back up (e.g. attachments, discussions, revisions)?
• Once “backed up”, how easily can it be restored?
• Will the links still work in our preservation or backup copy?
• If the backup includes raw wiki markup, do you have the capabilities to re-render this as HTML?

And questions like these are no less relevant when considering your uses of blogs and other social software: I hope we’ll be able to look at them more closely in another post.

On a recent post on the UK Web Focus blog I described how the Open University has set up an institutional Twitter account. And a number of responses to the posts described similar institutional Twitter accounts for Edge Hill University (illustrated), Birmingham City University, Coventry University and Aston University. We can also expect departments to follow the example of the School of Law at the University of Sheffield  which is using Twitter to syndicates its Law School News blog.

Many fans of Twitter may feel that issues of preservation shouldn’t intrude in what is normally used as a individual productivity and social tool. However if is often the case that new technologies which may have initially been provided for individual use and for social purposes, quickly seem to be used by early adopters in teaching and learning and research contexts. And soon afterwards institutions which are willing to explore the potential of such emerging technologies to support the needs of the institution will set up Twitter accounts, areas on YouTube, iTunes, etc. as, for example, the Open University has done.

Hence the need, I would argue, for institutions to ensure that they have considered the preservation and management implications of their tweets, even if the institutions feels that it would be inappropriate to have heavyweight policies on personal use of micro-blogging technologies. But perhaps before we establish the institutional policies we need to think about the different ways in which such micro-blogging applications may be used and also what the potential risks may be. 

Any thoughts? 

Kevin is correct – these distinctions are very real. And different sectors will may well have differing views as to the importance of preservation the underlying data or the user experience – this has surfaced at recent repository events, with some groups arguing that PDF provides a satisfactory means of preserving the user experience whilst others feel that it is more important to preserve the data which was used to create the PDFs.

But rather than revisiting such arguments in this blog I would like to reflect on a comment made by Chris Rusbridge in response to the same post mentioned above. Chris described how:

this grump came about partly because a number of organisations which are supposed to have a commitment to long-term access to information managed to destroy access through re-launches. Richard, I do like continuity, and also long-term accessibility (gets both angles!) rather than preservation…

Persistent URIs are not about technical solutions, they are about commitment. We must make sure we never break URIs!

We should note that Chris isn’t engaging with the argument of whether it’s the experience of the information which he wants to be preserved – rather it’s the means of access he wants to remain in place.

And this, I feel, is one of the most challenging aspects of Web site preservation – preserving the access mechanisms for the end user. This, then, is very different from preserving that valuable historical parchment which might be moved from public view,  send off to a company for renovation and then send on tour as part of a travelling exhibition. In this case the resource may be being curated, but access to end user is not available – or even expected.

In the case of Web resources a failure by an organisation to manage digital assets may result in the organisation losing valuable information. But what if the Web resources are simply migrated to an alternative location? Or the resources are embedded in other aspects of the organisation’s work? In such cases the organisation will argue that it hasn’t lost anything.  Rather it is the end user who may feel aggrieved – as Chris has clearly described.

So perhaps we have three key aspects to Web site preservation – preservation of the experience, the information and the access. Or, if you feel that access for end users is part of the experience, we might argue the need to preserve the experience and/or information to support the needs of the organisation and the needs of the user community.

So I was very pleased to read Alison’s Wildish’s blog post enitled “Web Preservation: should we make the time?“. In this post Alison (head of Web Services at the University of Bath) described the case study which she and Lizzie Richard (Archivist, Records Manager and FOI Coordinator at the University of Bath) presesented at the first JISC PoWR workshop. Alison described how:

“Neither of us felt web preservation was something we had expertise in nor the time (and for me the inclination) to fully explore this. Web preservation was something we could see as being useful (in the future) but I think we both felt it wasn’t a priority.”

The good news is that the discussions Alison and Lizzie had after I introduced them to each other and invited them to participate in the JISC PoWR projects have helped them to further their understanding of Web site preservation:

“Simply discussing preservation (from both sides of the fence) taught us a lot. We discovered the risks involved in simply side-lining it; the potential gap in University history and the benefits of embedding preservation into our digital strategy. ”

And now that Alison and Lizzie are better aware of the need to have a policy of Web site preservation they are  in a position to start working on one:

“So is it something we should make time for? Yes I believe it is.”

The JISC PoWR project is starting to deliver its goals of engaging the key stakeholders, making them better aware of the challenges in preserving Web sites but also willing to address those challenges

And I’m please to say that Alison has made the slides used at the workshop available on Slideshare – well worth viewing, especially if you are a records manager who is “a paper person [and] have enough trouble trying to preserve hard copy records without having to worry about the web … [who] can see the value in theory, but in practice it’s too huge [and] guess it might be a good idea, but no one much cares what I think I am interested though… ” or a Web person who has the view that “In all honesty it isn’t interesting to me… We struggle to keep the site current – never mind thinking Web Specialist about preserving the old stuff I am future watching… need to know what to bring in not how to keep hold of the past Why is it something I should think about now? I’m not really that interested“. 

Andy Powell, a former colleague of mine who used to work at the University of Bath, posted a Twitter message in response to my post in which he said:

@briankelly all pages prior to http://tinyurl.com/47pydq were mine – that’s what web design was like back then! – but all records now lost

03:56 PM June 18, 2008 from twhirl in reply to briankelly

But although formal records of the decisions made related to the home page (its design, the content, the links and the technologies used) may have been lost (or perhaps not even kept) I do wonder whether it may be possible to document such history based on anecdotal evidence from those who were either direectly involved with the decision-making process or perhaps who observed the results of the decisions.

From the museum’s sector and the experiences of The National Archive (with the public Wiki service) we know that the general public does seem willing to provide anecdotal information on resources such as old photographs.

This approach seems to reflect some of the discussions held at the first JISC PoWR workshop. As described in Ed Pinsent’s summary of the event “there was a lot of ‘folk memory’ and anecdotal evidence, also sometimes called ‘tacit knowledge’“.

Would it be possible, I wonder, to provide access to images of an institution’s old Web pages and, though use of social networking technologies, encourage members of the institution (and perhaps the wider community) to document their recollections of the Web site?

The scenario required an institution to provide ‘examples of how the web site has developed since it was launched’. (Brian Kelly has already broached this topic here). My colleagues discussed this conundrum with great vigour, but most had to admit they drew a blank when it came to producing even a basic screenshot from five years ago. Instead, there was a lot of ‘folk memory’ and anecdotal evidence, also sometimes called ‘tacit knowledge’. It was clear that everyone was aware that their web site had changed (and sometimes even improved) quite dramatically in the last 5-10 years. It’s just that no-one could lay their hands on any evidence of the changes. Imagine it like one of those ‘before and after’ reveals which you might get on Changing Rooms on TV. The problem with web sites is that we can’t always lay hands on the ‘before’ picture.

Some drivers for changes included:

• Corporate or institutional rebranding
• Move to a Content Management System
• Content provider change
• External consultancy

And the following sorts of web site elements were subject to change:

• Design, branding, colours, logos
• Accessibility
• Language
• Content – obviously(!) – but do we know what content was added and what was thrown away?
• Navigation – clicks and links became more consistent across the site
• More pages – a site which used to be one ‘main page’ for the entire institution is now much larger and more complex, and every Department in the University now has a suite of pages
• More interactive elements, including automated forms
• Site became more searchable
• More media, video and audio content was embedded
• Distinction between internal and external content became more pronounced

It’s also interesting that one of our colleagues thought that the CMS also added some constraints to working; what once was easy is now much more difficult. I’m sure this is a trade-off we find with most new ways of working and it isn’t meant to imply that a CMS is always evil.

Kevin Ashley heard this summary with some interest and jokingly suggested that perhaps some form of oral history was the relevant preservation solution here. Yet if we have no other evidence of a web site’s history, who knows – it may yet turn out to be the last resort.

There’s a real difficulty, though, in applying either of these preservation approaches in a environment of rapid technological development. And within higher education we are likely to see examples of such innovation, whether this is scientific researchers involved in new ways of visualisating scientific data or teaching staff who wish to ensure students gain experiences in use of Social Web technologies.

How are such tensions to be addressed? Should, for example, use of immersive environments such as Second Life be banned until preservation techniques have been developed which will ensure that such complex environments can be preserved? Such a draconian approach is alien to the educational sector’s IT development culture (although such approaches are taken in other areas such as biological and medical research). And as I’ve described in a post on “Is Second Life Accessible?” innovative technologies such as Second Life can bring substantuial benefits to the user community – in this case a user with cerebral palsy who feels that Second Life provides a really useful tool for people who are unable to get around, who have problems of mobility in real life “because you can have friends without having to go out and physically find them“.

The tensions between preservation and innovation perhaps reflect similar tensions between accessibility and innovation, with differing opinions being held by the various interested parties. In the case of Second Life (where we are seeing virtual worlds being continually assembled, developed and then redeveloped) there does seem to be an awareness of the need to preserve such virtual worlds, with the Maryland Institute for Technology in the Humanities having received funding from the Library of Congress’s National Digital Information Infrastructure and Preservation Program (NDIIPP) for a two-year project on Preserving Virtual Worlds. And yet the $590,000 funding for this project, which will not, of course, guarantee that a solution to the problem with be available at the end of the funding, indicates that the preservation of immersive worlds will not be an easy undertaking.

Returning to Kevin’s comment that there is a “distinction [to be] made between preserving an experience and preserving the information which the experience makes available. Both are valid preservation approaches and both achieve different ends. perhaps it is important to focus on these distinctions when we are seeking to preserve our innovative services. Might the video clip of the Second Life experience be the appropriate solution for the pioneers of this technology until the research programmes have devised ways of preserving the much richer and resuable environment? And might not this be an approach which can also be taken for our innovative Web services?

The main presentations are now available for download.

• Presentation 1: JISC-PoWR Workshop 1, Marieke Guy, UKOLN. Presentation: [Slideshare] – [pres1.ppt PowerPoint file]
• Presentation 2: Preservation of Web Resources Part I, Kevin Ashley, ULCC. Presentation: [Slideshare] - [pres2.ppt PowerPoint file] (audio – pres2.mp3)
• Presentation 3: Challenges for Web Resource Preservation, Marieke Guy, UKOLN Presentation: [Slideshare] – [pres3.ppt] (audio – pres3.mp3)
• Presentation 4: Bath University Case Study, Alison Wildish and Lizzie Richmond, University of Bath. Presentation: [Slideshare] – [pres4.ppt PowerPoint file] [audio - pres4.mp3].
• Presentation 5: Legal issues, Jordan Hatcher, opencontentlawyer. Presentation: [Slideshare] – [pres5.ppt PowerPoint file] (audio – pres5.mp3)
• Presentation 6: Preservation of Web Resources Part II, Ed Pinsent, ULCC. Presentation: [Slideshare] – [ pres6.ppt PowerPoint file] (audio – pres6.mp3)
• Preservation 7: ReStore: A sustainable web resources repository, Arshad Khan, National Centre for Research Methods. Presentation: [Slideshare] – [pres7.ppt Powerpoint file]. Audio: [pres7.mp3]

The presentations are also available from Slideshare. Audio files are available from the Internet Archive.

We are also using a Wetpaint Wiki to collate the feedback from the workshop breakout sessions. If you were there, please have a look and help us ensure that your suggestions are represented.

An ‘at the event’ report written on the workshop by Stephen Emmott has been published in the Ariadne Web Magazine.

Take, for example, the Web site for the National Open Centre, which is illustrated.

If you visit the Web site you will find a nicely designed and easy to use Web site for the National Open Centre (NOC) which is a:

“national policy institute, a think tank to understand and articulate strategies to make effective use of Open Source Software and Open Standards (OS&S) for the benefit of all. It will focus on nationally relevant issues leading to proactive strategies to ensure that the UK effectively exploits the opportunities that arise with OS&S. The NOC will be independent, strategic and proactive and seeks the participation of interested and informed people.”

A very worthy organisation, it would seem (and I should add that I was a member of the NOC’s Advisory Group and attended the first meeting). Sadly, despite having a launch event at the Houses of Parliament, the NOC was unsuccessful in its attempts to gain funding, despite having a launch event at the Houses of Parliament. To paraphrase the Monty Python sketch “the NOC is not resting. The NOC is no more! It is bereft of life. It is an ex-NOC!“.

But this isn’t what you’d think if you explored the Web site. The home page urges visitors to “Get Involved!” and describes how it has “established the first set of subject panels. The topics being researched/discussed are: Public procurement, Open Standards and Open Source/Open Standard for SMEs. The Get Involved page then encourages visitors to participate with the NOC in a number of ways, including joining the Advisory Board, Subject Panels or the NOC Community. The only subtle indications that the NOC is no longer operational are the dates on various pages (206 or 2007) and the broken link to the NOC’s wiki from the Get Involved page.

The failure to provide any indication that the NOC failed to receive funding may be embarrassing to the partners of the service, which are list on the home page. But as well as such possible embarrassment, what would happen if visitors arrive at the Events page and read details of the one-day event on Document Standards planned for 4 July, which is illustrated below.

There is no indication that this refers to an event which was planned for 4 July 2007.
And there are no details about registration, although a location for the event is given (NCC offices, London). What might happen if someone travels to London to attend the workshop (which covers interesting aspects related to open document formats, with apparent participation from companies such as Microsoft). If this happened, I’m sure the potential participants would be pretty upset to discover that the NOC folded last year.

This is, I would agree, unlikely to happen. But what if the information about the event had been held on one of the NOC’s partner organisations, such as Birmingham City Council?

This example is taken from the wider public sector. But within the higher and further education sector, with short term project funding provided for much development work, institutions may find themselves in a simple situation, with the intentions of a project team failing to be realised due to a failure to win funding, and perhaps a loss of project staff.

How should this possible scenario be addressed? This is something to be addressed in future posts, but for now your comments and suggestions would be welcomed.

Making effective decisions
At its simplest level, this means deciding what to keep and what not to keep. There may be many drivers for these decisions – institutional policy, legal requirements and research interests are just a few. The decisions need to relate not just to what is to be kept, but why and who for. That’s because those requirements may have a bearing on how you choose to go about the job, or whose responsibility it is to carry it out. Not everything needs to be kept, and even when it does, it may not be your institution’s responsibility to keep it.

Implementing those decisions
Carrying out your decisions – keeping things, throwing things away, or ensuring that other people keep things – can be the trickiest part of the process. You may know you want to preserve the prospectus for past years, but can you be sure that your CMS, or the Internet Archive, or some local use of web-harvesting tools is going to do this job effectively for you ? You may be being told that some part of your web infrastructure would be easier to preserve if you avoided the use of certain features, or used a different authoring system. Is that true, and if it is, what are the negative consequences of such decisions ?

The handbook which will be one of the project’s outputs will attempt to answer these quesions in a way that makes sense to everyone who might be involved in the process. We want to help to make it easier to take decisions about preservation and to know what tools, systems or working methods can be employed to help you implement them.

The workshops are the primary mechanism we’re using to test whether the handbook makes sense to the people it’s aimed at, and that they tackle the problems that people are actually facing.

Within the JISC-PoWR team there have been a number of interesting discussions that have highlighted how differently the different players see Web preservation. To quote Ed Pinsent:

“The fundamental thing here is bringing together two sets of information professionals from differing backgrounds who, in many cases, don’t tend to speak to each other. Many records managers and archivists are, quite simply, afraid of IT and are content to let it remain a mystery. Conversely, it is quite possible to work in an IT career path in any organisation (not just HE/FE) and never be troubled by retention or preservation issues of any sort. “

The cliched view might regard Web managers as concerning themselves primarily with the day to day running of an organisation’s Web site, with preservation as an afterthought, and records managers focussing mainly on the preservation of resources and failing to understand some of the technical challenges presented. And although this may be a superficial description of the complexitities of they ways in which institutions go about the management of the digital resources, perhaps like many cliches, there could be an element of truth in such views.

Yet it is likely that Web preservation can be seen as important to the various parties; indeed one of the aims of the JISC-PoWR project is to highlight the importance of Web preservation in order to help institutions give a greater priority and allocate additional resources to this area. To quote Kevin Ashley:

“…this certainly brings out the benefits of our two groups working together on this: by identifying what each of us finds difficult to understand about the other’s perspective on the world, we can work out how to help others bridge the same gulf of understanding.”

The interdisciplinary approach that is needed requires a commitment to communicating and exchanging skills. Do you have any ideas on how this can be achieved? The JISC-PoWR project is interested in hearing how institutions are seeking to engage the different players in working together on a Web resource preservation strategy. If you have any accounts you can share then please contact us or submit a case study.

In order to illustrate how an institution’s home page may change over a period of over 11 years the Internet Archive’s WayBack Machine was used to view the first occurrence of the University of Bath home page in every year from 1997 until 2007. (Note that in browsers which support Flash you can interact with the display and a more interactive access can be obtained if your install the PicLens plugin, although there are also links to the static images and an automated rolling display of the pages in the Internet Archive).

In addition to this display a 4 minute video with accompanying commentary has also been created, which discusses some of the changes to the home page over the 11 years. A screenshot of the video is given below:

Is this example of interest to other institutions? Would it be helpful if tools could be provided to assist the creation of a similar visualisation of the history of your institutional home page?

[Note image of video replaced by embedded YouTube video on 20 July 2009.]

The enthusiastic way in which web-site owners “re-brand” or “re-launch” their web-sites suggests that they are not particularly interested, long-term, in the details of the experience; continuous improvement means continuous discarding. One hopes that they are more interested in the information content, in some more abstract sense. Maybe we could measure this by tracking older pages across re-launches?

Perhaps a measure of commitment to the “look and feel” might be the lifetime since last reorganised?

Is this right? Don’t Web site owners care about preservation, preferring instead to continually add new features to their services?

I have to say that I disagree. Rather than continual changes to Web sites due to the Web site owners’ enthusiasms, I would argue that such changes usually occur in response to user needs and expectations, the growing importance of Web services (which mean that institutions have greater expectations of the services which will be provided) and an increasing understanding of the limitations of approaches taken to Web site development in the past.

One example of this has been the obligation (for legal and moral reasons) to enhance the accessibility of Web resources. Initially HTML authoring tools and Content Management Systems (CMSs) provided little support to enhance accessibility – indeed many CMSs generated low quality HTML which could not be processed by assistive technologies. But as the technologies improved there was a moral obligation to redevelop Web sites to enhance their accessibility. And with the DDA/SENDA legislation requiring organisations to take ‘reasonable measures’ to ensure that users with disabilities were not discriminated against, there was also a legal requirement to make use of technologies which could enhance the accessibility of the Web resources.

We are now seeking similar tensions arising from the popularity of Web 2.0 services. And although it might appear appealing to try to resist the call to engage with such technologies the reality seems to be that if institutions fail to deploy more engaging, usable and useful services the users will fail to use the services and go elsewhere. We need to remember that, for example, a AJAX interface isn’t just a new technology which Web developers wish to deploy because it’s new, but, if used correctly, a way of deploying services which are easier-to-use and more accessible.

So rather than Chris’s suggestion that doing nothing should be an indicator of a commitment to preservation, I would argue that there is a need for continual development to our Web services (and I suspect that the JISC Innovation Group would agree with me on the need for continued innovation). But this innovation needs to be addressed with an awareness of the need to address preservation concerns.

The challenges of Web site preservation in a Web 2.0 environment will be addressed at the second JISC PoWR workshop. These issues will also be addressed in a number of forthcoming posts on this blog.

I’m currently chair of JISC’s Repositories and Preservation Advisory Group, and ULCC’s representative on the DPC board. My proudest achievement is the creation (with my quondam colleague Martin Powell) of a founder member in the Useless Web Pages Hall of Fame: the ULCC web telephone dialler – often imitated but never, IMHO, bettered. Unfortunately, both the dialler itself and the Hall of Fame are no longer with us on the web, and those links both depend on the Internet Archive’s Wayback Machine. For that reason, and many others, I’m particularly interested in the success of PoWR.

After the project funding had ceased, additional funding from the EU was obtained to publish a new e-journal known as Cultivate Interactive which was launched in July 2000. However there was a need to define a policy and accompanying procedures for the preservation of the Exploit Interactive Web site and content.

As described in a case study document on Providing Access to an EU-funded Project Web Site after Completion of Funding the following policy decisions were taken:

• The Web site’s domain name will be kept for at least 3 years after the end of funding.
• We will seek to ensure the Web site continues for at least 10 years after
  the end of funding.
• We will seek to ensure that the Web site continues to function, although we
  cannot give an absolute commitment to this.
• We will not commit to fixing broken links to external resources.
• We will not commit to fixing non-compliant HTML resources.

The case study went on to measure the disk storage used by the Web site and to quantify the costs. The storage requirements of less than 500 Mg of disk space were not significant so it was agreed to continue to pay for the www.exploit-lib.org domain name until at least October 2008.

Periodic automated links checks were carried out on the Web site to ensure that the internal links on the Web site continued to work.

A similar process was established for the Cultivate Interactive e-journal, when its funding ceased in February 2003.

By 2007, however, it became clear that it would not be possible to continue hosting the Web sites for the two e-journals on an aging and unsupported Windows NT platform for much longer. Various options for migrating the software onto a new platform were considered including:

1. Migrating the content and functionality of the services to a more modern MS Windows server platform.
2. Migrating the content and functionality of the services to a Unix platform which provided emulation for an MS Windows Web server environment.
3. Migrating the content and functionality of the service to a Unix/Apache platform.
4. Mirroring the static content of the Web site to a flat file store structure on a Unix/Apache platform.

Option 4 was selected, in part due to the limited systems expertise in managing MS Windows server software, together with a desire to reduce the complexities of the servers managed by UKOLN’s systems team.

In April 2008 the Exploit Interactive service was mirrored to a new location, using the Wintrack Web site mirroring software.

The lessons we’ve learnt:

• It is possible to provide access to a Web site eight years after funding has ceased.
• The Web site may deteriorate over time, as the underlying technical infrastructure changes.
• There will be a need to define clear policies, procedures and responsibilities for managing the process.
• There are likely to be resource implications, but these may not necessarily be significant.
• Migration of the underlying technical infrastructure (e.g. a CMS or scripts) to a new environment may require significant resources.

And I should add that on 19 November 2007 we received an email from Network Solutions giving us almost a year’s notice that out domain would expire in October 2008. We took the opportunity to retain the domain for an additional 5 years, up to October 2013. At $20 per year this was the most economical rate – although as a colleague pointed out, the $50 does need to be accounted for from some budget.

The Web Continuity Project arises from a request by Jack Straw, as leader of the House of Commons in 2007, that government departments ensure continued access to online documents. Further research revealed that:

• Government departments are increasingly citing URLs in answer to Parliamentary Questions
• 60% of links in Hansard to UK government websites for the period 1997 to 2006 are now broken
• Departments vary considerably: for one, every link works; for another every link is broken. (TNA’s own website is not immune!)

Among the objectives if the Web Continuity Project are to ensure that:

• All links work in perpetuity
• No cited information is lost through deletion
• Information is preserved long-term, even if the Web is no longer the dominant publishing medium it is today

Its outputs will include:

• Guidance on creation and use of XML Sitemaps
• A website component (for MS IIS and Apache) that will redirect users to the Web archive if a link is no longer active but is in the archive
• Guidance to government webmasters on best practice for website design and maintenance for archiving purposes

We can see that the approach and aims of the Web Continuity Project are particularly pertinent to our deliberations. Its aims are to address both “persistence” and “preservation” in a way that is seamless and robust: in many ways, “continuity” seems a very apposite concept with which to address the particular nature of web resources (though I doubt we’d want to swap our nice acronym, PoWR, for CoWR!).

Many of the issues facing departmental web and information managers are likely to have analogues in HE and FE institutions, so the opportunity to share in research and expertise emanating from the National Archives and the British Library, among others, is an exciting one. We will be following the WCP closely over the coming months.

We think it extremely likely that institutional web resources are going to include digital materials such as “records created during the day-to-day business of an organisation” and “born-digital materials created for a specific purpose”.

What we want is to “maintain access to these digital materials beyond the limits of media failure or technological change”. This leads us to consider the longevity of certain file formats, the changes undergone by proprietary software, technological obsolescence, and the migration or emulation strategies we’ll use to overcome these problems.

By migration we mean “a means of overcoming technological obsolescence by transferring digital resources from one hardware/software generation to the next.” In contrast, emulation is “a means of overcoming technological obsolescence of hardware and software by developing techniques for imitating obsolete systems on future generations of computers.”

Note also that when we talk about preserving anything, “for as long as necessary” doesn’t always mean “forever”. For the purposes of the PoWR project, it may be worth us considering medium-term preservation for example, which allows “continued access to digital materials beyond changes in technology for a defined period of time, but not indefinitely.”

We also hope to consider the idea of life-cycle management. According to DPC, “The major implications for life-cycle management of digital resources is the need actively to manage the resource at each stage of its life-cycle and to recognise the inter-dependencies between each stage and commence preservation activities as early as practicable.”

From these definitions alone, it should be apparent that success in the preservation of web resources will potentially involve the participation and co-operation of a wide range of experts: information managers, asset managers, webmasters, IT specialists, system administrators, records managers, and archivists.

(All the quotations and definitions above are taken from the DPC’s online handbook.)

Has your institution has recently deployed a Web resource preservation strategy or embarked on Web resource preservation work? Would you be willing to share your experiences and discuss solutions to problem areas by submitting a brief case study? If so then please contact Marieke Guy.

Further details are available on the suggested format for case studies.

So whose responsibility is Web resource preservation then?

There are a number of parties who may have an interest in the preservation of Web resource. These range from the international institutions down to the individual.

Individual Approach

• The author of the resource – It may be important for individuals to retain a copy of papers, articles and other written works for future use, CVs etc.

Institutional Approach

• The publisher of the resource – Who exactly is the publisher of a resource? Is it the Web team or the institution itself?
• The institution – The Institution may be motivated by legal and records management reasons.
• The institution’s Library – Libraries have had a preservation role in the past and have experience in this area but the complex nature of Web resources means they cannot achieve this task alone.

National Approach

• The Academic Community – The united academic community may be able to achieve more than individual institutions.
• Archives – National archives may have a role to play here.
• Museums – Museums may be interested in the preservation of creative resources.
• The Government – The government may be interested in statistical information, cultural data and creating snapshots of a countries Web presence.
• Consortiums – For example the UK Web Archiving Consortium (UKWAC)

International Approach

• International Organisations – For example the Internet Archive.
• Commercial companies – Commercial companies may see the benefits of Web preservation from a legal and business perspective.

It seems that with so many having a vested interest in Web resource preservation the most successful approach could be cooperative working and shared responsibility. What Web preservation really needs is for us all to gain a true sense of ‘stewardship’ over Web resources.

So who has responsibility for Web resource preservation in your organisation?

If, however, you are using an alternative domain name (such as a .org, .org.uk, .co.uk or .com) then you will need to take care in managing the registration for your domain. If you have an annual re-registration for your domain, you will need to ensure that your internal administrative management procedures will ensure that the domain name is renewed prior to the expiry date – and be warned, that if you fail to renew the domain name, you may find that the domain is purchased by a third party. If you inadvertently fail to renew the domain name subscription in time and find that your domain has been purchased by a third party, you may find yourself in the position of having to pay what could be a significant fee to repossess the domain name.

You may ask why you would wish to make use of a non-.ac.uk domain in light of such possible dangers (JA.NET, the organisation responsible for managing .ac.uk domains, I am pleased to say, does not sell off its domains to the highest bidder). There may be legitimate reasons for this – the domain may be used for an EU-funded project, for example, or the domain may be used to support a cross-sectoral service for which it is not possible to obtain a .ac.uk domain. In light of this the advice to institutions may be to:

• Carry out an audit of use of non-.ac.uk domains.
• Ensure that such domains have adequate administrative processes in place to ensure that the domain name is not lost if, for example, project funding ceases and staff involved in the project leave.
• Carry out a risk assessment of the dangers of losing such domains and the costs your institution may be willing to pay to claim back the domain.

Note that a good example of an EU-project which probably has a valuable domain name is the DESIRE project, which used the www.desire.org domain name. The list of DESIRE partners includes the University of Bristol (the coordinating contractor), UKOLN (my organisation) together with several European organisations. It is pleasing that this Web site domain has remained in its current status since the project funding ceased in 2000. Does your institution have adequate safeguards?

For information on how to reserve a place see the workshop 1 page.

So although we will discuss, say, the benefits of use of open standards in a preservation context, we do acknowledge that simply saying that institutions must use open standards would be an over-simplistic approach. We recognise that the provision of Web-based services is a complex operation requiring many, and sometimes conflicting, goals.

We will seek to advise institutions on how they should address the preservation of Web sites and Web resources in a real-world context of conflicting demands, limited resources, institutional politics and existing services to run.

And we are very keen on hearing from institutions in the sector (and the wider community) on the approaches institutions are taken to managing medium and long-term access to their Web resources.

Starting, naturally, with digital preservation: this is defined as a “series of managed activities necessary to ensure continued access to digital materials for as long as necessary” (Digital Preservation Coalition, 2002). It’s best to consider the scope of digital preservation as much broader than digital archiving, though the terms are often used interchangeably. Because, in computing generally, “archiving” is the process of backup and offline storage of data, the term “digital preservation” helps avoid confusion when referring to the broader issues of managing digital materials and information in and about them.

The Digital Preservation Coalition (DPC) is a consortium of many leading institutions working in the field, including The British Library and The National Archives. Its online handbook contains much excellent information (though its online format could be improved), and includes a useful glossary.

A third term, digital curation, has recently gained prominence. This places greater emphasis on the activities required to maintain the integrity of digital collections over time, and keep them usable. It promotes a pro-active approach to managing digital resources and the use of technological solutions, like web services, to address the problems that technology itself has created. It also paves the way for the emergence of “digital curators”, continually monitoring collections and intevening when necessary – a role analogous to their non-digital counterparts. The best source of information about digital curation is the Digital Curation Centre, based at Edinburgh University.

In the next part we’ll look at some of the key concepts in digital preservation, including migration, emulation, and life-cycle models for digital objects. This will help us identify some of the things we should be considering when trying to preserve web resources.

A particular area of interest to me is what preservation means in a Web 2.0 environment in which organisations may be making use of third party Web sites. I will be writing posts on this topic, and would very much welcome feedback from institutions who have given thought to best practices in this area.

Any records manager or archivist will probably be able to give you half a dozen reasons for why digital preservation is very important. Some might well give you half a dozen more for why the preservation of Web resources in particular, which now play such a huge part in our daily lives, is very very important.

Unfortunately this critical activity isn’t easy. In fact the very nature of the Web means that the preservation and archiving of Web resources is actually a very complex task. A few of the major issues include:

• The transient and dynamic nature of the Web – The Web is growing at a rapid rate. The average Web resource’s lifespan is short and pages are often removed. On the Web publishing is an easy process and content may be changed often and not necessarily in an orderly way. Metadata is very much an afterthought. Web 2.0 content (comprising of data mash ups, blog entries, comments etc.) is even more dynamic.

• Selection issues – Of the billions of resources out there which and which instantiation of them should we preserve?

• The technologies involved – The Web is dependant on technology, it uses various file formats and follows many protocols, most of which evolve quickly. The look and feel of a Web page may be determined by a number of different elements such as the code, the http protocol, the user, the browser and the server. Which of these need to be preserved? Web resources are usually held on just one server, so are at greater risk of removal, yet for some resources countless copies are made. Again which do we preserve? Web sites are held together by hypertext links meaning parts of the site could be omitted (if for example they use a robots.txt file or pages are not actually linked to) if crawled by archiving software. Whole areas of the Web are held in problematical CMS or behind authentication systems and Web 2.0 applications use layered APIs, which use data in many different ways.

• Organisational issues – How is your institution using its Web site? Is it a publication or is it a record? Is the content being managed? Who is responsible and who has ownership?

• The legal issues – There are many IPR and data protection issues with Web content. Who owns the photos on Flickr, the comments on a blog or the details on a social networking site?

There is no easy answer! However despite the difficulties of Web preservation some institutions may be addressing some of these issues already. We are keen to hear examples of any approaches being taken.

“A formal records management approach may be the appropriate response to handling certain kinds of web material and this assumes active management of the material throughout its whole life-cycle, an approach which is intrinsic to all preservation-related approaches.”

We think it’s quite likely this line of thinking may apply to University and HFE websites, if the institutional website is being used as a place to conduct business; or is developing into an area where corporate records are being created and stored. A webmaster or a records manager could start to ask a few questions about the organisation’s website:

• Is it being used as a publication, or is it a record?
• Does it store copies of materials known to be held elsewhere, or does it store the only known copies? (That could apply to individual web pages, or materials attached to pages – PDF versions of a course prospectus, a financial report, or a project output).
• Are people – external and internal – making business decisions, based on the information they find on the website?
• Are actual transactions – financial or otherwise – taking place over the website? Does the University need to keep records of these transactions?

If your website is included within your formal records management programme, then the records manager should be pro-actively managing any record material that may exist on the web. Records managers have been doing this for digital records – sometimes called electronic records – for some time now, although often this can simply mean static office documents and e-mails stored on networks. If the institution has a really advanced RM programme, this might be done using an Electronic Records Management system, a corporate filing scheme, and carried out in accordance with corporate policies, mission statements, and guidelines. The Records Management programme will do such things as:

• Help the University / Institution operate efficiently – as a business and as an academic entity
• Protect the University’s vital records (for example, records which affect significant rights of an individual, a significant asset of the University, or the performance of an essential function of the University)
• Ensure compliance with information legislation – most notably FOI and DPA, but also IPR
• Keep material for as long as it’s needed to perform these functions
• Dispose of material when that business need has expired

For the purposes of this project, if your website does contain or produce records that have business value to the University, it may be appropriate to ensure that they are being managed in line with your RM policy. This leads us to all sorts of related RM areas – protection against loss or damage, ensuring authenticity, applying formal retention and disposal scheduling, and appraisal and selection. But these are all matters for future posts…

The team members are, from left to right, Ed Pinsent (ULCC), Jordan Hatcher (consultant), Marieke Guy (UKOLN), Kevin Ashley (ULCC), Richard Davis (ULCC) and Brian Kelly (UKOLN).

Marieke Guy (UKOLN) and Ed Pinsent and Richard Davis (ULCC) will be the main project workers, with Jordan Hatcher providing input into the IPR aspects associated with Web site preservation and myself and Kevin Ashley providing project management and strategic input.

The project team members will shortly be providing their introductions.

As well as providing a dissemination channel for the project activities we also intend that this blog will provide a forum for discussions on issues related to Web site preservation.