A rant, mostly, on the difficulties ahead

There is an interesting discussion going on at the Talis weblogs about the sometimes contentious relationships between library software vendors, technical librarians, standards in the library world, and the handful of library patrons who are navigating the maze of library online systems.

Without jumping into a minefield that I don't want to go to, I will make a few observations.

The version of the library that people see online is only a fraction of all of the computing and data processing that goes on. As a patron you don't see (except by glancing at the circulation desk) more than a pittance of the cataloging, book processing, shelving, interlibrary loan and other activities that go on. Most of it is opaque most of the time, and you only notice it when the system is down or when things mysteriously take a lot longer than you would expect them to.

It seems that libraries change their systems very slowly, and only then with a lot of hard work and frustration and pain. Without any data I'm guessing it's about a five year cycle between major system upgrades, and a six month project to haul out the old system with a forklift and move the new one in. This means that for the most part you are looking at something that is stable but not particularly innovative. Vendors really don't have that much interest in hearing what patrons care about, except via the purchasing process through library directors.

Standardization in the library world is a very slow, cumbersome process, and library standards bodies typically have vendors and very large libraries leading the way. Library patrons are an abstraction at many levels but don't appear to be a focus of the efforts. There is a lot of historical baggage (LCSH subject headings, ASN.1 and BER encoding in Z39.50, and a visible lack of cooperation with internet and web standards bodies) that manifests itself in protocols that are hard to understand and hard to implement. Simple things are hard, and hard things are impossible.

As a result the innovation from around the edges is happening in totally ad hoc ways without much care for standards but instead relying almost totally on parsing freeform output from catalogs. Talis's Paul Miller surmises:

Ed Vielmetti and others, as they do their interesting things with library data, are surely making calls to the ILS/LMS (or some 'safe' proxy for it, at least), rather than either scraping web pages on the OPAC, or calling some duplicate of underlying ILS/LMS functionality that a developer has chosen to repeat in the OPAC's code line...? And if they're not (for whatever reason), does it not make more sense that they should?

Well, no - for right now I'm scraping the catalog, and the next go-around I'm going to be working against John Blyberg's middleware layer where he is scraping a vendor's weird XML and trying to make sense of a messy tangle.

I wish it were otherwise, but for now, you deal with what you have.

Technorati Tags: standards, opac, library2.0

Comments

[Chris Deweese]

They reply as though vendors have something to hide.

Google sure has nothing to hide sharing their API and Data (for free). Same with Flickr and Yahoo! and the list goes on and on and on.

Why all the fear?

As a coder, it is EASIER for me to make my data accessible to myself (and indirectly others) because then I am able to code against in multiple ways without having some sort of proxy or middle layer.

With an open API I can code against it in Javascript, .NET, PHP, or whatever language I want to use as long as it can make webcalls and parse XML.

Why would anyone not want that? Abstraction just makes things harder to get at for yourself. Instead of loosely joined pieces you end up with this mammoth beast of code where one change requires a change in 50 other places.

What coder would ever want to make things harder for theirself?

[Dave Pattern]

Hi Ed

Taking your library system out to tender is a massive undertaking.

From memory, it took Huddersfield nearly a year to go through the entire tender process -- from drawing up the initial draft technical specifications to putting forward the final recommendations to the top-level library management.

For the best part of 6 months, most of the senior members of the library staff were tied up in meetings bashing out the technical specification, evaluating the responses, short listing the suppliers, organising vendor demonstrations, visiting reference sites, etc.

In the end, we decided to stay with Dynix (this was prior to the merger) but had we changed supplier, then there would be about another 9 months of migration to look forward to (and the inevitable loss of data during the conversion process). During that time, we'd need to run two separate systems in parallel and completely retrain our entire staff. It would also be taking as read that we'd have several weeks (possibly months) of teething problems once the new system went live.

Needless to say, most libraries only choose to replace their systems if their current supplier is really letting them down badly and if there's clearly a much better system available… otherwise it's simply not worth the pain!

I'm not sure we've really pushed our suppliers for open APIs in the past? I think it's only in recent years that libraries have spotted the huge potential in being able to get at their own data in new and interesting ways.

SirsiDynix are (touch wood) releasing Horizon 8.0 this year and that promises to have a full & open API layer and a modular design. You can imagine that I'm already drooling and gibbering like an idiot :-)

Until all the suppliers do this, it's down to us band of happy hackers to work with the tools we've been given and to make the best use of them.

The other point worth making is that vendors work on a development cycle of several years. So, once in a blue moon they'll look at undertaking a serious re-write of their code (Horizon 8.0 is supposed to be a total re-write of the system). Once they've done the re-write (which itself may take a year or two), they tend to just release patches and small feature upgrades to the system. Then, a few years down the road when their system is looking tired and out-of-date, they'll do another re-write.

Heck, I can name at least one major ILS vendor that requires you to purchase a COBOL compiler in order to run their system :-D

I doubt there are many libraries out there that would have the resources available to develop and maintain their own library system (except perhaps very small libraries?), so we're pretty much dependant on the suppliers.

regards
Dave
Library Systems Manager @ Huddersfield

[yoyology]

Does anyone know how good Koha is in this regard?

[Mark Lindner]

This post was a Ringleader's Selection for the Carnival of the Infosciences #25.

http://tinyurl.com/h2paa