I can’t remember if I was using my “posh telephone voice”, but Richard Wallis has just posted a podcast that was recorded yesterday afternoon with Patrick Murray-John.
It’s definitely worth fast-forwarding past my inane waffley bits to listen to Patrick’s comments, as he makes some great points. Using usage data for marketing purposes wasn’t something that had occurred to me, but it’s a fantastic idea!
Even though it was an informal chat, I kept feeling twinges of “job interview syndrome” — that horrible sensation you get when you’re busy talking and you realise you’ve forgotten what the actual question was :-S
For my sins, I’m going to be doing something about OPACs and usage data at the upcoming JISC Developer Happiness Days event along with Ken Chad.
p.s. Can I propose a drinking game for this podcast? The rules are you have to have a drink every time someone mentions Tony Hirst‘s name ;-D
Tag: usagedata
Presentation to the TILE Project meeting in London
About 90 minutes ago, I had the pleasure of doing a short presentation to the JISC TILE Project’s “Sitting on a gold mine” workshop in London. Unfortunately I wasn’t able to present in person, so we had a go doing it all via a video conferencing link. As far as I can tell, it seemed to go okay!
The presentation was an opportunity to formally announce the release of the usage data.
Our Repository Manager was keen to try putting something non-standard into the repository and twisted my arm into recording the audio… and I’d forgotten how much I hate hearing my own voice!!!
Anyway, as soon as SlideShare starts playing ball, I’ll have a go uploading and sync’ing the audio track. Otherwise, here’s a copy of the PowerPoint: “Can You Dig It?: A Systems Perspective” and you can hear the audio by clicking on the Flash player below…
[audio:https://library.hud.ac.uk/ppt/CanYouDigIt.mp3]
The workshop had a copy of the PowerPoint that they were running locally, so every now and then you’ll hear me say “next slide”.
I haven’t listened to much of the audio, so I’ve got my fingers crossed I didn’t say anything too stupid!!!
[edit]
Well, here’s my first attempt at SlideCasting…
Can You Dig It
View SlideShare presentation or Upload your own.
…I had no idea how much I go “erm” when presenting! :-S
Free book usage data from the University of Huddersfield
I’m very proud to announce that Library Services at the University of Huddersfield has just done something that would have perhaps been unthinkable a few years ago: we’ve just released a major portion of our book circulation and recommendation data under an Open Data Commons/CC0 licence. In total, there’s data for over 80,000 titles derived from a pool of just under 3 million circulation transactions spanning a 13 year period.
https://library.hud.ac.uk/usagedata/
I would like to lay down a challenge to every other library in the world to consider doing the same.
This isn’t about breaching borrower/patron privacy — the data we’ve released is thoroughly aggregated and anonymised. This is about sharing potentially useful data to a much wider community and attaching as few strings as possible.
I’m guessing some of you are thinking: “what use is the data to me?”. Well, possibly of very little use — it’s just a droplet in the ocean of library transactions and it’s only data from one medium-sized new University, somewhere in the north of England. However, if just a small number of other libraries were to release their data as well, we’d be able to begin seeing the wider trends in borrowing.
The data we’ve released essentially comes in two big chunks:
1) Circulation Data
This breaks down the loans by year, by academic school, and by individual academic courses. This data will primarily be of interest to other academic libraries. UK academic libraries may be able to directly compare borrowing by matching up their courses against ours (using the UCAS course codes).
2) Recommendation Data
This is the data which drives the “people who borrowed this, also borrowed…” suggestions in our OPAC. This data had previously been exposed as a web service with a non-commercial licence, but is now freely available for you to download. We’ve also included data about the number of times the suggested title was borrowed before, at the same time, or afterwards.
Smaller data files provide further details about our courses, the relevant UCAS course codes, and expended ISBN lookup indexes (many thanks to Tim Spalding for allowing the use of thingISBN data to enable this!).
All of the data is in XML format and, in the coming weeks, I’m intending to create a number of web services and APIs which can be used to fetch subsets of the data.
The clock has been ticking to get all of this done in time for the “Sitting on a gold mine: improving provision and services for learners by aggregating and using learner behaviour data” event, organised by the JISC TILE Project. Therefore, the XML format is fairly simplistic. If you have any comments about the structuring of the data, please let me know.
I mentioned that the data is a subset of our entire circulation data — the criteria for inclusion was that the relevant MARC record must contain an ISBN and borrowing must have been significant. So, you won’t find any titles without ISBNs in the data, nor any books which have only been borrowed a couple of times.
So, this data is just a droplet — a single pixel in a much larger picture.
Now it’s up to you to think about whether or not you can augment this with data from your own library. If you can’t, I want to know what the barriers to sharing are. Then I want to know how we can break down those barriers.
I want you to imagine a world where a first year undergraduate psychology student can run a search on your OPAC and have the results ranked by the most popular titles as borrowed by their peers on similar courses around the globe.
I want you to imagine a book recommendation service that makes Amazon’s look amateurish.
I want you to imagine a collection development tool that can tap into the latest borrowing trends at a regional, national and international level.
Sounds good? Let’s start talking about how we can achieve it.
FAQ (OK, I’m trying to anticipate some of your questions!)
Q. Why are you doing this?
A. We’ve been actively mining circulation data for the benefit of our students since 2005. The “people who borrowed this, also borrowed…” feature in our OPAC has been one of the most successful and popular additions (second only to adding a spellchecker). The JISC TILE Project has been debating the benefits of larger scale aggregations of usage data and we believe that would greatly increase the end benefit to our users. We hope that the release of the data will stimulate a wider debate about the advantages and disadvantages of aggregating usage data.
Q. Why Open Data Commons / CC0?
A. We believe this is currently the most suitable licence to release the data under. Restrictions limit (re)use and we’re keen to see this data used in imaginative ways. In an ideal world, there would be services to harvest the data, crunch it, and then expose it back to the community, but we’re not there yet.
Q. What about borrower privacy?
A. There’s a balance to be struck between safeguarding privacy and allowing usage data to improve our services. It is possible to have both. Data mining is typically about looking for trends — it’s about identifying sizeable groups of users who exhibit similar behaviour, rather than looking for unique combinations of borrowing that might relate to just one individual. Setting a suitable threshold on the minimum group size ensures anonymity.
Coming soon, to a blog near here…
Okay — I’m the first to admit I don’t blog enough… I still haven’t even blogged about how great Mashed Library 2008 was (luckily other attendees have already blogged about it!)
Anyway, unless I get run over by a bus, later on this week I’m going to post something fairly big — well, it’s about 90MB which perhaps isn’t that “big” these days — that I’m hoping will get a lot of people in the library world talking. What I’ll be posting will just be a little droplet, but I’m hoping one day it’ll be part of a small stream …or perhaps even a little river.
(view slideshow of Mashed Library 2008)
Dewey friend wheel
I’ve been meaning to have a stab at creating something similar to a friend wheel, but using library data, for a while now. Here’s a prototype which uses our “people who borrowed this, also borrowed…” data to try find strong borrowing relationships…
I picked three random Dewey numbers and hacked together a quick PerlMagick script to draw the wheel:
- 169 – Logic -> Analogy (orange)
- 822 – English & Old English literatures -> Drama (purple)
- 941 – General history of Europe -> British Isles (light blue)
The thickness and brightness of the line indicates the strength of the relationship between the two classifications. For example, for people who borrowed items from 941, we also see heavy borrowing in the 260’s (Christian social theology), 270’s (Christian church history), and the 320’s (Political science).
The next step will be to churn through all of the thousand Dewey numbers and draw a relationship wheel for our entire book stock. I’ve left my work PC on to crunch through the raw data overnight, so hopefully I’ll be able to post the image tomorrow.
Playing with Processing
Iman first mentioned Processing ages ago, but it’s only recently I’ve gotten around to having a play with it.
So, this is my first stab at coming up with something visual and it’s in the same vein as Dewey Blobs…
…you’ll need Java installed to view it.
Rather than lay Dewey out on a 2D gird, I’m using a 10x10x10 cube (000 is at the front-top-left and 999 is at the back-bottom-right of the cube). The code then cycles through all of the check-outs (orange) and check-ins (blue) from a single day, with a zigzagging 3D line linking up the previous transactions.
What I originally wanted to achieve was to have two curving lines, snaking their way through the cube, but figuring out how to do the Bezier curves made my brain hurt 😉 Anyway, if you want to see a version where the line runs more quickly, click here — it’s harder to read the book titles, but the lines fade away more realistically. Or, here’s a 3rd version that doesn’t include the Dewey classification or book title.
A word of warning: the Java might chomp away at your CPU, so I’m not sure how well it’ll run on a slower PC.
Dewey Blobs
I’ve been fascinated by data visualisation for a year or two now, and I’ve recently been chatting to my good friend Iman about doing something with our circulation data. In particular, something that will be visually interesting to look at, whilst also giving you a feel for the data.
I’ve tried a few different things, but the Dewey Blobs are currently my favourite…
(items borrowed on 23rd June)
The transactions are placed on a 32×32 grid based on their Dewey classification (000-999). Each transaction is shown as a semi-transparent circle with two attributes:
1) colour — based on the School the student making the transaction studies in
2) size — based on the popularity of the book (the larger the circle, the more times it’s been borrowed before)
Where many students from the same school borrow from the same Dewey classification on the same day, the colour is reinforced. If the borrowing is from multiple schools, then the colours begin to blend to create new hues.
For example, on this day the vast majority of transactions in the 300s were by Human & Health students (green)…
…but a couple of days later, the borrowing in the 300s is more complex, with students from several schools appearing (Business students are red and Music & Humanities students are blue)…
You can browse through a few of the blobs on Flickr.
Show Us a Better Way
Thanks to Iman Moradi for highlighting this site:
Show Us a Better Way
Tell us what you’d build with public information and we could help fund your idea!
Ever been frustrated that you can’t find out something that ought to be easy to find? Ever been baffled by league tables or ‘performance indicators’? Do you think that better use of public information could improve health, education, justice or society at large?
The UK Government wants to hear your ideas for new products that could improve the way public information is communicated.
To show they are serious, the Government is making available gigabytes of new or previously invisible public information especially for people to use in this competition.
Go on, Show Us A Better Way.
The UK Government has come under a lot of criticism in the last few years for not making publicly funded data available, so does this mark a sea change in attitude?
My second thought when I read the web page was that you could do the same with your library… although I’m not suggesting you offer a top prize of £20,000!
Show Us a Better Way
Tell us what you’d do to improve the library and we could make it a reality!
Ever been frustrated that you can’t find out something that ought to be easy to find? Ever been baffled by library resources or the library services on offer?
We want to hear your ideas for new ways that we can improve how our services.
Go on, Show Us A Better Way.
Alternatively, as we begin to make our library data available for re-use, this would be a great way of promoting unintended uses.
Google Graphs
We’ve had loan data on the OPAC for a couple of years now, although it’s only previously been visible to staff IP addresses. Anyway, a couple of months ago, I revamped it using Google Graphs and I’ve finally gotten around to adding a stats link that anyone can peruse — you should be able to find it in the “useful links” section at the foot of the full bib page on our OPAC.
As an example, here are the stats for the 2006 edition of Giddens’ “Sociology“…
Stuck on a train
The joys of free wifi slightly make up for the fact that I’m stuck on a train that’s going nowhere fast — the East Coast main line is suffering long delays due to the train which was ahead of mine managing to damage and bring down the overhead power lines :-S
Today was the first meeting for the JISC Towards Implementation of Library 2.0 and the e-Framework (TILE), held at the impressive LSE Library in London.
I’m still not 100% sure what the TILE Project will manage to achieve, but it was great to spend 4 hours in a room full of people who are keen to liberate and share library data, primarily for the benefit of our users.
