Using Altmetrics to measure societal impact

The value of using alternative metrics to collect evidence of impact for a scholarly work appeals to me, because it opens up the notion that the “general public” are thinking and reading also, not just academics.   I am involved with lots of communities that I guess you’d call grassroots: activist and music communities particularly, which are not connected to universities, but are often political in nature (feminist, post colonial and queer theory contributing significantly).  The open and cheap dissemination of information and ideas is important to these networks and while zines have long played a big part in this, often introducing the theories of seminal thinkers (see for example, this zine, Judy!, a tongue in cheek zine about Judith Butler, recently digitised by QZAP – the Queer Zine Archive Project), the internet has obviously by and large taken over (though zines still live on!) and these same communities continue to share theory on Tumblr, Twitter, Facebook etc.  Gathering evidence that scholarly works are being discussed outside of the ivory tower I think not only is gratifying for the scholar, but also provides an important channel of feedback as academics are able to see the context in which their work is being used.

Working in an academic library I hear a lot about the Research Evaluation Framework (REF), which is basically how funding decisions are made for research in Universities.   As said on the Ref home page: “The assessment provides accountability for public investment in research and produces evidence of the benefits of this investment”. Which I guess is a fancier and more money-focused way of saying what I said above.   Clearly, altmetrics will play a significant part in proving the worth of areas of research, particularly with the shift to Open Access that is also being hustled along by the REF.

As we discussed in our DITA lecture, altmetrics cannot be relied upon for the whole picture.  Tools such as Altmetric rely on documents having DOIs, and also due to the ever-shifting nature of social media, results are not stable, they will only ever provide a snapshot for a moment in time.  Five minutes later, things could be different.  Not only that, but the way that we share things on social media can often be flippant and superficial; i.e. just because I share a link doesn’t mean I’ve really read it.  However, as Ernesto Priego points out on the Altmetric blog, using altmetrics often (but not always) means you can pinpoint data such as the geolocation of the person sharing the link, which can give added weight to the significance of the share.

AltmetricsUsing Altmetric Explorer last week was an interesting experience.  I was a bit frustrated that the keyword search didn’t seem particularly accurate, for example, I wanted to search for mentions of “Aotearoa”, which is the Maori word for “New Zealand”, as I thought it would cut out the chances of picking up articles about “Zealand” in Denmark. However, despite the uniqueness of the name, some of the articles returned did not contain the word, or even have anything to do with NZ at all.  I couldn’t get to the bottom of this. Also I noticed that mostly Science-based journals were being discovered, but I guess this is probably due to the these journals having a higher proportion of DOIs over journals in the humanities and literature, which is probably where I was more likely to find the topics I was interested in.  One thing I wondered about what whether there was any kind of correlation between how “populist” the article topic was, and what kind of social media was used to share it, e.g. perhaps including Pintrest and Tumblr in my search scope would reveal something rather different than if I stuck to news sites and blogs…however, it was difficult for me to judge that from the results I received (partly because the notion of what’s “populist” is subjective I think).

Looking at the Altmetric “doughnuts” was far more pleasing and easy to take in at a glance than using Excel spreadsheets, and I will definitely be going back to this tool and hopefully will be able to get more out of it with practice.