Beyond survey design: take survey data to the next level

by Carolyn Doi
Education and Music Library, University of Saskatchewan

You’ve designed a survey, found the right participants, and waited patiently while responses come streaming in. The initial look at responses can be thrilling, but what happens next? I’ve used questionnaires as a data collection technique, and made the mistake of thinking the work is over once the survey closes. Kelley, Clark, Brown and Sitzia warn us about treating survey research as a method requiring little planning or time:

“Above all, survey research should not be seen as an easy, ‘quick and dirty’ option; such work may adequately fulfil local needs… but will not stand up to academic scrutiny and will not be regarded as having much value as a contribution to knowledge.”1

Let’s consider some steps to explore once data collection has been completed.

1) Data cleaning and analysis
Raw survey data is usually anything but readable. It takes some work to transform results into meaningful and shareable research findings. First of all, familiarize yourself with some of the relevant terminology, before moving on to actually working with the data. Before touching the dataset, you’re going to need to create four worksheets, one for raw data, one for cleaning in progress, one for cleaned data, and one for data analysis. Each worksheet shows a stage in the process, which will allow you to backtrack, or find errors. If you haven’t taken a stats class recently, I like this introductory Evaluation Toolkit, which clearly describes the processes of cleaning, tabulation, and analysis for both quantitative and qualitative data.

2) Visualization and reporting
Consider data visualization to bring your survey data to life, but remember to choose a visualization tool that makes sense for the data you’re trying to represent. The data visualization catalogue is a handy tool to learn more about the purpose, function, anatomy, and limitations of a wide range of visualizations. It includes links to software and examples of each visualization. There are lots of free or inexpensive programs to help create visualization including Microsoft excel, Google sheets, or Tableau Public. If you’re looking for some inspiration, take a browse through the stunning work of Information is Beautiful for ideas.

Likely you will want to share the outcomes of your research, either at your institution or in a paper or presentation. Kelley, Clark, Brown, and Sitzia provide a great checklist of information to include when reporting on any survey results, including research purpose, context, how the research was done, methods, results, interpretation, and recommendations.2 Clarity and transparency in the research process will help your audience to better understand and evaluate the research and its applicability to their context.

3) Data preservation and access
Consider an open data repository such as the Dataverse Project to make your data discoverable and accessible. Sharing your data comes with benefits such as “web visibility, academic credit, and increased citation counts.” You may also be required to archive your data to satisfy a data management plan or grant funding requirements, such as those from the Tri-Council. When archiving in a repository, remember to share your data in an accessible file format, and include accompanying files such as a codebook, project description, survey instrument, and outputs such as the associated report or paper. As a rule of thumb, aim to provide enough documentation that another researcher would be able to replicate your study. A dataset is a publication that you can cite in your CV, ORCID profile, in a paper, or presentation. Doing so is a great way encourage others to learn about your research or to build on your research project.

Getting your hands dirty and working directly with survey data is where you’ll be able to explore and eventually tell a compelling story based on your research. Be curious, persistent, and enjoy the process of research discovery!

1KATE KELLEY, BELINDA CLARK, VIVIENNE BROWN, JOHN SITZIA; Good practice in the conduct and reporting of survey research, International Journal for Quality in Health Care, Volume 15, Issue 3, 1 May 2003, Pages 261–266, https://doi.org/10.1093/intqhc/mzg031

2Ibid. p. 265.

This article gives the views of the author(s) and not necessarily the views of the Centre for Evidence Based Library and Information Practice or the University Library, University of Saskatchewan.

Walking the (Research Data Management) Talk

by Marjorie Mitchell
Librarian, Learning and Research Services
UBC Okanagan Library

Librarians helping researchers to create data management plans, developing usable file management systems (including file naming conventions), preparing the data for submission into repositories and working through the mysteries of subject-specific metadata schemes are at the forefront of the data sharing movement. All this work leads to research that is more reproducible, more rigorous, has fewer errors, and more frequently cited (Wicherts, 2011) than research that isn’t shared. In addition to those benefits, shared data leads to increased opportunities for collaboration and, potentially, economic benefits (Johnson, 2016). However, are we doing what we are asking our researchers to do and ultimately making our research data available and open for reanalysis and reuse? Are we walking the talk? Or is this the case of the carpenter’s house (unfinished) and the mechanic’s car (needing repair)?

When I’m speaking of data I use Eisner and Vasgird’s description of data as “a collection of facts, measurements or observations used to make inferences about the world we live in” (n.d.) because the research done by librarians consists of wide varieties of data: numerical, textual, photographic images, hand drawn maps, or diagrams created by study participants. Almost all have the potential to be shared openly and to act as a springboard for further research, subject to appropriate ethical considerations.

I started searching to see what data I could find from Canadian librarian researchers in repositories. I have not finished my search, but my early results show some interesting things. To date, this has not been a rigorous study, but more of a curious, pre-research “let’s see what’s out there” browse, and therefore must not be misconstrued as the basis for conclusions. I briefly looked internationally for a few studies and found a wider variety of topics with available datasets than I had found in Canadian repositories, which was what I expected to find.

Two things jumped out at me right away. First, when data is available, it is either from large, national or multi-institutional studies, or it is from studies that have been repeated over time, such as LibQUAL+®. Far fewer institution-specific or single researcher/research team datasets are “available.” Some of those have “request access” restrictions, meaning it may be possible to access the data with permission from the creator, but that is not guaranteed. The second thing I noticed was how difficult it is locate these datasets. Although there is a movement to assign unique and persistent identifiers to datasets, this has not, as yet, translated into a search engine that can comprehensively search for datasets.

I am happy to see a steady increase in the amount of librarian-generated research data being made available. Librarian-generated research is not alone in this trend. It is happening across the disciplines. While little library research is externally funded, it is worth noting some funders are requiring data management plans with the goal of data sharing. Some scholarly journals, particularly in the sciences, have strong policies about data sharing. Each change, minor or major, moves us more toward data that is shared as a matter of course, rather than data shared only reluctantly.

If this all sounds like “just another thing to do” or maybe “I don’t have the skills or interest to do this,” consider research data sharing as an opportunity to partner with another librarian who has those skills but perhaps lacks the research skills you have. Research partners and teams can allow people to contribute their best skills rather than struggling to compensate for their weaknesses throughout the process.

Finally, have a look at the data that is out there just waiting to be reused. Cite it, add to it (if allowed), and share your new results. I am confident this will add greater context to your research and highlight subtleties and nuances that might have remained invisible otherwise.

References

Eisner, R., & Vasgird, D. (n.d.) Foundation Text. In RCR Data Acquisition and Management. Retrieved from http://ori.hhs.gov/education/products/columbia_wbt/rcr_data/foundation/index.html

Johnson, B. (2016). Open Data: Delivering the Benefits. Presentation, London, UK.

Wicherts, J. M., Bakker, M., & Molenaar, D. (2011). Willingness to Share Research Data Is Related to the Strength of the Evidence and the Quality of Reporting of Statistical Results. PLoS ONE, 6(11). doi:hOp://dx.doi.org/10.1371/journal.pone.0026828

This article gives the views of the author(s) and not necessarily the views of the Centre for Evidence Based Library and Information Practice or the University Library, University of Saskatchewan.