| By Don Boyden |
In the study of literature, knowing the underlying theme is essential to really understanding a work. Literature homework assignments frequently ask students to compare, contrast, and explicate the themes of a given work. Themes are one of those things that frequently get a “yes” answer to the universal question, “Is this going to be on the final?”
Analyzing Usage Logs Gives Insights to Student Search Practices
At Gale, we’re always investigating how we can better serve the needs of students who use our online products. In support of that, I wanted to see what insights our usage analytics could provide on how students currently use our products to learn about the themes in literary works. This could in turn help focus our editorial and technical efforts to make their journey more successful.
Being a Systems Analyst, I started by going to the logs that capture the search terms people use in our products. I filtered the logs down to literature products and for the last 12 months (to accommodate cyclical assignments), then grabbed just the top 10,000 rows which represented over 1.5 million searches. Even at row 10,000, there were 23 people who submitted the same four-word query.
What the logs revealed is that there are as many different ways to search for themes as there are different themes—and they all get different results.
Students Try Many Different Approaches to find Themes
Some start out too broadly. There are thousands of searches, for example, on just the word “theme,” the search equivalent of a shot in the dark. The logs also show a block of searches on the words “themes literature,” which is a little better, but since these users were already in a literature product, the results were still going to be pretty broad.
Next, I saw many blocks of searches on specific themes. One of the most popular, used thousands of times, was the word “revenge.” A search on “revenge” does retrieve good, solid articles where revenge is a central topic, but it also pulls thousands of articles where the word is simply mentioned in the article title or a few times in the text. Our relevance sort tries to overcome these tangential mentions by boosting articles with the word in the subject, title, or keywords, but the results can still be mixed.
They Don’t Look for Authors
I wondered what other patterns I might find for the theme searches students submit. For example, do students search for a theme combined with an author name? I looked for all occurrences of the names of the three top authors in the search logs and how often they were paired with a theme:
- Out of 739 searches that combined Shakespeare with a keyword, at best 328 of them paired his name with a theme (women, love, gender, feminism). There were thousands of searches that paired him with a specific work.
- Shelley got no pairings with a term that looked like a theme. Over 1,000 searches paired her name with just “Frankenstein.”
- Fitzgerald had no pairings with a theme, but 1,120 searches paired his name with a work.
So, students are usually not searching for themes in association with a particular author.
But They Love the Works
Next, I checked to see if students search for themes in association with a particular work. Three of the most popular works that are searched in our products are Frankenstein, Hamlet, and The Great Gatsby. I grouped all of those searches and looked for patterns.
Statistically, the most common theme-based search pattern in our literature products is the pairing of a theme with the title of a work.
- For The Great Gatsby, 11% of all queries that included the title of the work paired it with a theme in the search.
- For Frankenstein, 14% of searches paired the work and a theme.
For Hamlet, 25% of searches on the title included a theme.
A Solid Approach – with Limits
Having found the most common approach students took to find information on themes, I did a little digging to see how well it worked. There are several issues that affect the quality and quantity of results that a work/theme search will return.
Inconsistency in Phrasing of the Theme Affects Search Results
I compared a number of the endless lists of themes I found on the Web that were compiled by literary experts. Some terms are on just about every list and are worded the same, such as “American Dream” and “Coming of Age.” But, many themes are worded differently from list to list, such as “Madness / Insanity” and “Women / Female Roles / Feminism.” No lists contain the same number of themes. Even top 10 lists have little overlap.
Student’s search phrases exhibit the same inconsistencies as the expert’s, including all the popular variations. Using similar search terms to get at the same theme, though, doesn’t always yield similar results. For example, students searching in our literature products variously paired the themes of materialism, money, wealth, and possessions with The Great Gatsby, and the results were all quite different.
Little Things Matter in a Query
Building a query that will return exactly what you want is not always easy to do. Little things like including a “noise” word (e.g.—madness in Hamlet), or inadvertently using a “noise” word that is also a Boolean operator (e.g.—madness and Hamlet), or adding unnecessary punctuation (e.g.—“madness in Hamlet”) will produce different search results. In all likelihood, students will not realize how such subtle differences will affect their search results. These are counts from the above searches in Literature Resource Center:
- madness hamlet: 214 total results
- madness in hamlet: 110 total results
- madness and hamlet: 2833 total results
- “madness in hamlet”: 10 total results
How to Make Things Simpler
On the assumption that students should not have to become experts on search engine syntax or be expected to automatically know the favored wording for a particular theme, what can we do to make the search process simpler and search results more consistent? This is a question that involves many teams across Gale, and as we delve deeper into user behavior and challenges we are actively working to provide better results.
Themes, Motifs, and Topics
When I started working on this blog, I assumed that the study of themes in literature was the domain of high school and college students. In doing research for the topic, though, one of the most helpful Web sites I found was actually created by a third grade teacher for her class! Interestingly, in the comments section of the Web site, she was criticized by some of her peers for allegedly confusing themes with motifs and topics. Hmm…what’s the difference between themes, motifs, and topics? Perhaps that’s fodder for a future blog.
Don Boyden is a Senior Systems Analyst at Gale. He enjoys tuning the search features in our online products that the much smarter people on the team actually build. In his spare time, he’s creating a steampunk graphic novel based on antique toys he’s collected.