To anyone looking to expand the types of dissertation work that students in your program create, or to any programs grappling with how best to evaluate a next-generation dissertation, consider the following:
- Start with what you know. Rather than starting by trying to define requirements for next-generation dissertations, first make sure that your program has clear standards in place for more customary dissertations. If your department doesn’t yet have this, it’s an important place to begin.
- If such a rubric already exists, great! The overarching metrics for next-generation dissertations will likely be very similar, if not identical.
- As part of the dissertation proposal process, provide students with a sample rubric. Let this be a formative process, though—ask students to read, reflect, and offer feedback and suggestions for how their own project can best be evaluated.
- Iterate. Build in a feedback mechanism so that students, faculty, and staff can offer suggestions for key values that may be missing or unarticulated.
One reason that it is so important to have clear standards in place for all dissertations is that it reduces the burden on students to uncover tacit knowledge. This burden tends to hit first-generation and underrepresented students especially hard, so clarity is an equity issue.
Rubrics
Any rubric will need to be adapted to particular needs and desires of a program. That said, rubrics can be flexible; at their best, they establish a framework rather than a comprehensive prediction of all possible forms. To ensure that the rubric is flexible and adaptable, build it based on values (e.g., contribution to field, quality of product, etc.) rather than by format or section (e.g., intro, lit review, etc.). These values should be consistent across genres and forms, and may help surface unspoken values of more traditional models.
The spreadsheet below (also available here) offers a sample rubric that can be adapted to meet your program’s needs. The sample rubric is divided into five key categories: scope, research, analysis, contribution to the field, and product quality. Because this evaluation is format-agnostic, the specific ways in which it is used will vary by project. For example, “product quality” in a monographic dissertation would pertain mainly to writing style and polish; for a film, this part of the evaluation would focus on production quality, effective use of sound/image, etc.
Your field may require additional categories. When creating a more specific rubric, consider the following:
- What methodological skills does a successful dissertation need to demonstrate in your field?
- What content or subject-based expectations are there for a dissertation in your field?
- What, if any, formal/structural components are essential for your field? Be selective. For instance, clear and informative citation is essential, but is a specific citation style truly necessary?
- Would these requirements hold equally well for a standard dissertation?
Self-Evaluation
In addition to the five categories in the rubric, consider one more metric: the strength of a candidate’s self-evaluation. Few programs currently include such a component as part of the proposal or dissertation process, but allowing students to self-evaluate has a significant pedagogical purpose and may greatly facilitate the work of evaluation, particularly for next-generation dissertations.
Students who engage in self-evaluation do so in a mode of meta-reflection in which they assess their own learning and work. When this is set up as an iterative process, perhaps with self-evaluation starting at the proposal phase and continuing through the defense, this may help a student to notice and improve potential issues at an earlier stage. For instance, a student who is constantly reflecting on whether their project has an appropriate scope may be more likely to have a final project that is neither too ambitious to be feasible nor too modest to show the level of research needed for the degree.
In addition, because next-generation dissertations may present project formats that are unfamiliar to advisors or committee members, a self-evaluation provides a guide for how the student thinks their work should be interpreted. It gives the student an opportunity to articulate what makes their work substantive and rigorous, and to provide a kind of map or key that suggests how to evaluate their success.
Select Examples of Evaluation in Action
What follows are a few examples of how evaluation in each of the categories might look in different types of dissertations.
Scope
This category assesses the candidate’s ability to select, plan, and execute a project that is of appropriate depth and length. It also assesses the appropriateness of the project type to the research question.
Example: We Rock Long Distance by Justin Schell (University of Minnesota, 2013; Comparative Studies in Discourse and Society)
This full-length documentary is part of ethnomusicologist Justin Schell’s multimodal dissertation investigating geographical, historical, and cultural underpinnings of the work of three Minnesota hip-hop artists. The project is ambitious and complex, and yet sharply focused. By limiting his consideration to a small number of artists in a constrained geographic area, Schell keeps the documentary tight while allowing for depth of analysis.
Schell talks about defining this scope in an essay reflecting on the project. “I interviewed more than sixty MCs, DJs, radio personnel, record store managers, and other community figures. [. . .] As the project developed, it became clear that it would need a more well-defined scope. I chose to focus on three particular artists from the Twin Cities who have roots elsewhere in the world.”
The research questions that Schell investigates are deeply grounded in his discipline while also being compelling to a broader, non-specialist audience. This is not a dissertation that anyone could do—rather, it is a remarkable example of one scholar’s rigorous, creative, unique approach to his work.
Research
This category considers the candidate’s research strength—their grasp of the field, engagement with existing research, sophistication of understanding, and intervention in the field.
Example: My Gothic Dissertation by Anna Williams (University of Iowa, 2019; English)
Anna Williams’ dissertation is a remarkable intervention into not only the gothic novel, but also the state of graduate education. Her use of an audio format makes it possible to move the reader through the argument with an intentional pace, and with auditory cues that signal the ways we are meant to read. She can lighten the tone while also adding signals (such as auditory ‘dings’ to signal footnotes) when deeper research is available in ancillary materials. The podcast format creates an ideal environment to use a wider range of research methods, combining textual and theoretical analysis with many interviews, personal observations, and material from mainstream media sources. The research depth is all there, in her insights and connections and background. And using a format that feels more conversational renders that research all the more powerful. The result is a nuanced, sophisticated exploration into the problems facing graduate education through a lens of literary criticism and analysis.
Analysis
This category evaluates the quality of research analysis, including methods, tools, argumentation, and findings.
Example: Multimodal Meaning Making by Sonia Estima (University of Illinois at Urbana-Champaign, 2020; Education)
In Multimodal Meaning Making, Sonia Estima undertakes a meta-analysis of the scholarly and pedagogical value of multimodal research. Her work combines video, audio, text, and other ancillary materials in order to examine similar multimodal projects with depth and rigor. As a language instructor, Estima has shared that she finds audio cues to be incredibly important to learning—what we hear in someone’s voice that we might not be able to glean from the page. By analyzing complex dissertations—including some featured on this website!—using a similarly complex mode of interpretation, Estima signals the plurality of readings that become possible when considering not only content, but form as well.
Contribution to the Field
This category assesses the overall importance of the project’s ideas and argument in relation to other research in the focal area.
Example: Owning My Masters by A.D. Carson (Clemson University, 2015; Rhetorics, Communication and Information Design)
A.D. Carson’s groundbreaking dissertation advances hip-hop studies through hip-hop, demonstrating the art as a form of knowledge in itself, not only as an object of study. By using the object of study—spoken word and hip-hop—as the mode of articulation for his scholarship, Dr. Carson insists on the validity and richness of hip-hop as a medium. More than that, the final work resonates with an audience that goes far beyond the academy. The album is engaging, high-quality; an excellent listen. As such, it opens up rigorous scholarship to a broader range of readers and listeners. It pushes traditional academic readers to question their modes of reading and knowledge production, while also uplifting and celebrating modes of communication and art that have not typically been embraced in academic spaces.
As Carson says in an interview with Clemson News, “The central thesis of my dissertation is: Are certain voices treated differently? I’m trying to examine how an authentically identifiable black voice might be used or accepted as authentic, or ignored, or could answer academic questions and be considered rightly academic. So, I have to present a voice rather than writing about a voice.”
The final product advances the field in ways that a standard dissertation likely never could.
Product Quality
This category pertains to the overall quality of the finished project in terms of its formal execution. It is not dependent on the quality of ideas or content. Rather, this section considers the project’s professionalism, aesthetic value, writing/production quality, or other markers of formal and stylistic excellence depending on the project genre.
Example: Vanishing Leaves by Jesse Merandy (The Graduate Center, CUNY, 2019; English)
Jesse Merandy describes his dissertation, Vanishing Leaves, as “a location-based mobile experience which takes players to Brooklyn Heights to learn about Walt Whitman in the neighborhood where he wrote and published the first edition of Leaves of Grass.” The game is immersive, and uses the embodied and situated act of walking in a particular location to engage deeply with Whitman’s work. The game is particularly notable for its pedagogical value; it could be used exceptionally well as a teaching tool.
The production of the game itself required significant technical skills, and the result is a high-quality user experience that fosters deep and organic interest in the subject matter.
These examples give just a small hint of what becomes possible when structure is not a limiting factor for a work of research. So much more is possible.