Writing about Controversies in American Politics: Using AI to Examine Counterarguments and Definitions
| Category | Writing assignments Developing AI literacy |
| Instructors | Peter Joachim Katzenstein, Professor |
| Amelia C. Arsenault, Graduate Teaching Assistant Musckaan Chauhan, Graduate Teaching Assistant | |
| Department | Department of Government |
| College | College of Arts and Sciences |
| Course | GOVT 2817: America Confronts the World |
| Discipline | Political Science |
| Course-level | Undergraduate, Introductory to Intermediate |
| Course size | 73 students |
| Implemented | Fall 2023 |
Learning Outcomes Targeted
Developing AI literacy
Writing Skills
Critical Thinking
Examining Multiple Viewpoints
Brief Summary
Students use large language models (LLMs) like ChatGPT iteratively to identify and co-produce counterarguments used in the debate of controversial subjects in ways that assist with, rather than replace, their own written work. Through a series of guided steps, they are tasked with using LLMs to co-produce written argumentation. Students are prompted to provide an analysis of their experiences and realizations regarding where and when LLMs can be used most effectively.
Context
This political science course asks students to engage with the multiplicity of viewpoints and visions within American politics and foreign policy. In this writing-intensive course, students complete several assignments that ask them to examine controversies in political policies and how words like ‘democracy’ can be used in different ways with different meanings in political argumentation. In the fall of 2023, the teaching team was contemplating how best to explore the ongoing discourse in academia regarding large language models and the utilization of AI in the classroom. After seeing a variety of reactions from institutes of higher education, ranging from banning to embracing AI in the classroom, they concluded that they should teach students how they can use these tools in ways that will be most useful for them; specifically, how they can thoughtfully and critically integrate AI into their writing as opposed to utilizing AI to generate their work for them. Assignments asked students to explore how ChatGPT handles prompts around counterarguments or definitions of key concepts, as a way of viewing different perspectives around a controversy.
Implementation
Throughout the course, students were prompted to use generative AI toward the co-creation of written work. Through a series of drafts and self-reflections, students critiqued the use of AI by identifying its strengths and weaknesses. Students were given three directives for their use of AI; “Brainstorming a Prompt with ChatGPT,” “Identifying Counter Arguments with ChatGPT,” and “Explaining and Defining Concepts with ChatGPT.” First, students conducted their work solo, based on course readings and discussions. Then, they asked ChatGPT to conduct the same exercise. Students were asked: Does your detailed analysis match the definition that the chatbot came up with? Why? Why not? What are the differences and similarities between the chatbot definition and your articulation of the term? In another assignment, students were asked to use a ChatGPT counterargument to strengthen their original thesis.
Challenges
When first using the AI tools, students felt that they had to use the first output they received. This was problematic to the goal of having students engage in a critical analysis of the information, so the teaching team looked at how they could encourage students to go through the iterative process of fine-tuning the prompts and the output. After varied attempts, they found the solution was to be very explicit with students regarding the expectations and directions.
“I emphasized that this is an experimental sort of process – we want you to learn how you have to prompt the machine to give you what you need and what you want.” Amelia Arsenault, teaching assistant.
Reflection and Future Directions
“To our surprise, most students found ChatGPT to be cumbersome in generating objections to their arguments, with most students citing a lack of specificity.”
Peter Katzenstein
Students shared that they enjoyed the chance to explore the use of LLMs and discuss how they may be useful in their writing process. While in the beginning, the students were very polarized – embracing or skeptical – in their response to using AI, in the end, the majority emerged more neutral on its utilization and pragmatic in their approach.
“To our surprise, most students found ChatGPT to be cumbersome in generating objections to their arguments, with most students citing a lack of specificity.” Peter Katzenstein, instructor.
The instructors also note that students engaged critically with the nuances of how different terms such as “democracy” or “authoritarianism” are used by different authors and also how they are defined by ChatGPT. Using the AI and non-AI prompts together helped students build the critical thinking skills that are a key component of the course.
“Now that people are more familiar with LLMs we could have more prompts. Instead of compare and contrast, which is a great starting point, more creative prompts would be really fun to do with students.” Musckaan Chauhan, teaching assistant.
How to Implement This in Your Class
To use generative AI constructively in a course, consider the following:
- Engagement: Have regular engagement with your students, either through 1-on-1 check-ins with the instructor or with teaching assistants. Take special care to ensure that students can properly discuss and vet their ideas
- Experimentation: Encourage students to fully engage with and explore the limitations of the AI tool, to push the boundaries of how it can be used, and to recognize its shortcomings. Ask students to provide feedback and self-reflection on their endeavors using AI as a co-producer of their work. What has worked well? What has failed?
- Expectations: Share with the students the expectations of the assignment; specifically, how they are meant to use AI as a thinking and writing assistant. Be sure that they understand how they will be graded. Students also need guidance in proper citation practices regarding the use of generative AI.