Oral Assessments to Reinforce and Evaluate Student Learning: A Case Study in Fluid Mechanics
| Category | Assessment, Group Project |
| Instructor | Brian Kirby, Meinig Family Professor of Engineering |
| Department | Sibley School of Mechanical and Aerospace Engineering |
| College | College of Engineering |
| Course Number and Name | MAE 3230: Introductory Fluid Mechanics |
| Discipline | Engineering |
| Course Level | Majors and minors, with prerequisites |
| Course Size | 150 students |
| Implemented | Fall 2023, Fall 2024 |
Brief Summary
In an introductory fluid mechanics course, students demonstrate their learning and their ability to justify engineering decisions through two oral assessments: a video in which teams present what they learned from their fluid mechanical dissection project, and a mock job interview in which students must show they can discuss class concepts and ideas as a professional in the field would. Both assessments offer students the opportunity to practice skills they will need when they enter the workforce.
Learning Outcomes
Oral Communication
Expert Analysis
Team Collaboration
Context
Engineering education has traditionally been focused on graded problem-solving and calculations, with exams that reflect this narrower view, rewarding specific types of students. In the real world, however, engineers don’t make calculations in the sterile environment of an exam. They do them as part of a project, with real equipment, working with a team, and in communication with others. New engineers often find themselves unprepared for this reality when they pursue positions in the workforce or graduate school. In 2019, Kirby decided to change this and began giving his students oral assessments that recreated realistic engineering scenarios. His mock job interview and fluid mechanical dissection presentation assessments foster his students’ ability to communicate their engineering decisions and reasoning clearly, while also helping them meet wider course learning objectives. In addition, the mock interview creates connections between the course material and career outcomes, reinforcing the relevance of the material.
In the real world, however, engineers don’t make calculations in the sterile environment of an exam.
Implementation
The mock job interview and fluid mechanical dissection presentation reflect two facets of oral assessment: a conversation between experts in the field and a formal team presentation.
Kirby designed the mock job interview as a four-minute interactive conversation drawing from 35 questions shared with students at the beginning of the term. Students were not told ahead of time which of the 35 questions would be discussed during the interview. They were evaluated on how well the conversation approximated a conversation between practicing experts discussing technical aspects of engineering, as might occur as part of a job interview. The exercise emphasized students’ ability to articulate field knowledge and engage in conversation, rather than focusing on the precision of their responses. Grades reflected students’ engagement in the conversation, as well as how well they expressed their knowledge and incorporated key terms and concepts illustrating the synthesis of fluid mechanics.
Students signed up for a time slot for their interview. Kirby found that six-minute time slots worked well for a four-minute interview, to allow for a buffer between interviews. With a class of 180 students, this assessment took approximately 18 hours over the course of a week. Note that these interviews replaced having an exam, which might have required an equal or longer time to prepare and grade. Students were encouraged to work together to prepare by practicing answers to the set of interview questions.
For the fluid mechanical dissection, student teams spent the final three weeks of class dissecting a piece of equipment, such as a carburetor or a hedge trimmer, identifying how it works and what fluid-mechanical decisions were made in its design. After analyzing the system performance, teams described what they did in a five-minute video for a technical audience. Students were expected to use technical terminology correctly, select and justify engineering models, and identify the fundamental fluid machines that were used in the device. They were also expected to collaborate as a team, share the work, and hopefully enjoy the process of taking apart a machine to understand how it works.
Students evaluated the videos of four other teams, based on a set of questions submitted through a form. The goal of the peer evaluation was not for students to grade another team, but to enhance their knowledge of fluid mechanics and evaluate effective communication techniques. Each student’s grade was determined by a combination of the quality of their team’s video and the quality of their assessment of their peers’ videos.
Challenges
It took Kirby several years to fine-tune the logistics of the mock job interview to reduce the time involved to six-minute intervals. While longer interviews are easier for instructors, they were not possible for a class this size.
Kirby also found that managing student expectations and explaining why he was using oral assessments required more effort at the start of the semester and before each assessment to allay student concerns about an assessment style that was new to them. Clear communication about what was expected of students in the interviews and providing the questions well ahead of time helped to mitigate some of these concerns. Similarly, clearly explaining the parameters for the team video project was key to helping students successfully learn from the project and understand its purpose.
“The fluid mechanical dissection and mock job interview were genuinely useful and helped me learn more about fluids than worry about my grades.”
– Student
Reflection and Future Directions
Kirby found that offering students assessments of engineering skills not captured with typical homework or exams emphasized the importance of those skills, while also empowering a wider range of students. Giving students first-hand experience with real engineering equipment helped them synthesize and remember the material, and (for many students) it was much more fun. Kirby noted that “they smile a lot when they first get to the inside of that engine or HVAC system or hard drive or inkjet printer and finally see how it is put together. These assessment methods have been a big success and have become a large part of what defines this fluid mechanics course.”
“Giving students first-hand experience with real engineering equipment helps them synthesize and remember the material and, for many students, is much more fun… and they remember it when I ask them about it a year later.”
– Professor Brian Kirby
How to Adapt This Approach
- Note specific ways students would use your subject matter in the real world.
- Outline scenarios or experiences you can provide to students via group or individual projects and tasks.
- Pilot the assessment with a modest size class or group of students (20 or less). It is much easier to do a 15-minute mock job interview than a six-minute one.
- Have students analyze the task or project itself and/or its outcomes or results.
- Analyze the process and revise learning activities accordingly.