Aligned teaching and punishing student success

This term I’m teaching (and coordinating) a 1200 person Integral Calculus course at UTM. Term Test 1 went surprisingly well for students, and Term Test 2 was an unqualified success (very high average, lots of people got 100% on the test).

This lead to the following (paraphrased) question on our course message board:

Is the exam going to be a lot more difficult because the averages on the term tests were so high?

I took the time to think about exactly why I don’t like adjusting the difficulty of exams based on students doing well previously. Here’s the answer I posted on our message board.

Some definitions and terms

Here are some terms I use specific to our course and UTM:

1. MAT135 – Differential Calculus, the big prerequisite course at UTM with 1400 students.
2. MAT136 – Integral Calculus, the big integral calculus course at UTM with 1200 students.
3. Piazza – an online message board for our course.
4. WeBWork – An online platform for students to answer computational calculus questions. It is for grades, but it is low stakes.
5. FSG – Facilitated study group. Think optional, small tutorials lead by TAs about specific topics.

My answers

There’s a short answer, and a long answer.

Short answer

No, we won’t artificially make the exam super difficult just to achieve some historical average. No, we don’t “curve” anything after the fact. (We occasionally, but very rarely, drop a question after the fact because it was unfair.)

Long answer

The way that I teach/coordinate is called an “aligned” approach. This means that:

1. As a community (including students, community members, and other faculty if possible) we come up with the course-level goals (Learning objectives/Outcomes or LOs). This is what we hope students will be able to do at the end of the course. These are public knowledge, and are included in the course syllabus. Here are ours for MAT136:

On successful completion of MAT136H5, you should be able to solve problems related to integral calculus, which includes definite and indefinite integrals, integration techniques, improper integrals, sequences and series, area and volume problems and other related applications. A list of topics can be found below.

You should aim for a level of understanding that allows you to:
(1) carry out computations with ease;
(2) use your conceptual understanding of the material to solve a range of problems, even ones that are different from, or a variation of problems you’ve seen before; and,
(3) give an explanation of your solutions to someone who has not seen the material before (i.e. you should aim to understand the
material well enough to be able to explain each step in a calculation, but also the general idea behind the solution).

2. Every week/class/section/tutorial/assignment, we create LOs for the week/class/section/tutorial/assignment. These are (typically) public and I include them in my slides and they are on the tutorial handouts. The course-level LOs are the broad, zoomed-out goals, but these weekly LOs are the specific, concrete skills. The design is such that if students can do all the weekly LOs, then at the end of the course they should comfortably have achieved the course-level LOs (and at the end of your degree all these course-level LOs should add up to degree-level LOs…)

3. (This is an aside.) I really like this approach as a coordinator because it allows all the instructors and TAs to know the common expectations, and they can teach in any way they think is best so long as it helps students achieve the LOs.

So, blah, blah, blah, we set some expectations for students.

Tests and exams

“If I want to know if you can write a haiku, I won’t ask you to write a haiku specifically about quantum mechanics.”

Now, here’s where the assessments (tests/exams) come in:

1. We create tests/exams that help students show they’ve achieved the LOs in the course. The questions we ask are designed specifically with the LOs in mind. Think about Term Test 2 Q2, this was exactly designed to assess the Week 7 LO ” Determine if an improper integral converges.”. We made sure the algebra wasn’t too difficult, so that students could focus on the “improper integral” part of it and not unrelated material. As a guiding principle we try to make each question assess one relevant LO, not many distinct skills.
2. Our goal is not to surprise students or have “gotcha” questions. Some questions will be uncomfortable, or unusual, and that’s mostly because we’re trying to assess skills related to that. This was Term Test 2 Q4.2: The challenge was in reorganizing the fraction into two easy integrals; we gave you a chance to show off that you can recognize which integration techniques are appropriate (a week 7 LO).
3. We only assess you on things you’ve learned in the course. If we test you on a question that you couldn’t have learned in the course, then what we’re really assessing is “who is privileged enough to have a strong background in math”; that’s an evil and inequitable thing to do. We assume everyone has the posted course prerequisites (a 50 in MAT135), but we don’t assume anything else. When possible we simplify models and language so that it doesn’t distract from the things we’re trying to assess. (Think about all the related rates questions in MAT135 tests/exams that used circles, triangles, and rectangles instead of more complicated geometry; this was a conscious decision by the instructors.) If I want to know (for example) if you can write a Haiku, I won’t ask you to write a Haiku specifically about quantum mechanics.
4. The tests don’t care about the results of other tests/courses. This is a direct consequence of the previous point. Ideally I can give any term test to any previous or future generation of students, and it will still accurately and fairly assess their abilities.
5. (An aside.) The assignments are very different from tests, and are not designed to assess your skills, they are designed to develop your skills. Obviously you haven’t done most of the stuff before that we’re asking you to do on the assignments; we know that. It should feel uncomfortable (at first), and you should get stuck (and then ultimately succeed). This is why we tell you the grading schemes ahead of time, and give you access to a lot of resources (20+ office hours a week, FSGs, Piazza). You’re U of T students; you’re awesome, and most of your life you’ll spend it solving problems that you’ve never seen before and no one has ever solved.

So, blah, blah, blah, our tests are designed to directly assess the stated LOs.

Do we mess up sometimes? Yeah, of course. We’re not perfect, and satisfying the needs of 1000+ students who are stressed and uncomfortable (and who occasionally send you insulting/threatening emails) is a difficult challenge. It’s the hardest thing I’ve done in my life so far, but it’s also been one of the most fun and rewarding things I’ve done in my life. You don’t see all the (many!) private posts on Piazza where students start out struggling, and then have major breakthroughs. I live for that.

Academic integrity and online teaching

“Don’t punish good-faith students for the actions of bad-faith students.”

My values:

1. I want students to succeed. I try my best to set you up for success in every way I know how.
   1. Artificially adding in hoops to jump through disproportionately hurts students acting in good faith.
   2. Don’t punish good-faith students for the actions of bad-faith students.
2. Don’t do cop shit. In short, I start from a place of assuming that students are acting in good faith and that we are colleagues, not adversaries.
   1. Online exam proctoring is mostly evil.
   2. Most surveillance is needless, and should be avoided.
   3. There’s one piece of “cop shit” that I do: I ask you to click on the academic integrity statement for a test. (Even then, about 10 people per test contact me, completely stressed out about it.)
3. Set up situations that naturally discourage cheating. This is why you can drop your lowest term test, and 2 tutorial activities, and have lots of time to submit WeBWork and written assignments. Why cheat when you can just not hand something in?
4. Make expectations clear. Every test and assignment tells you what is and isn’t allowed on page 2.
5. Allow lots of tools. All the tests are open-book, you can use calculators, and while you technically are not allowed to use Desmos, we create the tests so that it doesn’t really help.
6. Provide lots of support. Piazza, 20+ office hours a week.

So blah, blah, blah, we tend to encourage good-faith acting rather than overly focus on preventing bad-faith acting.

All that is still not enough for some people, and some people will set up group chats, use Chegg, or get people to write their tests for them. The actions of those people cause a huge amount of time to be spent by the instructors and TAs. I alone spend about 20-40 hours per assessment following up with students about potential academic misconduct. That is time that I can’t spend teaching or making the course better.

Summary

All this is a bunch of blah, blah, blah to say:

1. We set up transparent goals.
2. We assess those goals as directly as possible.
3. We assume most people are acting in good-faith.
4. I don’t really care about historical averages.

I’ve been teaching online for 4 years now (I taught online courses before Covid) and I’ve learned a lot about how to run an online course. Students tell me what they like, what they don’t like, and what they need to succeed. I’ve learned how to balance fair assessments with academic integrity.

I still have a long way to go though, so continue giving me feedback and keeping me honest. Thanks for reading.