Learning means doing
For skills courses, active learning is a must. That means exercises, and lots of them.
Students need feedback. Grading has two goals. The more familiar goal is summative assessment, which measures the student. If Molly gets a B+ in a writing course, that tells you about Molly. Knowing that she got a B+ doesn’t help Molly learn how to write.
For students to learn from an exercise, they need formative feedback. That is, a list of things they did right and wrong.
Students should have a chance to correct their mistakes on exercises, and resubmit. Why? So they read the feedback, think about their mistakes, and figure out how to correct them. This maximizes the learning students get for their effort on an exercise.
Notice that the feedback has to be individual. Molly gets feedback about her work, Kevin gets feedback about his, Sandra gets feedback about hers, and so on.
So we need:
- Individual formative feedback
- A chance to improve
Computers can’t replace people
Computers can’t grade essays, reports, programs, etc. It takes a human to give formative feedback. If there are 50 students in a course, with 20 exercises per student per semester, that’s 1,000 submissions to be graded.
What usually happens here is that professors give up. It isn’t practical for them to grade 1,000 submissions. They switch to test banks and multiple choice questions. Student learning suffers.
If we take skill learning seriously, we can’t do this. The facts are clear: students need formative feedback from humans. We need to find a way.
Cyco makes it economically feasible to give formative feedback at scale. Cyco does two things:
- Streamlines the grading process, using “clickable rubrics.”
- Allows grading work to be distributed to many people.
Let’s see how this works.
Authors create many exercises, and embed them in their Cycos. They create rubrics for each exercise. A rubric is a standard for assessing exercise solutions. Graders use the rubrics to assess student work.
An example. Here’s an author creating an exercise:
Notice the token `[custom:rival-school]`. Authors create whatever tokens they want. The tokens are replaced by different text in different contexts. For example, one of the University of Michigan’s rivals is Michigan State University. Here’s what a student at the University of Michigan might see:
The author isn’t done yet. S/he must also create a grading rubric for the exercise. It consists of one or more rubric items. Here’s the author creating a rubric item:
The feedback phrases on the right are comments given to students, depending on whether they met the standard, got part way there, or didn’t meet it at all. Authors don’t have to complete comment lists ahead of time. Graders can add new phrases as they go.
Exercises can have as many rubric items as authors want. This one has two:
Authors can also give model solutions, and grading notes. They’re optional.
Authors don’t type exercises directly into course pages. They create exercises separately, and then embed the exercises in course content. Here’s content from the course the author is working on:
!(bordered)/sites/default/files/bucket/authoring-content.png(Embedding an exercise)
Separating exercises and content lets authors work on them separately. Authors can change exercises, add rubric items, etc., without needing to mess with course content.
Also, a team of authors can divide up the work. One might do content, and another exercises. They can work separately, without stepping on each others’ toes.
This is what students see:
Stu is a student in the class. He clicks the `Work on it` link to…, er, work on it. A separate popup windows appears, and the user enters his/her solution:
Notice that students can type text, or attach files.
A grader assesses each submission. Here’s a grader about to start grading the submission:
The rubric items defined by the author are on the right. Notice the feedback phrases in blue. The text fields let graders add new phrases. Over time, the phrase set will become more complete. This is an example of continuous improvement, important if courses are to last a long time.
The grader clicks two phrases, and the `Create feedback` button. Those three clicks generate a feedback message:
The text in the lower right was written by Cyco, not the grader. The message has the student’s name (Stu), and the grader’s (Jen). Graders define greetings, signatures, etc., in their own voices, using tokens. Here are some greetings and signatures in different voices:
- Surfer dude greeting: Yo! [feedback:student_first_name]!
- Formal greeting: Dear [feedback:student_first_name] [feedback:student_last_name],
- Hippie signature: Peace out. Pat.
- Rocker signature: Rock on, y’all! Jiiiiiimmmmyyyyy.
Graders define as many of these as they like. If a grader has four signatures, Cyco will choose one randomly.
Notice that it took only three mouse clicks to generate the personalized formative feedback message. The grader can edit the message at will, adding suggestions, links, whatever s/he wants. However, the basic three-click assessment could be done in less than a minute.
You don’t need a Ph.D. to do the grading. There are potentially millions of people around the world who could do this work.
Next time Stu (the student) logs in, he sees:
The student clicks `Review`, and sees:
Stu has work to do. He creates a new version of his submission:
He corrects the first line of the limerick:
The grader looks at it, and gives feedback. This time, the grader decided that Stu had completed the exercise.
Here’s the message:
Stu can get a report on all of his submissions. It shows:
There’s a completion badge for the exercise. A badge count could be the basis for a leader board.
To learn skills, students need to do exercises and get feedback. They need a specific list of things they did right and wrong. Every student needs feedback about his/her own work, individually.
They also need a chance to correct their mistakes, and resubmit. This makes them look over their work, see where it’s lacking, and fix it.
This is a lot of grading. However, if we want to take skill learning seriously, there is no choice but to do the work. Cyco has a grading interface designed for formative feedback at scale. Graders can be anywhere in the world. They need not be professors.