It's thesis time again, which means that I'm facing a small pile of theses to review, and together with another course I've been teaching this semester, I thought to explore the topic of qualitative assessment a bit. Let's start with the opposite: quantitative assessment, which is the clearer of the two.

In the Logic, Algorithms and Data Structure course, some of the questions relate to subjects such as matrices and set theory. From a student and teacher assessment perspective, it's rather easy to see what needs to be improved: if a student can not make calculations with matrices or sets, then the student needs to improve in these areas. This is often a binary relationship: either you know how to, or you will struggle. It's usually very clear what needs to be improved for a higher grade or future development.

From a student perspective, this is also seen on an overall examination level: "If I get 27 points, I will get a Pass, but I only answered enough questions to get 24 points, so I need to answer more questions (correctly)". What the student needs to do in order to get a higher grade is clear, and this provides guidance for the student in assessing their own level of progress and knowledge in relation to the learning objectives of a course.

Qualitative assessment is much more difficult, and focuses on rather high level cognitive skills. You can find these assessment quite easily in an examination if the examiner asks questions which contain key words like "explain", "elaborate", "contrast" or "analyse". With such questions, it's quite possible to quantitatively answer all questions, but still receive a low assessment from peers and teachers. Thesis writing is perhaps an extreme example of this, and in thesis writing, even the question will be assessed (since you often formulate the question yourself)!

But what then, is the difference between a good and bad answer? From a student perspective, only knowing your grade, will not help you improve, and will find you at a loss. Imagine yourself if you have someone tell you that "the quality of your work is not good enough." How can you improve from such a statement, especially if you're in the early stages of an academic career and you have little experience from reading high quality academic material.

At the same time, it's difficult from a review perspective to dive into details, and I find this to be a bit hazardous. I've faced this once or twice when I was first teaching at a university level many years ago. A student came to be to inquire about his grade (he had failed on a written report). In order to help the student understand what he needed to do in order to pass the course, I indicated the paragraphs which I found demonstrated more clearly the level of quality of his report, and I rewrote one of the paragraphs as an example of what would constitute a higher quality. After about a week, I got his report back with all the paragraphs I indicated rewritten, about 20% longer, and with British English instead of American English. Nothing was changed in the rest of the report.

When the student again got a failed grade on his report, he was more than upset. His major argument was that I had told him which paragraphs to rewrite, and he did rewrite those, so how could he then fail, if he did everything I asked? This shows that the student had missed the point of my comment; that the quality of his argumentation and writing was not there. Instead, he focused on the quantitative assessment. 25% of his report needed to be rewritten, so he rewrote 25%, and then 100% of his report ought to be good.

At the same time, not giving direction is just as damaging. For someone to improve, it's necessary to give feedback in a constructive way. I will use this article to highlight two different examples of writing to demonstrate the difference in quality between a weak and strong grade. This is fondly adapted from the University of North Carolina at Chapel Hill. Given a question of comparing and contrasting the reasons why the North and South fought the Civil war, the following three paragraphs illustrate three different levels of quality:

• The North and South fought the Civil War for several reasons. Many of the reasons were the same for both North and South, but there were also differences.
• While both sides fought the Civil War over the issue of slavery, the North fought for moral reasons while the South fought to preserve its own institutions.
• While both Northerners and Southerners believed they fought against tyranny and oppression, Northerners focused on the oppression of slaves while Southerners defended their own right to self-government.

The first example is a simple restatement of the original question and contributes nothing new. This would be a clear example of low quality in academic writing, and would not pass an examination. It doesn't even offer any room for the reader to guess at what the intention of that paragraph might be. Answering the question "what was the reason?" might lead you to the second example paragraph: it states what the issue was in the Civil War, and provides the basis for an argumentation on the difference of motivation between North and South.

Going deeper into the subject might lead to the third example paragraph. It is a much clearer statement than either of the previous two and captures the main qualities. Keep in mind that the above examples only highlight one interpretation of the Civil War, taken out of context; it is by no means an absolute answer, and the rest of a thesis starting with the above, would continue to provide justification for this line of argumentation.

A second example would be that you're asked to analyse some aspects of Huckleberry Finn. Again, we can look at three different levels:

• Mark Twain's Huckleberry Finn is a great American novel, and highlights many different issues. It's actually called Adventures of Huckleberry Finn and was published in 1885.
• In Huckleberry Finn, Mark Twain develops a contrast between life on the river and life on the shore.
• Through its contrasting river and shore scenes, Twain's Huckleberry Finn suggests that to find the true expression of American democratic ideals, one must leave "civilized" society and go back to nature.

Again, the weak example of a statement contributes nothing that would actually answer the question asked. It restates part of the question and introduces facts which are not relevant to the question (perhaps in an attempt to increase the length of the answer). The second example answers the question, but not in any great level of detail, whereas the third one provides the basis for a good analysis of the novel, which is exactly what the question asked for.

These are just two examples, and a couple of weeks ago I came across other examples demonstrating the same principles but with a slightly different focus. I will make an attempt to find them, since I believe that examples of different levels of quality, will probably help students more to understand the difference between a good and a bad paper, much more so than a quantitative indication of their flaws.