Some Common Weaknesses in Assessment Systems

Assessing Student Learning
George Brown with Joanna Bull and Malcolm Pendlebury
(London, Routledge, 1997)

h2.gif (1649 bytes)


1.

Overload of students and staff
2. Too many assignments with the same deadline set in the department/school
3. Insufficient time for students to complete the assignments in the time available
4. Insufficient time for staff to mark the assignments before the next semester
5. Inadequate or superficial feedback provided to students
6. Wide variations in assessment demands of different modules
7. Wide variations in marking across modules
8. Wide variations in marking within a module
9. Wide variations in marking by demonstrators
10. Fuzzy or non-existent criteria
11. Undue precision and specificity of marking schemes or criteria
12. Students do not know what is expected of them
13. Students do not know what counts as a good or bad assignment/project
14. Assessment viewed by some departments/schools as an extra rather than a recognised use of staff time
15. Dissertation supervision seen as an extra or the real time involved is not recognised

Tick those that apply to your department and put a cross by those that do not. Compare your checklist with that of a few colleagues. What could be done to minimise the weaknesses of your department/school's assessment system?

Some Suggestions

Look through these suggestions, add to them and discuss their potential for assessment on your courses

1. Do an audit of the assessment procedures and time taken to mark various forms of assignments
2. Look at the degree regulations to see if they need changing
3. Consider whether one needs all the detailed marks for Part One, if Part One is essentially a pass/fail examination
4. Simplify the re-sit procedures
5. Specify deadlines, word limits for assignments and the date of returning the assignments to students
6. All course work should be handed in to office or technical staff
7. Require if possible all work to be word processed
8. Set some assignments as part of class time
9. Do some assignments during class time
10. Cut down on some feedback and on some assessments
11. Train some postgraduate research students or others to mark the work
12. Use some peer- and self-assessment
13. Make the criteria in use more explicit so that they are easier to use
14. Redesign the course so that the amount of time spent on assessment is less
15. Do a survey of student opinion of the organisation of assessment

Improving Assessment

There are four areas of assessment that should be explored for the purposes of improving assessment:

diamond-blue.gif (84 bytes) The design of assignments
diamond-blue.gif (84 bytes) The management of the assessment process
diamond-blue.gif (84 bytes) Marking for feedback (Formative assessment)
diamond-blue.gif (84 bytes) Marking for grading (Summative assessment)
diamond-blue.gif (84 bytes) Procedures for evaluating the assessment process
diamond-blue.gif (84 bytes) Procedures for evaluating the learning of the cohort of students

Which of the above have you considered recently?

What counts as a good assignment?

What counts, in your view, as a good assignment task or examination question?


Spend a few minutes jotting down your views and then compare them with a couple of colleagues.


Skills and content

The Dearing Report is likely to place greater emphasis upon transferable skills and preparation for work.

A common fallacy which is enshrined in that report is that skills can be learnt and assessed independently of content. Skills are always learnt in context. The narrower the context, the less likely the skills will transfer to other contexts. If you do wish to develop your students' skills then you have to look carefully at the range of assessment tasks that you set, what feedback you provide and how you introduce the notion of skills to students. Often students are unaware of what skills they are learning when tackling an essay, an oral presentation or a group project.

What skills do students learn on your history course?

Do you discuss them with students?


Which assignments and examination questions assess which skills?



Types of Essay Questions

There is no clear cut classification of types of essays, but there are families of essays that share common characteristics. A linguist, a post-modernist, or a philosopher might enjoy doing a research project on essay questions and their hidden meanings.

Some types are
Speculative
Quote to discuss
Assertion
Write on
Describe or Explain
Compare and Contrast
Discuss ...
Evaluate
Design
Problem-based Essays
Witty Questions


Some taxonomies for exploring assignments and examination questions


Below you will find a few examples of taxonomies that are useful for assessing the design of examination questions or for marking them. Look through the examples and decide which one might be helpful for you in designing or marking assignment or examination questions. Discuss one or two of them with colleagues.

Hierarchy of the Cognitive Domain

6. Evaluation Ability to make a judgement of the worth of something
5. Synthesis Ability to combine separate elements into a whole
4. Analysis Ability to break a problem into its constituent parts and establish the relationships between each one
3. Application Ability to apply rephrased knowledge in novel situation
2. Manipulation Ability to rephrase or paraphrase knowledge
1. Knowledge That which can be recalled

Based on Bloom. B.S. (1965) A Taxonomy of Educational Objectives Handbook 1: Cognitive Domain New York:
McKay 2nd Edition

At which level do you normally assess? At which level do your students normally respond?
Compare your questions with those of a couple of colleagues.

A useful activity is to try to design 6 questions on the same topic, one question at each level.

Forms of Understanding developed during revision

A Absorbing facts and details and procedures related to exams without consideration of structure
B Accepting and using only the knowledge and logical structures provided in the lecture notes
C Relying mainly on notes to develop summary structures solely to control examination answers
D Developing structures from strategic reading to represent personal understanding but also to control exam answers
E Developing structures from wide reading which relate personal understanding to the nature of the discipline

From Entwistle, N.J. and Marton, F. (1994) Knowledge Objects: Understanding constituted through intensive
academic study British Journal of Educational Psychology, 62, 161-178


Which approach do your students take?

Could you use this to develop a marking scheme or a set of criteria?


 

SOLO

Biggs (1997) describes how his taxonomy, SOLO (Structure of the Observed Learning Outcome), may be used for analysing learning tasks, hierarchies of objectives and for assessing students' work. The five levels of Bigg's schema are:

Pre-structural The task is not attacked appropriately. The student hasn't understood the point.
Unstructural One or a few aspects of the task picked up or used but understanding is nominal.
Multistructural Several aspects of the task are learnt but are treated separately.
Relational The components are integrated into a coherent whole with each part contributing to the overall meaning.
Extended abstract The integrated whole at the relational level is re-conceptualised at a higher level of abstraction. This enables generalisation to a new topic or area, or it is turned reflexively on oneself. (Understanding as transfer and as involving meta-cognition).













Pre-structural
could be regarded as barely satisfactory and Extended abstract as outstanding
performance.

Biggs, J (1997) 'Enhancing teaching through constructive alignment', Higher Education (in press)

Can you translate the above into your own language and approach assessment? How could you use this hierarchy in your assessment procedures?


The beginnings of some criteria

Each subject has its own emphases. So descriptions offered here should be adapted for use in your subject.

I Gets to the heart of the matter. Evidence of wide reading, analysed at depth to support arguments. All major points covered. Outstanding organisation and presentation for an undergraduate. Substantial evidence of personal interpretation. Virtually no irrelevant material. Correct referencing.
II.1 Wide reading. Issues understood and interpreted intelligently. Major points covered. Well organised and presented. Evidence of personal interpretation and a coherent argument. Material relevant. Correct referencing. Appraises critically each segment of the evidence and links them in a coherent informed argument. Hints at his or her personal interpretation.
II.2 Evidence of reading. Issues understood. Presentation and organisation clear. Most major points
covered. Provides the evidence and reports views on it. In so doing provides a fairly coherent answer to the question. Correct referencing.
III Provides evidence and reports views but does not relate them clearly to the question. A few major points not covered. Some evidence of organisation. Errors in referencing.
Pass Some major points not covered. Contains much irrelevant material. Little evidence of organisation. The question almost ignored.
Fail Very little evidence of reading or of understanding of issues. Insufficient or misinterpreted evidence and views. Jumbled. No or little attempt to answer question.




This page was last updated on 12 May 2000