The Learning Objectives Questionnaire (LOQ)

 

Introduction

The Learning Objectives Questionnaire (LOQ) was designed by Norton, Horn and Thomas (1997) as an ‘alternative’ course evaluation instrument, which is flexible and can be adapted for any course. It has been written about in my book (Norton, 2009) but colleagues are invited here to use it and adapt it for their own purposes.

 

Rationale

The LOQ uses a combination of closed and open-ended response questions which takes account of the extent to which students have actually understood what lecturers are trying to teach them. In this way, it differs from more traditional course evaluation instruments which can be affected by factors such as students’ expectations, study approaches, preferences and learning styles (see for example, Entwistle & Tait, 1990; Carkenord & Stephens, 1994; Sprinkle, 2008).

The LOQ was designed to minimise these effects by actually telling students what the main learning objectives are for each element of the course and then asking them to rate how successfully they think these objectives have been achieved.

The advantages of this type of approach to course evaluation are threefold:

  1. The lecturer’s aims are clearly specified, so regardless of students’ preferred approaches, they specifically evaluate the elements that lecturers have identified as important.
  2. Constructing the LOQ encourages lecturers to think in a more focused way about precisely what they want students to learn in their courses – an integral part of course design.
  3. Concentrating on lecturer specified objectives, makes it easier to determine what specific modifications might need to be made to improve the effectiveness of a course.

 

Using the LOQ for pedagogical research

This tool could be adapted to evaluate the effectiveness of a pedagogical intervention, where a teaching or assessment issue has been identified as problematic.

 

Constructing a Learning Objectives Questionnaire

  1. Give some form of general instruction at the beginning of the LOQ to explain the rationale which shifts the focus from more traditional course evaluation to students’ perceptions of how successful lecturers have been in conveying their learning objectives.
  2. Select those elements of your course that you want to evaluate and present your learning objectives in the form of simple statements. There is no limit to the number of objectives you could list but if you have a large number of course elements, it might be advisable to limit the objectives to no more than three.
  3. Ask students to respond to each specified learning objective on a 5 point rating scale : Very successful, Successful, Unsure, Not successful, Very unsuccessful which are scored as 5,4,3,2,1, respectively. Thus a high score indicates a positive student evaluation and a low score indicates a negative evaluation suggesting the learning objective has not been perceived as successful.
  4. Obtain scores for each individual learning objective and total to give an overall score for the course element.
  5. After each course element and related learning objectives, ask the open-ended question:
    ‘What is the single most important thing you felt you learned from…’

    This question is based on Angelo and Cross’ (1993) ‘minute paper’ where they suggest that after each taught session, students are asked to write brief answers in response to two questions about what they had learned and what they felt they had not yet learned. Angelo and Cross argue that the advantage of this method is that it can be used to indicate to the lecturer where misunderstandings have arisen so that they can be discussed in the next session and put right. This idea has been adapted for the LOQ by asking just the one question about what students had learned. In this way it is possible to capture some important learning experiences that the LOQ might otherwise miss. It also enables lecturers to use the responses to interpret the ratings and identify areas of potential misunderstandings between lecturers and students. An example of how this can work is reported in Norton, Horn & Thomas (1997) where it is argued that using the LOQ can be a useful part of course design and modification, particularly when it is used as an ongoing and interactive course evaluation tool.

 

Example LOQ items

To illustrate the procedure an extract from a Counselling Psychology LOQ is given below.  These items are taken from the Counselling Psychology course reported in Norton, Horn & Thomas (1997) and are chosen from a much longer instrument to illustrate how the LOQ can be used to incorporate different course elements in the same format. The three course elements here represent a lecture, a workshop and an assessment task.

 

The lecture element

This questionnaire has been designed to find out how successful you think we have been in achieving specific aims on this course. What we have tried to do is identify examples of different elements of the course and, for each of them, state what we intended would be the learning objectives. What we really need to know is whether they have worked or not, so please be absolutely honest.  For each objective will you please circle the one response that most accurately represents your view, where:

    VS = Very Successful,
    S = Successful,
    U = Unsure,
    US = Unsuccessful,
    VUS =Very unsuccessful.
Learning objectives for the  lecture on the person-centred theory of Carl Rogers
To understand the fundamental principles which apply to nearly all therapies in counselling VS S
To understand that a theory is just a theory and has weaknesses as well as strengths VS S
To question for yourself to what extent childhood experiences determine the type of person you become as an adult VS S
What is the single most important thing you felt you learned from this lecture?

 

The workshop element

Learning objectives for the workshop on the life-script exercise
To apply some of the theoretical principles of Transactional Analysis to your own life VS S U US VUS
To help you develop a greater self-awareness VS S U US VUS
To begin to build up trust between members of the group VS S U US VUS
What is the single most important thing you felt you learned from this workshop?

 

The assessment task element

Learning objectives for the individual, peer assessed seminar presentation
To develop your skills of oral presentation VS S U US VUS
To increase your reflectiveness about a topic you chose for yourself VS S U US VUS
To critically evaluate fellow students’ presentations VS S U US VUS
What is the single most important thing you felt you learned from this assignment?

 

Template to create your own learning objectives questionnaire

Instructions

This questionnaire has been designed to find out how successful you think we have been in achieving specific aims/learning objectives on this course. What we need to know is whether you think they have worked or not, so please be absolutely honest.  For each objective will you please circle the one response that most accurately represents your view, where:

    VS = Very Successful,
    S = Successful,
    U = Unsure,
    US = Unsuccessful,
    VUS =Very unsuccessful.
Learning objectives for ..
VS S U US VUS
VS S U US VUS
VS S U US VUS
VS S U US VUS
What is the single most important thing you felt you learned from ….? 

 

References

Angelo, T.A.& Cross, K.P. (1993) Classroom assessment techniques.  San Francisco: Jossey-Bass

Carkenord, D.M. & Stephens, M.G. (1994) Understanding student judgments of teaching effectiveness; a “policy capturing” approach, The Journal of Psychology, 128, 6, 675-682.

Entwistle, N. & Tait, H. (1990)  Approaches to learning, evaluations of teaching, and preferences for contrasting academic environments  Higher Education, 19, 169-194.

Norton, L.S. (2009) Action research in teaching and learning. A practical guide to conducting pedagogical research in universities, pp. 159-162. Abingdon: Routledge.

Norton, L. Horn, R. & Thomas, S. (1997)  Innovatory courses: matching lecturers’ objectives with students’ perceptions and academic performance.  In C. Rust & G. Gibbs (Eds.) Improving student learning.  Improving student learning through course design.  Oxford: The Oxford Centre for Staff and Learning Development. Part X, chapter 30, pp 331-343.

Sprinkle, J.E. (2008) Student perceptions of effectiveness: an examination of the influence of student biases, College Student Journal, 42, 2, 276-293.

 

Citations

To use the LOQ, please use the following references:

Norton, L. Horn, R. & Thomas, S. (1997)  Innovatory courses: matching lecturers’ objectives with students’ perceptions and academic performance.  In C. Rust & G. Gibbs (Eds.) Improving student learning.  Improving student learning through course design.  Oxford: The Oxford Centre for Staff and Learning Development. Part X, chapter 30, pp 331-343.

Norton, L.S. (2009) Action research in teaching and learning. A practical guide to conducting pedagogical research in universities, pp. 159-162. Abingdon: Routledge.

To use any information from this page please use the following citation:

Norton, L. (2011, June1). The Learning Objectives Questionnaire (LOQ).  An ‘alternative’ method of obtaining student evaluations. Retrieved (insert date i.e.month, day, year) from http://linnorton.co.uk/

Comments are closed.