Sunday 2 September 2012

Evaluation of an existing course assessment

The assessment I have decided to concentrate on forms one of twelve small evaluations within the Hospitality Operations course in the Certificate in Applied Travel and Tourism.  A one year, level three programme with minimum level 2 NCEA and an interview prior to entry.

1st Semester (All papers)
Day in the life of a Tourist
Day in the life of a Conference coordinator
Day in the life of a Travel agent
Day in the life of a Tour guide

2nd Semester (Choose 2)
Hospitality Operations
Travel agency
Tour guide
Each paper must gain a pass mark in order to obtain the Certificate in Travel and Tourism.

The paper is an introduction to the Hospitality Industry with an emphasis on Front Office and Reservation roles.  The students have access to the course outline, information, activities and assessments through Wiki educator http://wikieducator.org/Hospitality_Operations and a computerised virtual Hotel reservation system.  This system enables them to check customers into a hotel, add expenses to their account and check them out, this is assessed by providing role play opportunities, written assessments and a virtual hotel closed book evaluation.

A brief outline of all the assessments within this paper;

  • Unusual Hotels (presentation)
  • Impacts of mass Tourism (case study)
  • Hotel packages (presentation)
  • Hotel reservation/cancellation policies (word processed)
  • Reception Theory assignment (word processed)
  • Telephone reservations (role play)
  • Checking guests into the hotel (role play)
  • Checking guests out of the hotel (role play)
  • Business correspondence (word processed)
  • Alcoholic beverages (practical/closed book written assessment)
  • Virtual hotel assessment (closed book computer assessment)
Grading; The course is set up in a logical, sequential manner so that there is rich learning on the way and there are opportunities to build on and reinforce their learning opportunities at different stages.  As the emphasis is on building on what they have just learnt rather than the final outcome the grading is pass or not passed.  
( Having said that, with the move towards statistical outcomes and lecturer accountability, I do struggle with the ethics of pushing students through at the last minute just to catch them up, how much learning has been done along the way for them? )

My focus for this topic is the 3rd assessment in the group namely, Hotel packages.  The students are required to do some background research, collect evidence and explain their own hotel package in the form of a presentation.

 http://wikieducator.org/Hospitality_Operations/Activities/Packages


 What is the purpose of this assessment?
Good question as often I think to myself, I know what I want to achieve but does this actually do it?
The aim of this assessment is twofold and hopes to provide an opportunity for the students to do some
research and to start to create an awareness of where and how these packages are advertised.
Secondly, to create their own package and present it in such a way as to sell the dream to their audience.
I see this assessment as a fair way to test their 'understanding by explaining their ideas and concepts'
(Bloom's revised Taxonomy;2001)which, with the appropriate level of feedback will provide
opportunities for the students to 'apply their understanding in another familiar situation'. (Bloom, 2001).


Does this happen?
In general, the quality of the presentations is very high and the students really enjoy producing
their own package, however some do find that doing a presentation and having it marked at the
time quite stressful, I think this is an area for improvement.  I would like to create a clear feedback
form to be used during the presentation so that guidance can be given one on one rather than in the group.
To extend this further I would like to design a simple peer feedback form for the other students to use 
during the presentation, which may help to focus their attention while their colleague is speaking.


How does it enhance students' learning?

I think the concept of this assessment helps to enhance the students learning if adequate research
is done and adequate exploration of hotel packages is pursued. The assessment is relevant
in helping the students to understand why packages are produced, the importance of
creating a sales concept to actively promote the hotel during the slower months/periods.
Formative feedback is given directly after the presentation, I may need to work on my technique
for this after Bronwyn's workshop on Friday!

How effectively does it measure student's learning outcomes?

This is where I feel I need to do some work on revising the assessment. The concept is good but the way
the assessment is laid out and the wording needs work. I do not think it is very easy to understand and
it needs to align with the learning outcomes more closely. The marking sheet needs to offer a better
guideline for the students in its layout and design.
Example of marking sheet;
I think that the assessment has a reasonably good basic structure but the instructions are not clear
and the marking criteria is not accurate enough to enable feedback to be given in an easy, clear and
transparent manner.  There is confusion with what is required for the other questions in the assessment
and work handed in has sections missing and has to be handed back for re-sits.  This is time consuming
and unproductive. 




2 comments:

  1. Hi Helen
    This sounds like a fun assessment and one that must really challenge the students to learn lots.
    The link to an example of a marking sheet appears to be missing.

    You have provided some very good background information about the assessment. I can see exactly where it fits in the programme. You have also provided a description of the actual assessment (a link to it would have sufficed) so I can understand what the students are required to do.

    As this is an evaluation of an existing assessment, most of the description for this assessment needs to be about your evaluation. So far your responses to the guiding questions are a good beginning description. You mention that the assessment is fair because the students are able to "explain[ing] their ideas and concepts".

    Could you please expand on what you mean by this. I am not sure how you believe that the assessment meets the values of assessment - fair, valid, reliable, usable and clear etc. (p. 3 Course Outline assessment instructions).

    I can see why the assessment is directly relevant to the course topic. However, more information about how students are supported during the exploration phase - are they given feedback on this phase of the work? Do they have opportunities to get peer feedback? After the presentation - why is the feedback that you give going to help learning or not? More information about what is working in the assessment and what you see as gaps - how this assessment helps learning or not - would be helpful.

    Also, it would be good to see an example of why you believe that the assessment instructions do not align with the learning outcomes. Have you had feedback from students that they are confused by the assessment instructions or do their outputs give you this information?

    it would be good to know if what is intended in this assessment actually works. At the moment I do not get any sense of this. For example, do the students actually do any exploration of packages, and is this learning reflected in the design of their packages that they present?

    Would their learning be enhanced if they had to evaluate the packages they explore more formally? Perhaps they could prepare a reflective summary of what is good or not so good for the intended purpose of the package - attracting business and customers.

    In your evaluation, you also need to come up with some recommendations for what needs to be changed and why...otherwise you wont be able to re-design the assessment.

    ReplyDelete
  2. Helen
    Your adjustments now give a clearer indication of the gaps in the assessment process. Well done.

    ReplyDelete