Assignment 1: Identify and articulate an evaluation scenario

By Evan Houghton Evaluation Scenario

A workplace intervention I have identiﬁed to conduct an evaluation plan for is the implementation of a computer programming learning initiative into my VET Engineering class. Which focuses on the use of technology and software for data display, visual design and robotics. The vocational education and training sector has a key responsibility to provide valuable skills and knowledge that employers value (Insources, 2024). By implementing this innovative learning initiative and evaluation plan, more relevant industry practices and technologies can be implemented into my VET Engineering lass.

My workplace context is a full-time, vocational education and training (VET) teacher, working in the government high school sector. With over one hundred year ten and eleven students enrolled in the Certiﬁcate II Engineering Studies. The school is situated in a low socio-economic area, located in the Northwest region of Victoria. According to the Australian Bureau of Statistics (2021) website, technicians and trade workers have the highest occupation rate for this area.

When evaluating a program, it is common practice to judge its effectiveness against the learning achieved (Caley et al., 2021). But just as important, Caley and associates article claims that the workplace context and systems related will inﬂuence the outcomes of the intervention (Stufflebeam, 2001, as cited in Caley et al., 2021). Thus, for my intervention, I aim to evaluate the learning initiative or program against what works, in what conditions and for whom, in respect to what extent and why. Aligning my evaluation plan with Pawson and Tilley's realisticevaluation framework or model (Pawson and Tilley, 1997, as cited in Caley et al., 2021). In support of this aim, as an educator, we have the responsibility to provide information about learning to the students, administrators and the stakeholders which we serve (Lilian, 2020).

The stakeholders involved in the evaluation plan will include:

• 1st Year students enrolled into the VET engineering program

• Parents of students enrolled into the VET engineering program

• VET Course provided - RTO’s / Readcloud - AIET

• Hume Central Secondary College’s VET leadership team

• Local engineering ﬁrms

• Program Sponsors

The main purpose of the evaluation is to collect information if the program goals were achieved, resources were utilised efficiently and to inform future planning (Lilian, 2020). Also allowing an opportunity to identify implications or constraints on what has not worked, described by Perrin (2002, p. 10) as “to help avoid reinventing the square wheel”.

Kirkpatrick’s four-level evaluation model which consists of satisfaction, learning, behavior and impact is a useful framework and organisational starting point. But, according to Caley and associates (2021) lacks guidance on how to address the nuances of organisational context. This is why I’m choosing to combine Kirkpatrick’s model with the realistic evaluation model.

The realistic evaluation model has a standard set of concepts which Pawson and Tilley's (1997) article describe as embeddedness, mechanisms contexts and outcomes. Which focuses on the human action within a range of social processes and reality (Pawson & Tilley's, 1997). By combining the two together a more thorough evaluation approach can be analysed.

References:

Australian Bureau of Statistics. (2021). Broadmeadows Census Data.

https://abs.gov.au/census/ﬁnd-census-data/quickstats/2021/210051242

Caley, L., Williams, S. J., Spernaes, I., Thomas, D., Behrens, D., & Willson, A. (2021).

Frameworks for evaluating education programmes and work related learning: a scoping review. The

Journal of Workplace Learning, 33(6), 486–501. https://doi.org/10.1108/JWL- 09-2020-0157

Insources. (2024). Quality education in the VET sector. https://insources.com.au/quality-

education-in-the-vet-sector/

Lilian H. Hill. (2020). Assessment and evaluation in adult and continuning education. In T. Rocco,

T. S. Rocco, M. C. Smith, R. C. Mizzi, L. R. Merriweather, & J. D. Hawley (Eds.), The Handbook of

Adult and Continuing Education (2020 edition.; First edition., pp. 140–149). Stylus Publishing.

https://ebookcentral.proquest.com/lib/uts/reader.action?docID=6373351&ppg=152

Pawson, R. and Tilley, N. (1997), Realistic Evaluation, Sage Publications Ltd. London. Perrin, B.

(2002). How to - and how not to - evaluate innovation. Evaluation, 8(1), 13–28.

https://doi.org/10.1177/1358902002008001514

Stufflebeam, D. (2001), “The metaevaluation imperative”, American Journal of Evaluation, Vol.

22 No. 2, pp. 46-62.