Public defense: Roger Bakken

My thesis, Holistic Assessment in a Vocational Didactic Perspective (HELVYRD), is a study of assessors’ holistic assessment of apprentices in vocational education and training (VET) in Norway. Different forms of assessment are used throughout the education and training, where the final trade examination, after four years (two years school-based and two years work-based apprentice, normal, but with some deviation) is a summative assessment. 

HELVYRD is a part of the MECVET-project (Measuring Competence in Vocational Education and Training), supported by the NFR (Norwegian Research Council), which has tried out large-scale assessment using the German COMET-model (Kompetenzdiagnostik im Berufsfeld Elektrotechnik) adapted to the Norwegian context. 

The HELVYRD study is conducting research on assessors’ holistic assessment in electrician where three different assessment systems are used, the MECVET-model and two Norwegian systems: centrally given written examinations and trade examinations. 

The study seeks to find out how the assessors within the three different assessment systems can assess the holistic and complex vocational competence of a skilled electrician.

In my theoretical review, I have developed my own definition of holistic assessment:
Holistic assessment is to incorporate various aspects of work activity related to a professional practice in a specific context, where the assessor uses his/her experience, expertise and judgment to assess, as much as possible, the professional competence against the requirements for the profession.

The three research questions in this study are:
- How can we understand holistic assessment in the perspective of vocational didactics?
This question focuses on an area which, in the thesis, is the subject of theoretical discussion and which provides a conceptual framework for analysing the empirical material.

- To what degree do the assessors conduct holistic assessments in the selected assessments systems? 
This question is answered by describing the incidence of holistic assessment (descriptive) in the three assessment systems.

- What factors affect the assessors’ holistic assessment of the last years’ apprentices’ answer papers?
How do the assessment systems affect the assessors’ holistic assessment (explanatory)? 
Of the various factors being investigated, the assessments systems will be central, and thus the vocational didactical principles on which these systems are based. 

The theoretical framework has two main areas: vocational didactics and holistic assessment. The discussion about vocational didactics is based on both the international (Nordic and German) and the national research. 

This will be summarized in six perspectives for vocational didactics using the six assumptions developed by Gessler and Herrera (2015). It has not been possible to limit the theoretical framework for holistic assessment to one specific theoretical view, but emphasis has been on competence-based assessment (Wolf, 1995). 

The literature used has been systematized and processed with support in the didactic relationship model.

The study’s research design and methods are developed on qualitative interviews conducted with thirteen assessors distributed within the three assessment systems and supported by the thinking of both “stimulated-recall” (Calderhead, 1981; Lyle, 2003) and “think-aloud” (Patton, 2015; Kuusela and Paul, 2000). 

Document analysis is a minor part of the data collection. The theoretical views of Charmaz’s (2014), Strauss & Corbin’ (1998) have supported the coding and categorisation of the empirical data. The results and discussions are presented in the respective categories. A further discussion have been summarized in four main dimensions.  

The presentation of the study's central findings related to the three research questions:
In the study’s findings on the understanding of holistic assessment in a vocational didactical perspective, it became evident that little has been written on this subject. Through my development of a theoretical framework on holistic assessment, different perspectives emerged, and defined further by the vocational didactical framework. 

This is about highlighting a variant of VET assessment, where the assessor sees different aspects of a complex professional practice in a particular context by applying his experience, competence and judgment and assessing theses against the requirements for the profession.

Holistic assessment incorporates a view of assessment as one that promotes seeing an occupation as a professional whole.  By highlighting holistic assessment in selected vocational didactical perspectives, opportunities, challenges and limitations are revealed. In particular, a vocational didactical affiliation and influence from working life will be central. 

The constant development in the trades is evident and affects singularly both the field of assessment and the field of vocational didactics, but also how these may be seen in context. This again affects the understanding of what holistic assessment comprises. The confidence in the assessors who perform the assessments will be of significant importance here.

Findings show that the degree of holistic assessment amongst the assessors is high. This can be explained by the assessors’ approach to the electricians’ curriculum content, the incorporation of their own professional background, and how they applied assessment criteria in the assessment context. 

What promotes the holistic assessment is the ability to use the professional work-life experience in the assessment. The findings show that being part of an assessment network is central. In such a network, assessors can be informed, developed and updated. These aspects are important in electrical work, which is under constant development.

Findings also indicate that the form of the assessment system as well as current development in the electrical trade primarily affect the assessors’ holistic assessment. Assessment systems affect the assessors by the design of the tasks, the structure of the assessment process and the given frameworks. 

The differences between the assessment systems are considerable and affects the assessors to different degrees. Findings show that the tighter and less flexible the system, the more it reduces the ability of the assessors to conduct a holistic assessment. The assessor’s involvement and ownership of the assessment process is reduced and results largely in standardization and automation, where significant aspects of assessing holistic vocational competence will be overlooked. 

The assessment systems must adapt to the rapid development of the electricians' profession and give the assessors room to assess relevant professional competence. Central findings show that use of case-based tasks such as a work order, and the assessors’ being given the opportunity to participate in assessment networks, contribute to the development and use of assessment criteria, have a positive impact on being able to make a holistic assessment.

In conclusion, the study indicates that there is a need for further research in several areas. These include assessment in other VET-programmes: different forms of assessment systems: development and application of goals and criteria: the importance of writing: and not least the development of vocational content.