• ::info
• ::profile
• ::test
• accreditation
• ::project evolutions

Certification and Accreditation
of the European Standard
of Forestry Workers

• Introduction
• The Leonardo project Learn for Work
• • General European Function Profile
  • Test Instruments
• Certification of the European Standard of Forestry Workers
• • Function profile
  • Test Instruments
  • An established and warranted testing procedure
  • A reliable organisation to judge the test results and give certificates
  • A certificate with a validity of a limited time period
  • A common accessible and updated register of certified forestry workers
• Accreditation of the Certification System
• • Levels of Accreditation
  • ISO certification and accreditation
  • Cito certification and accreditation
• Implementation of a Certification an Accreditation System
• • Discussion and Decision Structure
  • Important Documents
  • Test instruments
  • Training of Assessors
  • Registration
  • Supervision and Audits
• Addendum A: Complete test grid

Introduction

Both at national and international level, purchasers like forestry companies require guarantees about the quality of services supplied to them by forestry workers, since the variety on offer is nowadays huge, especially when forestry workers of other countries are involved.

A European certificate of Forestry Workers can then serve as an independent description of the ability of a forestry worker to conduct the general elementary forestry activities, like for instance felling a tree with a power saw in a safe and regulated way. A forestry worker can do his or her job not only in his own country, but also in the other countries where the certificate is known and recognized as an important and valuable description of his or her ability. Such a certificate can even become obligatory for forestry workers who want to work at forestry companies in other countries.

In this proposal the first chapter describes the Leonardo-project Learn for Work, in which an attempt was made to develop a European function profile of a forestry worker. Based on this function profile a set of tests was constructed and empirically researched in several countries involved in the project. The last phase of the Leonardo-project was to describe a certification an accreditation system to establish, maintain and guarantee an objective and valid evaluation system to assess the European Standard of Forestry Workers. This report is the result of the last phase.

The second chapter describes a certification system in general and possible choices, that can be made when implementing such a certification system to maintain a European Standard of the ability of Forestry Workers, which is valid in all countries of Europe.

An important condition to be able to assess, guarantee and maintain a valid certification system is an accreditation of this certification system by an expert and independent institution. Different levels of accreditation and methods to validate these levels are described. Cito in The Netherlands is an organization which is able to conduct an expert and independent accreditation.

The last chapter of this proposal treats the implementation of the certification an accreditation system. Possible implementation methods are described and advantages and disadvantages of the possible choices are discussed.

The Leonardo project Learn for Work

The aim of the Leonardo-project Learn for Work was to develop a general European standard of a function profile of the forestry worker. The activities of this project are planned during the three years of 2004, 2005 and 2006.

Two or more members of institutes of Forestry Work in six different European countries are involved:

The National Institute of Test Development in the Netherlands, called Cito, was asked to advise and support the project activities.

General European Function Profile

The first phase considered the research, analysis and development of a European function profile of the most important activities of forestry workers. This function profile is based on the descriptions of the activities of forestry workers in the six countries Austria, Belgium, Finland, France, Germany and the Czech Republic. The general common activities in these six countries are analyzed and described in a table.

Then all activities were formulated as relevant aspects in a questionnaire. In the six countries some 50 forestry workers and ten privileged people rated all activities as very important, important or not important at all. So an ordered list was made of the most important activities of forestry workers.

This list can be regarded as an operationalisation of the activities of forestry workers in Europe to be examined to assess the important general abilities of forestry workers. The list was considered as a general description of the most important content of the test instruments. In a so-called ‘test grid’ every important topic was described, the test method was determined and was given a ranking of importance. The more important a topic, the more questions or scoring points must be given to this topic in the test instruments.

Test instruments

Based on this ‘test grid’ a practical test and three different versions of a paper based test were developed. To evaluate the performance level of a forestry worker candidates must pass a theoretical paper based examination and a practical examination.

The paper based examination consists op 60 multiple choice items and two open answer-items. The content of the questions is based on the test grid in the following page. For all content subjects listed in the first column in the table is determined whether knowledge (K), Skills (S) or Attitude (A) are involved. The a relative percentage was determined based on the relative importance of the subject in the test. The consequence of this percentage was translated in a number of multiple choice items and open questions in the test. The complete test grid is in Addendum A.

Two versions are initially tested in a limited field test in five countries. Unfortunately France was not involved in this field test.

The third test version is offered as an example test on the website of the Leonardo Project Learn for Work. Forestry workers can use this test as an exercise to test their basic knowledge and to get a picture of the items asked and the content the items are concerned with.

The practical examination consists of two parts:

1. a small task involving the maintenance of a chain saw or a brush cutter, like assembling a chain saw or brush cutter, sharpening the chain or the saw blade, or adjusting the carburettor,
2. a complex practical task in the forest. This task is designed to examine forest workers’ knowledge and capabilities at meeting forest production work requirements in using the power saw in regeneration operations. Also the forestry worker must do two small additional tasks in a site nearby:
   1. A candidate must mention the common and Latin name of a tree and four different species (animals or vegetation), and
   2. If possible, a candidate must select trees for selective thinning in a small area of five to six future trees.

The practical examination lasts about 120 minutes; the small maintenance task about 15 minutes plus 15 minutes preparation and evaluation, and the complex task in the forest lasts about 90 minutes in total.

Certification of the European Standard of Forestry Workers

To develop a standardized examination system of a European standard of Forestry Workers this chapters describes a possible examination system as a proposal to discuss about in the six different countries. When agreement is reached on the different aspects of this examination system, the examination system can be developed, tested and implemented in the different countries all over Europe.

By the way of person certification an employee or (commercial) enterprise can show objectively that he or she is competent to execute a certain function on a certain specified level. The characteristic of certification is that independent organizations and experts warrant that the necessary company demands really are tested. In some cases the general government offers these guarantees. In other cases (international) professional organizations can define these standardized demands. Then a register of all certified persons can be developed, which is maintained by a certification or an accreditation organization. The essence of person certification is that based on clear and explicit formulated demands of a function profile persons are tested in an objective and thoroughly elaborated way and that the certification only has a limited validity of for instance a period of three to five years.

For the certification of persons the following elements are important:

• A common and clearly formulated function profile
• An objective set of test instruments
• An established and warranted testing procedure
• A reliable organisation to judge the test results and give certificates
• A certificate with a validity of a limited time period
• A common accessible and updated register of certified forestry workers.

These elements will be discussed in this chapter.

Function profile

In the Leonard- project Learn for Work a function profile is formulated and commented by forestry workers and privileged people in the six European countries involved. This function profile can be considered as a description of a European standard of forestry work. The function profile can be used as a common framework of a certification system of forestry workers in Europe.

Test instruments

In the Leonard- project Learn for Work also a set of test instruments is developed and validated in a very limited field test.. These instruments can be considered as examples of test instruments. They need further research and analysis. As a consequence of the research results more exam versions and other test instruments need to be developed.

An established and warranted testing procedure

A testing procedure must be elaborated, discussed with the important representative persons from the different European countries involved and be desctribed in a clear and explicit way. This Standard Testing Procedure can be tested, adjusted and be approved by all people involved. An important issue is always the clear identification of the candidates.

After the first development an accreditation organization checks and warrants this testing procedure. This can be considered as the second phase of the process. Finally the accreditation organization tests the execution of the procedure by checking a sample of test administrations on a regular base.

A reliable organisation to judge the test results and give certificates

Not only the testing procedure must be considered, but also the process of scoring the several test results, judging the scores into ratings and finally distributing the unique and individual certificates. This can be considered the last steps of the testing procedure ad also has to be developed, discussed and agreed upon, tested and adjusted and approved by all organizations involved.

The accreditation organization will also consider the procedure of scoring, judging the scores and the distribution of the right and unique certificates. After checking, adjusting and warranting, this procedure will also be checked at a regular basis by samples in the different countries.

A certificate with a validity of a limited time period

All countries involved must discuss the conditions, the content and the validity period of the certificates. The validity period determines the frequency of obligatory examinations. This frequency can function as a way to guarantee the commercial enterprises that a certified forestry worker every five years for instance must learn the latest methods of harvesting. The frequency can therefore be used as a way to enforce forestry workers to keep up with the latest technical developments in their field of forestry work.

In combination with the regulations about the certificates an examination document must be written, with all examination prescriptions.

A common accessible and updated register of certified forestry workers.

An important part of a certification system is a method of verification of all certificates by establishing a general accessible list of certified forestry workers, per country. So every enterprise and every individual involved can check for themselves whether a certain forestry worker is certified and if so, when the validity period of his or her certificate is expired.

Accreditation of the Certification System

The last chapter describes the important elements of a certification system. To test and monitor these certification processes an independent and expert organization has to accreditate the certification processes on different levels. This chapter will elaborate the possible accreditation levels.

Levels of Accreditation

In general five different accreditation levels can be discriminated:

1. The Function Profile; the general description of the important knowledge, skills and attitudes to be assessed during the certification process.
2. The Certification Instruments; the tests and practical exercises must be researched in an empirical way in a field test and must be validated. The final goal is to develop an itembank. An itembank is a database with items of forestry work, on which test instruments can be composed which are empirically justified.
3. The Assessors involved; the assessment processes of the practical exams must take place in an accurately way, that is objectively neutral, reliable and validated. The development of complete and extensive descriptions of the assessment processes of the practical exams and a thorough training of the assessors can enhance the assessment processes.
4. The Management Level of the Certification System; by executing audits of representative samples of the certification procedures of the paper based tests and the practical exams on a regular base, non-conformities (or irregularities) and cheating can be discovered and rectified timely.
5. Finally the Organization Level of the Certification System; the transparency of the certification system can be enhanced by a central and complete recording of all data of candidates, assessors, testing situations, testing dates and places, relevant documents, audits conducted and certificates. Parts of these data have to be accessible for all persons involved in the process. Training and examination centres for instance can check the examination data, places and times. Commercial enterprises can check the certificates of (future) employees.

ISO certification and accreditation

A well known method of certification is called the ISO-certification. The ISO-certification is mainly focussed on the management and organization level of the processes. All paperwork must be accurate, all processes must be described in so-called manuals and every complaint and action must be recorded in a transparent way. The ISO-system is therefore mainly focussed on the fourth and fifth accreditation level.

Cito certification and accreditation

Cito is the dutch National Institute of Test Development in The Netherlands. Cito is an indepent organization, founded in 1968. It belongs to the top-3 institutes of test development of the world.

Cito develops all national examinations of the secondary education and supports the schools of primary education by providing diagnostic and final tests for the admission to the schools of secondary education. Also (parts of) examination systems for vocational education and for commercial enterprises are developed, tested and implemented. All these activities are supported by the large Research and Development Department of Cito.

Cito can conduct an accreditation of the certification system of the European Standard of Forestry Work. The advantage of Cito that all five accreditation levels can be checked, adjusted and warranted, not only the last two levels. So Cito can support the further development and implementation of the function profile. The construction and testing of the certification instruments is the core business of Cito. Cito can conduct and support the second accreditation level by all means very well. Also the training of assessors is a standardized training module of Cito Vocational Education, so our experience in this field will prove to be very valuable.

Implementation of a Certification an Accreditation System

To be able to implement the certification and accreditation system of the European Standard of Forestry Workers, all components of the last two chapters must be described in detail, must be communicated and discussed in the relevant national organisations of forestry education and regulations and finally be warranted by a group of national and European representatives.

Discussion and Decision Structure

An important implementation aspect is the discussion and decision structure chosen. A lot of European countries will be involved in the certification system en it will be of great importance to give all countries possibilities to discuss and decide about the relevant aspects of the certification and accreditation system. All partners and countries can make their comments and annotations. So the approval and cooperation of all countries and organizations involved is optimized.

To determine an appropriate discussion and decision structure it can be necessary to make a national analysis of the vocational education system of forestry work and of the relevant players in the society of each country involved. For instance in one country the government can play an important role by regulating the certifications system of vocational education. In another country on the contrary the unions or commercial enterprises highly influence the certification of vocational education. This analysis can determine the important national factors in the countries.

A discussion and decision structure can be oriented nationally. All countries must then indicate one or two large and common institutes as the most important deciding factor in the country. This indication can be based on the afore mentioned analysis. This institute seeks partners in it’s own country. Possible partners can be commercial enterprises, education centres or governmental departments for instance. So for every country a large committee will be established to discuss about all relevant documents and give recommendations for the important decisions to be made in the international committee.

Representatives of these large and common institutes in al countries form the international Committee of the European Standard of Forestry Work.

Important Documents

Also all necessary documents must be developed and elaborated in further detail, for instance:

• The description of the function profile.
• The description of the testing procedure, included the scoring of the test results and the decision process to assess whether a candidate passes or fails the prescribed ability level of forestry work.
• The description of the testing instruments.
• The description of the testing organisation, including the way a candidate can register, can pay, is invited to the different forms of exams, and gets his or her results.
• The description of the examination regulations and the availability and access of all persons involved in the certification system.
• The description of the certificate: the lay-out, the examination data and the unique personal data on the certificate, the length of the validity period, etcetera.

All these documents must be discussed in all countries involved. Final decisions can then be made about the relevant topics in the documents both at a national level in all countries and at an international level in the international Committee of the European Standard of Forestry Work.

To enhance the final approval of the documents by the selected accreditation institute or organization, it is recommended that the accreditation organization is consulted in an early stage of the process and that the recommendations of this accreditation organization are processed in the important documents.

Test instruments

Although the test instruments were tested in a limited way in five different countries, the amount of data in this field test is too limited to be able to draw statistically sound conclusions. Therefore it is recommended that the two paper based test versions and the practical test will be researched more thoroughly in a larger field test with al larger group of candidates in all countries involved.

Per country each paper based test version must then be taken by at least 150 to 250 candidates to analyze the tests in general (for instance the reliability of the total test, the average score, the standard deviation) and relevant parameters per item (like the p- and a-values per alternative of a multiple choice item and the correlations with the total test score).

The practical test must be taken by at least 100 candidates per country, observed and scored by at least two independent assessors per practical test. So the objective scoring of the practical test can be analyzed by calculating for instance the average scores and the intra- and inter observer reliability.

Final goal is to develop an itembank of approximately 300 items for the paper based test versions and several formats of the practical tests, so a large variety of test versions can be established. Candidates then can get a different, but parallel test version in each country. Additional is the translation of the items in the languages of all countries involved.

Training of Assessors

All assessors must be trained to maintain the testing conditions of both the paper based tests and the practical test similar all over the countries and circumstances. Then the test results of one country can be compared with the test results of the other countries.

The initial training of assessors can be done on an general and international level. For instance two or tree assessors of each country involved in the European Standard of forestry Work can be selected to get their elementary training. They then can also discuss the necessary adjustments on a national level, for instance because safety regulation procedures differ per country.

These trained assessors must then train the other assessors in their own country. It can be recommended that to establish objective assessments in all countries, the first national training meetings must be visited or supported by members of the international Committee of the European Standard of Forestry Work or by auditors of the international accreditation organization, including translators.

Registration

To enhance the transparency of the certifications system a common accessible and updated database with all relevant data of certified forestry workers must be developed. This database must not only contain data about the certified forestry workers, but also about test data and places, data of the certified assessors involved in the certification system, data of education centres and examination possibilities and all other important documents and facts about the certification system.

An important part of a certification system is a method of verification of all certificates by establishing a general accessible list of certified forestry workers, per country. So every enterprise and every individual involved can check for themselves whether a certain forestry worker is certified and if so, when the validity period of his or her certificate is expired.

Supervision and Audits

Finally the certification system must be supervised in a clear and acceptable way for all partner countries involved. So the way of supervision, for instance the inspection and review of all documents involved in the certification system and the sampling of test situations to audit, must be agreed upon and described in detail in a separate document.

Then the accreditation organization can execute these supervision agreements or have these supervision acts executed by partners in all countries. At first the goal of the supervision will be to get an accreditation of the certification system, in later years the goal of accreditation is to continue the accreditation of the certification system.

Addendum A: Complete test grid