ESIB Statement on a Peer Review

General Framework for introducing a system of peer review of QA and A agencies

As part of the realisation of the European Higher Education Area the Ministers of Education held a
Summit in Berlin in September 2003. In the resulting Communiqué the Ministers called ENQA in
association with ESIB, EUA and EURASHE to fulfill certain mandates. One of the mandates called
for the development of a general framework for introducing a system of peer review of QA and A
agencies. The four organisations (ENQA, ESIB, EUA and EURASHE, hereafter called E4) have
been meeting on a half-yearly basis for the past two years and this group was considered the best
forum for realising the mandate from the Ministers. In order to facilitate this ESIB has produced a
number of papers based on our previous policy papers and would like to elaborate more specifically
on these in this statement.

Higher education is based upon the principle of maintaining and improving the quality of provision.
This system is based upon each institution developing a quality culture and taking a proactive
approach to quality assurance at the programme and institutional level. The reliability and scrutiny
of these operations is dependent on an independent and transparent system of quality assurance
which is usually fulfilled by national quality assurance agencies. When developing a European
Higher Education Area it is essential to ensure that there is transparency and trust between national
quality assurance systems. The Ministers in agreeing the Berlin Communiqué set out a framework
to achieve this through a system based upon peer review.

At the European level, there already exists a network of qualification agencies, namely ENQA. This
is useful since it already provides a framework in which peer review can operate and as such may
form the basis of any future system of peer review. However it should be recognised that any peer
review system requires wider involvement than just national quality assurance agencies. Such a
system should ensure both ownership by all stakeholders and an external perception of independent
scrutiny.

The aim of peer review is to ensure that all QA and A agencies carry out effective quality assurance
at the national level that is trusted and as such the review of these QA and A agencies should ensure
that they meet a set of criteria. When carrying out a review of a QA and A agency it is important
that the review is seen as independent and is carried out by those best placed to assess the work of
the agency.

ESIB believes that this peer review or “study visit” should consist of the main stakeholders in
higher education, namely, representatives from higher education institutions, quality assurance
agency representatives and students. Since this is being done on the European level, we would
recommend that higher education representatives are nominated by either and/or EUA and
EURASHE, quality assurance agency representatives nominated through ENQA and students
nominated by ESIB. Experts from outside Europe may also be considered for the panel. To ensure
effective and perceived independence, these representatives should not come from the same country
as the agency being assessed, although due account must be taken of language issues to ensure
effective translation and interpretation on documents and meetings to ensure the accuracy of the
study visit.

When the report from the study visit is carried out it is important that it is dealt with in a way that is
both fair and seen to be fair and so should report to a body made up equally of the representatives
and stakeholders in higher education more widely, for the purposes of this paper we will call this
body the “Board”. This Board could possibly be within a wider framework that is made up
primarily of HEI representatives, QA representatives and student representatives and could also
include governmental representatives (e.g. a representative of the Bologna Follow-Up Group) and
employers and employees organisations as observers.

This Board should assess the study visit report and monitor the implementation of the
recommendations, decide on the need for follow-up visits and in most extreme cases decide on the
ongoing membership of the organisation. The agency assessed will of course be able to see and
comment on the report before it is presented to the Board. In addition to this a representative of the
national agency should be invited to the Board to allow an opportunity for comments and questions.

Once the report has been approved by the Board it will be made public and whilst there may be
some initial fears about this publication it will help engender a spirit of trust and transparency both
within the higher education community and in the wider public where issues such as accountability
of public funding and trust of quality in education are key issues. It will also be a useful tool for
those agencies that have not yet been assessed to outline both what is expected and as a way of
sharing best practice. These reports have the primary role of ensuring high quality education across
Europe and so should look at ways of supporting the member to continually improve, and
developing partnerships to support this work. The removal of membership of the European level
organisation should only be used as the very final stage in the process once all other methods for
improvement have been exhausted.

The reports should be based on evaluating whether the QA and/or A agency meeting the
membership criteria for European level organisation which should be strengthened to ensure that all
members are seen as legitimate bodies to carry out their function.

In practice how the peer review could be carried out:

A national quality assurance agency from Europe has applied to the Board to be recognised as a
quality assurance agency at the European level and receive the so-called “ European Label for
Quality Assurance and Accreditation Agencies” to operate within the European Higher Education
Area. The agency has submitted its statutes in both the native language and in English as well as a
letter outlining why they would like to be recognised. The Board having decided that they broadly
meet the criteria, agree to investigate their application at the next stage of the application process
and set up a study visit of that agency.

The study visit is made up of a representative from ENQA (nominated by their steering group and
taking into account gender representativity), a representative from ESIB (nominated by their
Executive Committee), a representative from EUA (nominated by their Board), a representative
from EURASHE (nominated by their Board) and also a quality assurance expert from outside
Europe chosen by the Board.

This five person study visit, not including anyone from the applicant country, but assisted by a
translator as necessary, would then meet amongst themselves to discuss the papers presented by the
agency and decide on their plan of work. They would then visit the country for probably a three-day
period and have several meetings with the agency and also other stakeholders including the national
students union, rectors conference, Ministry and other bodies to ensure that the criteria are met.

When they are sure that they have enough information the study visit will meet again to create their
report based on their findings and also their recommendation as to whether they should accept the
agency or not. This report is then sent to the agency for discussion and any proposed amendments
are discussed with the study visit team. When the study visit team is happy that the report is a true
representation of the situation they will submit the report to the Board where they will make a
presentation of the recommendations. A representative from the agency will also be invited to
answer questions from the Board. After a period of discussion, part of which can be held in closed
session from the applicant, they will make their decision. The decision and also the
recommendations of the report from the study visit will then be made public to interested parties
including the press.

This process could also be used to decide membership of the European level QA&A grouping. The
review process of agencies must be a cyclical process, held approximately every five years
Any National agency unsatisfied with the result of the decision of the Board would then be able to
lodge an appeal with the organisation or group with responsibility for overseeing the
implementation of the Bologna Objectives or through a system determined by that group. This
recognises the fact that the legitimacy of the nature and instruments of the European Higher
Education Area stems from the Conference of Ministers and the Follow-Up Group. +A successful
appeal would result in the decision being referred back the Board of ENQA for re-consideration.

Criteria on which the agency is assessed:

· An agency must have a clear legal basis, its own budget and professional staff with
resources proportional to assigned tasks
· The agency must be a active in, at least, one of the countries within the Bologna Process
· Review results must in all cases reflect the findings of the reviews and not be subject to any
pressure from other sources
· Terms of procedures (inc the terms of references and identification of experts) of
evaluations must be defined by evaluation professionals of quality assurance agencies, in
consultation with representatives of students and HEIs not by other bodies
· The section on internal quality assurance mechanisms of the agencies must also include an
indication of no conflict-of-interest mechanisms in the selection of external experts
· Reports by the national agency must be made public.
· The process of review undertaken by the national agency should be cyclical.
Also the agency must have:
· Internal feedback mechanisms that includes procedures for reflections and subsequently
revisions on processes and methods
· Mechanisms that provide feedback from external expert panels
· Mechanisms that ensure dialogue and feedback from stakeholders. The latter may be
institutions or programmes that have been evaluated, stakeholders in general such as higher
education institutions and should be students and student organisations, while bearing in
mind representativity.
· Present on the agency website a public (i.e. non confidential) quality policy, including
agency goals, processes and methods and a description of their decision making processes
and governance structures to enable transparency and accountability.