Guidelines for Programme Design, Monitoring and Evaluation
Table of contents KEY TERMS 1. BASIC PRINCIPLES 1.1. Development cooperation as part of foreign policy • • • • • 1.1.1 Increasing coordination and coherence 1.1.2 Strategic planning sets the framework 1.1.3 Cooperation has various forms 1.1.4 Policies require action 1.1.5 Finland as a partner

1.2. Towards a common language • • • 1.2.1 An integrated approach improves learning 1.2.2 Project cycle - the life of a development intervention 1.2.3 Level of participation varies

1. 3. Achieving sustainable development • • • • • • • • 1.3.1 Policies must match 1.3.2 Better value for money 1.3.3 Institutional capacity makes a difference 1.3.4 People-centered development emphasises socio-cultural aspects 1.3.5 Participation enhances ownership 1.3.6 Gender equality and participatory development 1.3.7 Environment - not only ecology 1.3.8 Technology must meet the needs

2. PROJECT DESIGN 2. Situation analysis - the cornerstone of project planning • • • • 2.1.1 Background studies and the analysis of stakeholders 2.1.2 Problem analysis - key to the project’s framework 2.1.3 Objectives reflect an ideal future 2.1.4 Strategic choices begin by fixing the project purpose

2.2. Planning with logic • • • • • • • • 2.2.1 Logical framework is a practical tool 2.2.2 Intervention logic states the strategy 2.2.3 Assumptions must hold 2.2.4 Indicators make the plan concrete 2.2.5 Approach describes how 2.2.6 Organisation determines roles and responsibilities 2.2.7 Budget details financial framework 2.2.8 Various roles of the project document

3. MONITORING • 3.1. How stakeholders monitor

• • • • • •

3.2. Integrated approach facilitates monitoring 3.3. What is monitored and how 3.3.1 Progress reports 3.3.2 Annual Monitoring Reports 3.3.3 Other performance monitoring 3.3.4 Financial reports

4. EVALUATION 4.1. What evaluation is • • • 4.1.1 Evaluation has two main purposes 4.1.2 Different types of evaluations and other related assessments 4.1.3 Integrated approach and the Logical Framework

4.2. Issues to be evaluated • • 4.2.1 General evaluation issues and their relation to the logical framework 4.2.2 Factors ensuring compatibility and sustainability

4.3. How to plan and carry out an evaluation • • • • • 4.3.1 Terms of Reference 4.3.2 Planning of evaluation requires expertise 4.3.3 Participation improves quality 4.3.4 Demand for local evaluation capacity is increasing 4.3.5 Evaluation report - the first step

4.4. What to do with the evaluation report • • 4.4.1 Dissemination of conclusions and recommendations 4.4.2 Approval and operationalisation of recommendations

ANNEXES • • • • • • • • • • • • Annex I Integrated approach to project cycle management Annex II General format for terms of reference Annex III Project document format Annex IV Intervention profile/performance summary form Annex V Example of project specific management/monitoring set-up Annex VI Example of work plan format Annex VII Reporting, monitoring and evaluation issues Annex VIII Example of an expenditure follow-up table and related graphics Annex IX Evaluointikäsitteet ja looginen suunnittelukehikko (in Finnish) Annex X General format for terms of reference for evaluations Annex XI Programme Completion Report Annex XII Programme/Project Fact Sheet

Comments and suggestions to: Ministry for Foreign Affairs Department for Development Policy Katajanokanlaituri 3 PL 176, 00161 Helsinki, Finland E-mail: keo-11@formin.fi

KEY TERMS Activities Assumptions Action taken or work performed within a project to transform means into results. Important conditions for the success of the project that are not within its control, and which are worded as positive conditions. A limited group among the stakeholders, who will directly or indirectly benefit from the project. Planning undertaken by expatriate or local officials/technical experts alone, resulting in a detailed implementation plan. Planning based on the collaboration of officials/technical experts and the representatives of the key stakeholders. All types of planned undertakings with developmental objectives. The performance standard to be reached to achieve an objective. Method of managing the project cycle, using the logical framework and the same sustainability criteria throughout the cycle. Strategy for achieving the project purpose, consisting of results, activities and means, and contributing to overall objectives. Method for analysing and presenting the most important elements of a project and their interrelationships. Human resources, materials and other inputs required to carry out planned activities and manage the project. Long-term development goals to which the project makes a contribution. A way of planning where the initiative and leadership is taken by the beneficiaries, and in which external facilitators participate. Condition that must be fulfilled before a project can start up. In these guidelines, different types of development interventions, which are designed to achieve certain specific objectives within a given budget and organization, and a specific period of time. Model of the entire lifespan of a project. The reason for or focus of the project, describing the improved situation which the project is expected to achieve. Products of the activities that together achieve the

Beneficiaries Blueprint planning Collaborative planning Development intervention Indicator Integrated project cycle management Intervention logic

Logical framework Means

Overall objectives Participatory planning Precondition Project

Project cycle Project purpose

Results

project purpose. Not only physical outputs but a start to enjoyment of sustainable benefits. Stakeholders Groups of people, organisations and institutions who have a direct or indirect interest, or a role, in the project, or who affect or are affected by it. In the context of a single project, the continuation of its benefits and impact after the project itself has ended.

Sustainability

1. BASIC PRINCIPLES 1.1. Development cooperation as part of foreign policy Development cooperation is an important part of Finnish foreign policy and external relations. To meet various objectives, Finland uses different multilateral and bilateral channels. Whilst project assistance still holds the dominant position in bilateral cooperation, both sectoral and program assistance are becoming more important. These guidelines are meant primarily for design, monitoring and evaluation of projects, but the basic principles are applicable in all types of cooperation. Reduction of poverty, protection of the environment, and promotion of equality, democracy and human rights are the principal goals of Finland’s development cooperation. They are an integral part of the planning, implementation, monitoring and evaluation of all development interventions. 1.1.1 Increasing coordination and coherence The strategy of Finland´s Development Cooperation in the 1990’s was endorsed by the Government and Parliament in 1993. The justification and motives are stated as follows: "Enormous discrepancies in wealth and well-being between the developed and the developing countries are not acceptable by any human standards and constitute a threat to the prosperity and security of all nations. Through its foreign and economic policies, Finland, together with other developed countries, is doing what it can to reduce and eliminate these discrepancies." As preconditions for cooperation, the strategy requires that Finland’s partner countries commit themselves to the reduction of poverty, to combat global threats to the environment and to promote social equality, democracy and human rights as essential elements of sustainable development. The Finnish Government’s 1996 Decision in Principle reconfirms these goals as the guidelines for Finland´s development cooperation. External assistance cannot be a substitute for a partner country’s own national resources. At its best, external assistance serves as a catalyst for development, to enhance conditions where governments, private sector, communities and individuals take the initiative and lead into their own hands. To strengthen coordination and coherence between development, trade and other policies, the Finnish Government aims for a comprehensive strategy for Finland’s relations with developing countries. Membership of the European Union brought new opportunities and challenges to Finnish development cooperation. This document, based on the project cycle management principles of the European Union, is an important step in the coordination of Finnish development cooperation with the EU practice. 1.1.2 Strategic planning sets the framework Long-term objectives of Finnish development cooperation are specified in the rolling five-year financial and action plan. Annual budgets approved by Parliament give further guidance and confirm financial commitment authorisations for coming years. Country-specific objectives are elaborated in negotiations with partner countries. When setting priorities, Finland emphasises the primary responsibility of the partner country. Identification of development projects is founded on different forms of communication between partner countries, and is based on mutual negotiations on different levels.

The Ministry for Foreign Affairs also prepares strategies for larger geographic areas. Sectoral and thematic guidelines set policies to that effect. Separate manuals deal with issues relevant to the sustainability. 1.1.3 Cooperation has various forms The different channels for development cooperation, such as multilateral organisations and development funds, bilateral cooperation (including non-governmental organisations), and humanitarian aid operations, serve different purposes and are employed in accordance with need. Bilateral development cooperation is channelled mainly through regional or sector programme support, or takes the form of development programmes and projects. Programme assistance is support provided for vital development efforts at national level, such as structural adjustment programmes. It includes both general and sector programme assistance. General programme assistance consists of all contributions made available to a partner country for general development purposes. It includes balance of payment support, general budget support and support for imports of specific commodities. General programme assistance is not linked to specific sector programme nor project activities. Debt relief and food aid are normally included in this category. Sector programme assistance is targeted at a particular sector and often linked to specific policy conditions. Some bilateral donors view this type of assistance as a set of interlinked projects aimed at a specific sector. It is based on a broad assessment of sector-specific needs. Recently, donor agencies have further elaborated the concept of a sector investment programme. The intention is to enhance the partner country’s ownership of programme activities and to strengthen its institutional capacity. Other key themes are the disbursement of funds through the partner country´s own budget, and closer interagency coordination. Within Finnish development cooperation, the term ‘sectoral approach’ represents a practical solution to the problem of translating political and economic dialogue into a long-term development programme. Dialogue between the partners determines the framework for cooperation on policy reform which will contribute to the desired sectoral performance. Sectors can be defined in economic terms, such as agriculture, industry and energy, or in cross-institutional (i.e. thematic) terms such as environment, health, education, gender, employment or local government. Adopting a sectoral approach is possible only in situations where the partner country has, or is, committed to develop an appropriate institutional framework and policies which promote the intended activities. In this context the institutional framework includes both organisations and the working environment in its broadest sense, i.e. including the "rules of the game". Organisations in society perform specific functions; a school educates children and a water authority provides safe drinking water and sanitation. This broad concept of the institutional framework includes laws, norms and values, as well as the structures which constrain the behaviour of individuals and groups. Employing a sectoral approach means working to strengthen local institutions and building up their capacity. This process is much more fundamental than one which is related only to organisations. Until now, professionals working in development cooperation have considered agriculture and rural development, catchment area re-forestation, water supply and sanitation, and hydropower energy development to constitute four different sectors or separate programmes. Coherent planning and management of aid means that these different "sectors" can be looked at jointly, since it may well be that they use and protect the same water sources. In this particular case, a wider approach provides the incentive for broader programming, analysis and the establishment of priorities for the whole of the water resources sector. Once a broad national and regional analysis has been carried out, a framework for project type assistance can be created. This, in turn, makes it possible for the purpose and role of each project to be bound more tightly to the general policy framework for the sector in question. Project assistance aims at achieving specific development objectives within a given budget and time. Project assistance normally includes financial, technical and material support.

Whilst project assistance continues to be the dominant form of Finnish development cooperation, programme and sectoral assistance are of increasing importance. The Ministry for Foreign Affairs is making efforts to further strengthen the role of programme and sectoral assistance. These guidelines use the term project to refer to all the foregoing types of development interventions. The planning, monitoring and evaluation methods described in this document are applicable to all the different types of intervention. From projects towards sector assistance Since 1980 Finland has given project assistance to agricultural and rural development in Luapula province in Zambia. From mid-1997 onwards the Luapula Livelihood and Food Security Program (LLFSP) is financed through the local Agricultural Sector Investment Program (ASIP). In the new concept, the Finnish funds are divided so that the development funds are paid to the ASIP account of the Ministry of Agriculture in Zambia and the technical assistance funds directly to the consultant in Finland. The funds remitted through ASIP are earmarked for LLFSP and conditions for their release include that Zambia pays her financing share of the LLFSP to ASIP first. Finland also reserves the right to employ an international auditing company to audit her financial input to the ASIP. 1.1.4 Policies require action Reduction of poverty, protection of the environment and the promotion of social equality, democracy and human rights are the principal objectives and preconditions in Finland´s development cooperation policy. These aims must be advanced and taken into account throughout the lifespan of each intervention, from identification through planning, implementation and monitoring to final evaluation. 1. Poverty reduction Sustainable economic growth, its fair and equitable distribution and translation into broad-based social development constitutes one part of Finnish poverty reduction policy. It emphasises stable and conducive economic policies and development of human resources which are both one pre-requisite and the outcome of economic growth. Empowerment of the poor, through improving their access to productive assets like land, water, financing and other services, is another important part. Poor people of both sexes must have the opportunity to participate in the decisions that affect their lives. It helps to focus on these root causes of poverty that poor people themselves consider to be most relevant. The analysis of a project’s impact on poverty must be based on systematic assessments which themselves are based on reliable indicators and which are conducted on a regular basis. Existing poverty assessments and profiles are complemented, when necessary, by additional studies. 2. Protection of the environment Assessing environmental impacts together with the various parties involved, and the resolution of possible disputes, are important aspects in both the strategic planning of development cooperation and in project preparation, implementation and evaluation. As part of, or in addition to, projects with direct environment-related objectives, environmental protection is enhanced by strengthening a partner country’s institutional capacity and by assisting in the fulfilment of a country’s international environmental obligations. The assessment of environmental impacts in Finnish bilateral development cooperation projects is discussed in greater detail in Chapter 1.3.7 of this document - "Environment - not only ecology". 3. Human rights, equality and democracy Human rights are indivisible, universal and interlinked. They form the basis for broad-based, stable and predictable development. Finland does not accept cultural relativism if it means violation of human rights.

Equality, both between different social groups and between the sexes, is an integral part of human rights and must be promoted in all Finnish development cooperation. There is a special emphasis on the status of people suffering from disabilities. Democracy is a prerequisite for broad-based participation in society and promotes ownership of development cooperation activities. It leads to a more stable, fair and equal society. The promotion of human rights, equality and democracy are cross-cutting objectives that must be considered when assessing all development interventions. They are objectives which are directly promoted by specific actions such as support to electoral processes, other democratic institutions , good governance and rule of law, free press and non-governmental organisations. Finland is active in seeking new forms of cooperation which support human rights, equality and democracy and encourages an open dialogue with partner countries, international organisations and NGOs. Developing with disabilities UN estimates indicate that there are as many as 300 million persons with disabilities in developing countries. In other words, of the 1.6 billion poor people, every sixth is estimated to suffer from a disability. To achieve effective reductions in levels of poverty, this large group of people must be given special consideration. The UN Standard rules on Equalisation of Opportunities for Persons with Disabilities challenge UN member states to establish, by law, the rights of persons with disabilities, including accession to health services, education and social services, as well as the right to equal participation in society. The needs and concerns of persons with disabilities must be included as an integral component in development cooperation projects. The first step is to put the rights of persons with disabilities on the agenda when negotiating with the cooperation partners on joint development plans. The inclusion of disability dimension is most urgent in the social, health, education and employment sectors. It is often necessary to plan for a specific disability component within a project to assure the inclusion. A health project, for example, could have a component focusing on rehabilitation and technical aids for people with disabilities. Earmarking of financial resources can be used to enhance levels of participation by disabled people. An agreed proportion of an educational programme budget could, for example, be used for training teachers and parents in disability issues. When considering construction and renovation projects, disabled people’s accessibility and mobility requirements can be integrated into the planning of technical works. People with disabilities are the best experts on their own needs. Disabled people should be listened to directly, either through their own organisations or through other ways. 1.1.5 Finland as a partner Finland is committed to the development cooperation strategy adopted by the OECD in 1996, which states that cooperation between industrial and developing countries is based on a partnership where the developing countries themselves assume responsibility for their own development. The Finnish Government´s Decision in Principle of 1996 states that development cooperation cannot be a substitute for domestic and private resources of funding which are essential to finance development in any country. Cooperation can act as a major catalyst in the disadvantaged countries, producing favourable conditions for private sector, individual and community initiative. As a partner in development cooperation, Finland assumes a secondary role while the developing country partner has the primary responsibility for the cooperation process, including project design, implementation and evaluation.

Further, Finland seeks greater coordination between international donors, through strengthening the capacity of partner countries to coordinate development actions financed by different partners. 1.2. Towards a common language These guidelines harmonise the terminology and methods in Finland’s development cooperation with EU practices. A common language makes it easier for partner countries to work with a variety of donors. It also improves aid coordination. These guidelines are based on integrated project cycle management. A logical framework is used as the tool to systematise the planning, implementation, monitoring and evaluation of development projects. A project cycle divides a project into phases. It helps to administer the project and provides check points for administrators to verify relevance, feasibility, participation etc. Different phases of the project cycle may be repeated or interrupted. 1.2.1 An integrated approach improves learning These guidelines are based on integrated project cycle management. This approach helps those in charge of project planning, implementation and evaluation to focus on the elements and factors considered most relevant to success of a project. The elements of an intervention are structured using the logical framework. Figure 1: Logical framework matrix Overall objectives Objectively verifiable indicators Objectively verifiable indicators Objectively verifiable indicators Means Sources of verification

Project purpose

Sources of verification

Assumptions

Results

Sources of verification

Assumptions

Activities

Costs

Assumptions

Pre-conditions

The logical framework systematises the setting of objectives and assumptions for a development project and their analysis. It highlights causes and effects between the key elements and provides a tool to present them. The left hand column of the logical framework portrays objectives of a hierarchy, linking them to the indicators, their verification and assumptions in the other columns. The integrated approach means that the logical framework is used and the same elements and factors are applied throughout the life of a project. The factors adopted in Finnish development cooperation are described in Chapter 1.3 of this document - "Achieving sustainable development". As a consequence of the integrated approach, the documents used in different phases of the project cycle have the same format as shown in Annex I.

1.2.2 Project cycle - the life of a development intervention The project cycle is a detailed model of the entire lifespan of a development intervention, starting with its identification, going through the implementation, monitoring and evaluation phases, and ending with the lessons learned. Although variations to the standard model are common, the project cycle is a backbone used by the various donors in development cooperation. The division of a development project into distinct phases helps to define the roles of various parties. It helps to make sure that the decisions are based on relevant and adequate information. The project cycle clarifies the role played by meetings and documentation and makes them more useful. Based on the information produced at each phase, a decision whether to continue or to change course is made. The assessment of policy and issues affecting sustainability plays a key role in the making of the decision. Some phases may have to be repeated, some phases may proceed according to a revised timetable. For example, a project may be re-appraised if there have been significant changes in the economic or political environment of the partner country during the planning phase. Planning or implementation can be interrupted at any point to allow time for specific studies, assessments or evaluations that will guide the direction of further work. Terms of reference are used to explain what should be done at the different points and why. A general format for terms of reference is presented in Annex II. The following model project cycle is used by Finland in its main partner countries in large, multi-year development projects. Figure 2: Model project cycle

1.2.3 Level of participation varies To enhance ownership and sustainability, Finland emphasises the responsibility of the partner country in the planning and implementation of development cooperation. Dividing a project into distinct phases assists in the identification of various stakeholders, and in the definition of their roles. The diagram on the next page illustrates different levels of stakeholder participation during the planning phases, from initial identification through project formulation and appraisal to preparation of the project document itself. "Participatory planning" indicates that the initiative and leadership are taken by the beneficiaries in their communities. "Blueprint planning" suggests an expert- or civil servant-driven approach. In many cases, "Collaborative planning" , where planning is a joint exercise, is used. The need for technical expertise, or grassroots participation, varies between projects and between different planning phases. A planning phase where only technical expertise is used without people’s participation must be complemented by collaborative or participatory methods in the following phases. For example, a project identified and formulated by technical officials and/or experts is appraised by the communities. Thus, within a single project, "participatory", "collaborative" and "blueprint" planning may be used in its different phases. The reports and project meetings indicated in the diagram provide check points at which the project plan is verified and adjusted, and at which the level of participation is determined for the next phase. Any planning phase may be repeated. The project document produced varies in content and level of detail. Whilst a "blueprint document" provides an accurate implementation plan, a "process document" lays emphasis on the development options and is not precise in relation to activities or means. A detailed plan is useful, for example, when constructing a hospital, whereas flexible process is suited to a participatory rural development project. Figure 3: Participation in planning phases

1.3. Achieving sustainable development In the context of a single development project, sustainability means that the development impact of a project continues once the project itself has ended. Assessment of sustainability is necessary in all phases of the project cycle, in each element of the logical framework, from the overall objectives and project purpose to the activities and resources employed. Analysis of sustainability may lead to rejection of a project or corrective adjustments. The content and scale of analysis are tailored to each project in accordance with its nature and size. 1.3.1 Policies must match Development projects are always planned and implemented within the policy environment of the partner country. A project must be in line with political, economic, environmental, social and cultural environment, and this, in turn, must be conducive to the project’s objectives. Policy environment is analysed at the earliest stages of the project cycle, this analysis continuing throughout the project, normally in cooperation with different donors and the partner country.

Analysis of existing policies and the identification of necessary reforms must cover national, provincial, district or communal levels if they affect the project directly or indirectly. In the project identification, the emphasis is on the analysis of existing policy. Key questions which should be asked are: o o o o Which are the policies that affect the project directly or indirectly? Is the project in line with the policy environment, i.e. do the policies support the objectives of the project? Is it necessary, and if it is, possible, to change that policy environment? Would a policy reform as such be sufficient to improve the situation?

Without supportive policy environment, or a possibility to have it revised, the project concept must be reformulated or further planning is stopped. In the project formulation, the emphasis is on the identification of policy environment and possible reforms. These often become a part of the project preparation. Keeping in mind that policy changes often take a long time, the contents and timing of reforms must be carefully planned. The implementation of policy reforms often requires changes in the legal and institutional frameworks. The policy environment must be analysed for all projects.It is important that all analyses are conducted by the partner country or, at least, in close cooperation with it. Conclusions should be drawn up and agreed together with the partner country. Policy reform as a precondition for private sector development To promote private sector development in the Kyrgyz Republic, and thereby economic growth at large, a rapid reform of the financial sector is considered a precondition. To strengthen economic development, Finland has agreed to support the financial sector reform by providing technical and financial assistance to the banking institutions. The aim is to strengthen the banking supervision and to create internationally accepted practices for accounting and internal auditing in commercial banks. In view of the new skills requirements, special emphasis is on training. The Finnish intervention will be part of the Financial Sector Technical Assistance Project (FINSAC), financed by the World Bank and further supported by a number of other donors. The overall objective is to develop and deepen the financial system, restore public confidence, and to establish, over the medium term, an efficient system of resource mobilisation and allocation, which would become increasingly more liberal and marketoriented and offer a greater variety of instruments both to borrowers and savers. 1.3.2 Better value for money Assessment on financial and economic feasibility aims at making sure that resources are being used efficiently and effectively, and that the benefits will be sustained once external support has come to an end. Financial analysis deals with the financial viability of the project. The basic concepts and principles of financial accounting, planning, control and management are applied. The financial sustainability is the key. When assessing financial sustainability, the total costs incurred by the project are estimated and weighed against the partner government’s or institution’s ability to provide continued financing. Special attention is paid to longterm running costs, foreign exchange requirements and the depreciation cost of investments. User charges are often the best safeguard for sustainable benefits. Where such charges are not realistic nor socially acceptable, it is essential that reliable provisions for financing are made. The ability of the partner country to provide adequate public financing can in many cases be only reviewed in the context of public sector recurrent cost burden.

Economic analysis measures the impact of a development project on the partner country’s overall economy and/or on a specific sector. Key elements are: the identification and consistent valuation of all costs and benefits; an appropriate allowance for the phasing of costs and benefits over time; and the evaluation of project acceptability. In theory, a development project is economically efficient when the sum of the positive effects is greater than the total of resources utilised and any negative effects. In other words, the benefits are greater than the costs and the project produces a net improvement or economic profit for a society. In practice, the analysis is more complicated and must, for example, compare the efficiency of different projects. Cost-benefit analysis is a technique which aims at expressing efficiency as a single ratio, which can then be used to compare different projects. It is a demanding tool, mainly applicable to "hard" projects in which benefits can be measured in monetary terms. Both costs and benefits are identified in the broadest and most exact sense possible. Cost-effectiveness analysis determines the unit price of a given benefit, for example, of a single person being able to obtain clean drinking water. The analysis allows the comparison of projects with similar goals and assists in the identification of the most cost-effective way to achieve a specific objective. Cost-effectiveness can be assessed also for "soft" projects such as health or educational programmes. In theory, a project is most cost-effective when it produces the required positive results for the lowest possible cost or when it produces the greatest positive results for a given level of cost. The time factor complicates all economic analyses. For an investment decision today, costs and benefits must be considered, which, in many cases, are only incurred several years into the future. One technique is to discount future costs and benefits to their "present values" in order to facilitate comparisons. In many developing countries, monopolies, import restrictions, price regulation, taxes and subsidies distort market prices to the extent that these do not necessarily reflect true economic value. To tackle this type of distortion, so-called "shadow prices" can be determined and used when making economic analyses. Both economic and financial analyses require that project objectives and success indicators are clearly defined. Systematic use of the logical framework technique makes this task easier. Analyses conducted during the planning and appraisal phases of a project must be followed-up systematically during the implementation and evaluation phases. Data requirements must be built into the project monitoring and reporting systems. Someone must pay for the services In a water supply project, a variety of service levels are provided, ranging from handpump wells and standposts to property connections. The national water tariff is kept low for political reasons. Due to the low tariff, the operator of the water utilities cannot collect from the users a sufficient revenue to pay for electricity, which results in irregular water supply. The subsidies from the Government are insufficient and delayed. Whilst the users with property connections have roof tanks and are not affected by the irregular water supply, those served by public standposts have to queue and minimise water consumption. The project is not feasible unless the tariffs are raised and/or the government subsidies paid. 1.3.3 Institutional capacity makes a difference Adequate institutional capacities enable a country to utilise and manage both national and external resources efficiently. If local institutional capacities are inadequate, they must be identified and developed. Planning of capacity development requires analyses on issues such as: - the activities to be carried out by the institutions to accomplish their tasks. - the actors, institutions or individuals, who carry out the activities.

- the normative context i.e. the vision, values and policies that shape the development of capacity. - the societal context i.e. the political, social, cultural, demographic and geographic conditions to which the capacity development must respond. - the physical, intellectual, financial etc. resources required and available for capacity development. Development of local capacity often means that institutional reforms and human resources development become an integral part of the project. It requires an approach to planning and implementation that encourages mutual learning by donors and local stakeholders. Improved capacity at local level is a valuable end in itself and, as such, often one the most important objectives of a project. Analysing institutional capacity is particularly important in the identification and planning stages, but continues throughout the project cycle. 1.3.4 People-centered development emphasises socio-cultural aspects Different subcultures, the relations between them, values, perceptions of ownership and justice, customs, religions, gender roles and beliefs are examples of socio-cultural factors that influence the success and sustainability of a development project. The behaviour of individuals is determined by social structures and networks of social relationships and obligations, as well as by shared knowledge and values. These factors have the power to mobilise and bind people to a development project or to separate them from it. Socio-cultural analysis helps to identify the various actors involved in, or likely to be affected by, a development project and to understand their nature. Socio-cultural analysis means being alert to a large range of different perspectives and values. It describes what is happening in a society and whether existing values and relationships create an environment where individuals are participating members of society. Such analysis is an essential part of all phases of the project cycle. Culture should be looked upon as a positive factor and a necessary agent and partner in development. The existing cultural structures, which encourage stakeholders in the partner country to take the responsibility of the project from the very beginning, should be supported. Local people are the best experts concerning their own potential for change. If behavioural or organisational changes are necessary, the commitment of local people should be sought by enabling them to design and create the changes themselves . Indigenous cultural identity must be respected and the opinions of different sub-groups in a society must be taken into consideration. Local culture must be respected, also when new ways of action are being introduced. Attention to socio-cultural issues - both to improve the effectiveness of a project and to avoid risks - must be built into the project activities at the earliest possible stage, ideally during project identification. The following type of questions are helpful when assessing the socio-cultural sustainability of a development project: o o o o o o o Is the purpose of the project relevant to the local culture? Are the analyses, the approach adopted, and the methods of implementation in line with the local culture? Does the local culture mobilise social action? Is it included in conception of the project? Who will benefit, directly, indirectly, and how, from the project? What are the effects on the local context? Does the project take into account the different subgroups within the beneficiaries and the project‘s impact on them? Will other, perhaps more vulnerable, people be excluded? Could they be incorporated? How do the local people commit themselves to the project? In what ways could this commitment be improved? Do the persons, groups and institutions involved in the project possess full legitimacy, i.e. are they recognised and respected by the local people?

1.3.5 Participation enhances ownership

Better ownership means that decision and control are increasingly in the partner country, and transferred there from intermediary, implementing organisations to the beneficiaries. Development must be a participatory process, which means that the various stakeholders influence, and share control, over the development initiatives, decisions and resources which affect their lives. Development of a sense of ownership, a sense of authority and control, and empowering people to take control of their lives into their own hands, is an important target in its own right. It is important to assess the possibilities for participation, both from below (NGOs, local organisations) and from above. Government representatives should be encouraged to become more receptive to participatory approaches. In order to "implement participation", it must be defined in operational terms with regard to each specific project. Typical questions that should be asked are: o o o o o o o Are all potential stakeholders, women and men, and sub-groups within them, involved in the project? How intensive is the participation of the stakeholders? What does participation imply in the different phases? (see also Chapter 1.2.3, "Level of participation varies") Which levels are involved? How is the project integrated into local institutions? Have the partner country and the project beneficiaries demonstrated their commitment by e.g. contributing their scarce resources? Are the project procedures and finances transparent?

Costs in terms of time and money tend to be relatively high for participatory projects during their early phases. The initial investments in participation however produce a payoff in terms of increased efficiency, sustainability, and saving of time. 1.3.6 Gender equality and participatory development Efficient and sustainable development is possible only when the needs and roles of both women and men are fully recognised in the planning and implementation of projects. The term gender refers to the socially-constructed roles held by women and men, their responsibilities and the possibilities that are open to them, and also covers relationships between them. Gender is a dynamic concept and includes change over time and space, as well as change during the stages of each individual´s life. In development cooperation, gender analysis guarantees that both women and men have equal opportunities to participate in development, to influence and to benefit. Gender is not just the relationship between women and men. It also includes the concept of society, including culture, law, religion and politics, and its impact on the roles played by women and men. Analysis of the differences between gender roles should be key criteria when assessing the goals and results of development projects. Gender analysis is linked directly to efficiency and sustainability. Involving both women and men means that the resources will be used more efficiently. When both women and men take part in the planning and implementation of a project, the objectives are more widely accepted. The resulting benefits are more likely to be sustained once the project has ended. Gender analysis is carried out in a participatory manner, obtaining information from local women and men. It should also be considered in a framework of social analysis, to take fully into account the social and cultural context. Constraints on, and opportunities for, the participation of both women and men in development activities and the possibility of them receiving equal benefit, must be identified and taken into account in planning. Steps to reduce constraints and maximise opportunities must be taken.

Gender roles must be taken into consideration, at all stages, by including gender analysis in planning, appraisal, monitoring and evaluation and by including gender specialists in the work. 1.3.7 Environment - not only ecology All development projects have a direct or indirect impact on their natural or man-made environments. Making development sustainable requires that planners, decision-makers, implementing agencies and evaluators understand these impacts, and operate accordingly. One method of ensuring that adequate consideration is given to significant "external" impacts in planning and decision-making is to apply the principles of environmental impact assessment, EIA. EIA provides a warning which can help to screen out those projects which will have adverse environmental impacts. EIA is of maximum benefit to planning and decision-making when it is closely integrated into every phase of the project cycle. Environmental protection is not solely a matter of ecology. In addition to the sustainable use of natural resources and the protection of natural systems, it also refers to the preservation of valuable man-made environments. Traditions and available incomes strongly affect the use of natural resources, such as forests. Social, cultural and economic factors must thus be included in the analysis, which means that an EIA is to be conducted as interdisciplinary teamwork. A reliable EIA can never result from an academic exercise in isolation from the community. Only true participation can help to detect real problems and identify sustainable solutions. A participatory EIA, for example, should enable a technical investment to be efficiently sized to reflect the needs of the surrounding communities or the actual capacity which exists for management of environmental impacts. Every project idea requires a preliminary EIA, in which the extent of any further analysis is decided. The scope of this further analysis can range from a quick assessment to a full-scale EIA. It may be unnecessary to conduct more detailed analyses of some projects, usually smaller ones, in sectors that are not environmentally sensitive. The participatory EIA process is often a valuable end in itself. In many countries where national or community level environmental management is still lacking, an EIA can provide a forum for the concerns of the local communities and NGOs. In the best cases, it can also help to strengthen a country’s capability for environmental management. Negotiation skills are in demand The assessment of significance and acceptability of environmental impacts often involve a wide range of values and interests. Conflict may therefore occur between the different public and private parties involved. An EIA provides both a forum and systematic means by which to identify the various opinions, values and interests, and resolve the problems in a democratic manner using negotiation and mediation. For example, Finland has supported several countries in the preparation of master plans for their national forests. The planning processes have often involved a variety of environmental conflicts relating to issues such as land tenure, competing use of land and forest, and the rights of indigenous peoples. In planning situations such as these it is essential to identify the potential conflicts at the earliest possible stage, and to consider the values and interests involved. In planning situations involving conflict, external experts are easily considered to be a stakeholder group and therefore part of the conflict. It is important to make their role as facilitators and mediators clear. When selecting experts, a good understanding of the political aspects of natural resource management and the possession of good negotiation and mediation skills are extremely important. 1.3.8 Technology must meet the needs

Technology is part of a society’s products, processes and institutions and is a major force in social and political change. In a development project, technology refers to much more than the equipment or the tools which are used and this has far-reaching implications on the sustainability. It is important that end users participate in the assessment and selection of the technology to be used. The technology employed should be: o o o o o o socially acceptable (compatible with cultural and religious requirements, attractive in order to generate the willingness to pay for it). politically acceptable. affordable (to the end users or, alternatively, to the sponsors who are committed to subsidies) as both an investment and in operation. "localised", i.e. compatible with the available human resources, spare parts, necessary tools and "after sales service". replicable (so that it can be adopted by a larger clientele - this should encourage private sector participation). ecologically sustainable.

In its ideal form, the technology employed in a development project makes use of existing traditions and contributes to greater local, national or regional independence. It promotes creative human activity and should, in this way, lead to the creation of more appropriate technologies. For example, if a house is built relying on local skills and materials, such a project can encourage the local production of construction materials such as tiles. In development projects which are highly technology-oriented, it is necessary to assess the technology during the identification phase. Appraisal of such projects requires relevant technical expertise. For most development projects, the assessment of technology becomes significant in the planning phase. 2. PROJECT DESIGN 2.1. Situation analysis - the cornerstone of project planning The planning of a development project, or any other type of development intervention, starts with an analysis of the situation. The purpose is to identify the needs, interests, priorities and resources of the stakeholders, from the central government all the way to the final beneficiaries, and to assess the different possibilities for improving the situation. The identification process starts from a dialogue between Finland and the partner country. The analyses begin with general and sector-specific background studies and identification of the stakeholders. The stakeholders together state the problems. Analysis proceeds to a more detailed assessment of objectives and of the project purpose. Alternative strategies are evaluated in the light of Finland’s development policies and the sustainability of the development impact. The possible need for more detailed studies is identified. Analysis brings together the different opinions and views. It is a joint exercise carried out by representatives of the key stakeholders. 2.1.1 Background studies and the analysis of stakeholders The purpose of the background studies is to make available reliable, analysed data. Background studies, including analysis of the stakeholders involved, constitute the first steps in the planning of any development activity. Background studies are carried out at macro level e.g. as country and intersectoral studies, and include sectorspecific analyses as well as studies in the cross-cutting issues introduced in Chapter 1.1.4 "Policies require

action" and Chapter 1.3 "Achieving sustainable development". Evaluations carried out on similar interventions in the past are also consulted. At this stage it is also important to identify other donors and projects involved in the same sector and to coordinate with other actors. The scope of background studies depends on the complexity of the issues being addressed, and the availability of information, such as previous studies or government plans. Stakeholders are groups of people, organisations and institutions that will affect and may be affected by the project. Stakeholders must be involved in both the background studies and in the further project planning as early as possible. This helps to focus the project on relevant needs and opportunities, increases ownership and improves sustainability. Stakeholders can be categorised on the basis of each one’s interest. For example, government agencies, NGOs, and low-income households, are often distinct stakeholder groups. Analysis of stakeholders is conducted as a participatory exercise, often in the form of a seminar or workshops. The analysis covers the needs, interests and priorities of the different stakeholder groups as well as their human, institutional, economic and physical resources. A single project is unlikely to satisfy the interests of all stakeholders. The stakeholders whose needs are considered to have highest priority, will be the project beneficiaries. The beneficiaries are thus a limited group among the stakeholders. Other groups directly involved in the project may include the providers of the services which the beneficiaries will use and from which they will benefit. Critical points in stakeholder analysis Stakeholder grouping requires careful analysis and a thorough knowledge of the geographic project area, its inhabitants and their culture. Assumption that different groups of beneficiaries share the same values and needs can be misleading. This can be exampled by an education project, where it was assumed that the same programme could be applied within a whole administrative region. This did not work, because the region comprised two, quite distinct, sub-cultures with radically distinct expectations: those whose livelihood was fishing and those who were farmers. 2.1.2 Problem analysis - key to the project’s framework The analysis of problems guides the planning. This analysis constitutes a check point for the relevance of an intervention; either justifying it or proving it unnecessary or impossible. The identification of problems is most reliable when undertaken in a participatory way. It is important that planners take into account different groups, and consider both general and group-specific problems. For example, men and women often perceive problems in different ways. Problem analysis must go beyond a simple listing. The stakeholders should address questions such as why the problems occur and why they persist. Joint discussion of these questions is, in itself, a valuable forum for learning and can provide vital information. People know their problems Preparation of the project document for a Rural Water Supply and Environmental Program began with participatory appraisals. These were organised by local experts in order to incorporate the views of the rural communities. Information was collected on geographical, time-related, social, technical and

institutional issues. Consideration was given to the communities´ heritage, experience and knowledge and how this influenced attitudes and behaviour. In the participatory appraisals, communities expressed their views on the problems, put them in order of priority and suggested possible solutions. The second step in the project preparation was to organise a planning workshop. The data which was collected earlier was brought to the workshop, to enable government representatives at different levels to become familiar with the problems and solutions being proposed by the communities. The workshop produced recommendations for strategic approaches. The institutional set-up which was considered most acceptable and sustainable by the workshop, was assembled. The decisions made were based on an analysis of the institutional, social, technical and environmental sustainability. The workshop also carried out an analysis of the programme environment, including the relevant government policies. Detailed project targets and activities were not developed at the workshop. This allowed the final project document to have the flexibility required for processoriented management. The project document´s role is to be a guideline, on the basis of which plans are prepared for practical work. In the problem analysis, problems should be stated as a situation which needs to be improved, and not in a form which expresses the absence of a solution. For example, rather than saying "a lack of hospitals", the problem should be "high infant mortality". Listing of possible solutions at an early planning stage easily hampers objective and open-minded problem analysis. The "problem tree" technique is a tool which is useful in the identification and analysis stages. Users must have the knowledge and skills to use it and they must also understand the project environment. A problem tree usually consists of several levels, a hierarchy of interlinked problems. In this hierarchy, a problem at a lower lever (in conjunction with other problems at that level) leads to the problem positioned one level above it. Connecting lines indicate cause-effect relationships. In the following example, three levels of problems are shown (with the fourth, fifth etc. lower levels continuing downwards). Figure 4: Example of a problem tree

2.1.3 Objectives reflect an ideal future The identification of objectives requires the analysis of problems. The problem tree serves as a tool. The situations recorded as requiring improvement during the problem analysis are converted into corresponding positive statements, i.e. achievements to be made or objectives targeted. If a problem is described as "inefficient farming requiring more land to satisfy food needs", the corresponding positive statement is "efficient farming not requiring more land to satisfy food needs". Another diagram is then formulated where every cause-effect relationship between problems becomes a positive means-end relationship between objectives. Figure 5: Diagram of objectives This diagram represents an ideal future situation where the desired conditions are fulfilled. Attainment of an objective at a lower lever (together with other objectives at that level) leads to realisation of an objective one level higher. The validity, both of the objectives and their means-ends relationships must be checked. Objectives so derived might not always be realistic, in which case other solutions should be sought. If, for example, there is a problem stated as "low world market price of timber", the corresponding objective would become "high world market price of timber". This is unlikely to be an objective that could be attained within the framework of any single development project. However, "product diversification" or "further processing of timber" might be other solutions to the same problem. Each of the branches in the objective tree represents an alternative strategy for the future project. The diagram therefore becomes a useful tool when comparing and assessing the alternative ways of tackling the problems. One or more of the alternative strategies are included in the project plan. 2.1.4 Strategic choices begin by fixing the project purpose Screening of alternative strategies begins by defining the project purpose, i.e. stating either the focus of the project or the reason for it. The purpose is derived from the diagram of objectives. From the analytical point of view, each project can have only a single purpose. The project purpose describes the improved situation. The project purpose can relate to the economic (income, productivity), social (living conditions, health), or personal (fulfilment of aspirations, human rights) environment of the beneficiaries. The benefits must be sustainable. The project purpose should not be stated as if it were the delivery of a service, but as a sustainable utilisation of that service by the beneficiaries. In a housing project, for example, the project purpose should not be stated as "construction of houses for low-income households", but as "low-income households’ housing conditions improved". Selecting a project purpose high in the objectives hierarchy usually calls for a multi-component programme, i.e. a programme with several strategies which complement one another. A project purpose lower down the scale would suggest a smaller project. The resources of the different stakeholders, which were analysed earlier, thus lay ground for the selection of the project purpose. In the forestry project example, drawing the project purpose from the objective "sustainable use and management of forests" might call for an extensive programme covering the whole forest sector. Taking the project purpose from the objective "clear land use policy and tenure rights" would suggest a smaller project focusing on those aspects. Some of the objectives of the diagram, which are required in order to achieve the project purpose, are defined as "project objectives". The other objectives may be achieved by others and, in subsequent analysis, are considered as "external factors" which can affect the success of the project.

The figure on the next page demonstrates the division of objectives into the two categories. The project objectives set the framework for the strategy of the project.

Figure 6: Division of objectives into two categories !!!!!!!! "project objectives" and "external factors"

Selection of the project strategy (i.e. the division of the objectives into the two categories) is based on a number of factors, such as: 1. RELEVANCE The degree to which the proposed strategy corresponds to the needs of the beneficiaries. 2. IMPACT The proposed strategy’s predicted impacts on the beneficiaries. 3. EFFECTIVENESS The degree to which achieving the lower level objectives of the proposed strategy will contribute to achievement of the project purpose. 4. EFFICIENCY The estimated quantity, quality and availability of means and resources required by the proposed strategy. The presumed cost-effectiveness of the proposed strategy in transforming the means into results. 5. POLICIES Correspondence of the proposed strategy with local policies and its compatibility with the general objectives of Finland’s development cooperation policy. 6. SUSTAINABILITY The likelihood of the proposed strategy to:

- correspond with local policies; - be in line with, or strengthen, local institutional capacity; - be economically and financially sound; - be suitable to local socio-cultural context; - be based on participation of local stakeholders; - be based on participation of both men and women; - create positive impacts on environment; - apply appropriate technology. 7. ASSUMPTIONS AND RISKS Likelihood of external factors and risks to affect negatively the proposed strategy. Analysis using these factors results in each alternative strategy acquiring a different profile. These profiles can then be used as the basis for identifying the most pertinent and feasible strategy for the project. The Intervention profile/performance summary-form presented in Annex IV can be used to compare the different profiles of the alternative strategies. The same form is used for monitoring and evaluation as well, thus enabling the same criteria to be applied when assessing success or failure throughout the lifetime of a project. All the factors seldom support the same project strategy. For example, one strategy may be the most feasible from the environmental viewpoint, and another the most feasible from the financial viewpoint. In such cases, the stakeholders negotiate, and jointly determine, the decisive factor to be employed in strategy selection, i.e. they decide which factors are to be given priority and why. To be able to decide this, more detailed studies might be required. It may be necessary to interrupt planning activities to conduct the additional analyses. In general, more detailed studies will also be necessary once a strategy has been selected. The needs for such specific assessments are identified together with selection of the strategy. As a result of the analysis of alternative strategies, the following elements of the project can be presented: 1. The project purpose 2. The project strategy 3. The decisive factor for the selection of the strategy 4. The need for further studies Reaching this stage in the planning process is often considered as the end of the identification phase of the project cycle. 2. Planning with logic The situation analysis presented in Chapter 2.1 provides the information, which is required to define the project’s intervention logic, and this represents the basis for further planning. The logical framework method includes an assessment of the project environment; external factors have a major

influence on success or failure. The identification and analysis of these external factors therefore constitute a crucial task. The establishment of indicators makes the plan real. The indicators show, in concrete terms, what happens and where. Indicators serve their purpose only if they can be checked in a feasible manner. The chosen indicators and their sources of verification provide the basis for monitoring and evaluation of the project. 2.2.1 Logical framework is a practical tool The logical framework is a tool which improves project planning and implementation. It is a systematic method for setting and analysing the objectives of a development project and the assumptions behind it. The logical framework stresses the causal relationships between key elements and provides a standard model for their presentation. Figure 7: Intervention logic and the logical framework matrix

Overall objectives

Indicators

Sources of verification

Project purpose

Indicators

Sources of verification

Assumptions

Results

Indicators

Sources of verification

Assumptions

Activities

Means

Costs

Assumptions Pre-conditions

However illustrative the matrix presentation, it should be remembered that the logical framework is only a tool to be used in the process of development planning. The process of involving various stakeholders is an important goal in itself. Analysis of the background information plays a key role in successful application of the logical framework. Impact assessments in cross-cutting issues, such as the policy environment, institutional capacity and gender, are carried out in order to assess sustainability. Other planning tools can complement the logical framework. At the grassroots level, for example, Participatory Rural Appraisal (PRA) methods can be used to facilitate a participatory planning process. (See also Basic Principles: Chapter 1.2.3 - "Level of participation varies"). Even if PRA or other methods are used, the information produced is structured in the logical framework matrix. Correct application of the logical framework requires methodological, organisational and communication skills that can be acquired or improved through training. Thorough knowledge of the project environment, its physical area and population, are also required.

2.2.2 Intervention logic states the strategy The situation analysis presented in Chapter 2.1 provides the necessary inputs for further planning. The further planning is conducted using the logical framework matrix, which portrays the different aspects and their contribution to the success of the project. The proposed project strategy defines the intervention logic. As described in Chapter 2.1.4 "Strategic choices begin by fixing the project purpose", it is derived from the diagram of objectives. The intervention logic is dependent on the project purpose and, in addition, states the project’s overall objectives, results, activities and means. The project purpose states the specific objective of the project, i.e. the improved situation from the point of view of the beneficiaries (see Chapter 2.1.4 - "Strategic choices begin by fixing the project purpose"). The overall objective(s), on the top of the intervention logic, describe(s) the long-term development goals to which the project makes a contribution. Several development projects may contribute to the same overall objective. Its attainment often takes longer than a single project’s lifetime. The results are derived from the objectives which, together, lead to the project purpose. Results should be stated as an improved state of affairs such as "adequate legal framework exists" or "peasants are planting trees". Results may be, for example, services and infrastructure delivered by the project and used by the beneficiaries. Thus, results are not only physical outputs but a start to enjoyment of sustainable benefits. The activities are derived from the lower level objectives which, combined, produce the results. When these activities are carried out, the means are transformed into results that together will achieve the project purpose. In the project document, activities should be expressed as processes which will be then broken down into more detailed tasks in workplans. For example, the activity "training in occupational therapy" is in works plans broken down into "curriculum preparation", "theoretical classroom exercises", "practical hospital training", "student examinations", and "follow-up and continuing education". The means are human resources and material inputs required to carry out planned activities and manage the project. As in the case of activities, means are first defined in general terms, and then detailed later in terms of quantity, quality and finance. Although the building-up of the intervention logic is based on the diagram of objectives, the tool should not dominate the planning. Planners should be prepared to modify the intervention logic as various needs and ideas arise during the participatory planning process. The intervention logic forms the first vertical column of the logical framework matrix. Figure 8: First vertical column of logical framework in the forestry project example: INTERVENTION LOGIC

OVERALL OBJECTIVES

De-forestation and degradation under control

PROJECT PURPOSE

Sustainable use and management of forests

RESULTS

1) Adequate political, legal and institutional

framework 2) Clear land use policy and tenure rights ACTIVITIES

2.2.3 Assumptions must hold The intervention logic does not represent entire reality. A variety of external factors, even though they are outside the scope of the project, influence, or even determine its success. The identification of these external factors and an analysis of their influence play an important role in project planning. In total, the external factors constitute a project’s environment. The project objectives can be achieved only in a favourable environment. Figure 9: The project and its environment Some external factors arise from the objectives presented in the diagram of objectives. These are the objectives which are not included in the proposed project strategy and the intervention logic. Other external factors emerge during the participatory planning process or are identified through additional studies. According to the "project objectives" of the forestry project example (see page 29), the project will concentrate on the political, legal and institutional framework as well as on the land use policy and tenure rights. The "external factors" are however also necessary to reach the project purpose "sustainable use and management of forests". One of the assumptions is that there is adequate investment in renewal of forest resources i.e. in replanting of trees. This, in turn, depends on issues which give an economic value to trees and create incentives to plant them. Another assumption is that farming practices are improved so that forests do not have to be cleared for agriculture to increase food production. The third assumption derived from the diagram of objectives is that the partner Government gives high priority to forest conservation. This can also be a precondition to be met before project activities begin. In addition to the above assumptions, which are all derived from the diagram of objectives, there may still be other external factors identified in the planning process. In the logical framework matrix, external factors and risks are expressed as positive conditions i.e. assumptions important for the success of the intervention. To decide which assumptions to include in the matrix, an assessment of their influence and likelihood to materialise is necessary. In the assessment, the external factors are placed in the diagram of objectives at the appropriate level. The extent of their influence is assessed by posing the following questions: 1. Will the external factor be important for the success of the intervention? 2. Is the external factor likely to occur? 3. Is it possible to redesign the intervention if required by the external factors? The possible alternative answers and their consequences are illustrated in the following diagram. Figure 10: Analysis of external factors

Assumptions link different levels of the intervention logic. In the logical framework matrix, they are placed according to their position in the diagram of objectives. For example, results will be achieved only once activities have been carried out and the assumptions at that level have materialised. A precondition is a condition that must be fulfilled before project activities can begin (see the forestry project example on page 35). Assumptions form the last vertical column of the logical framework matrix. Figure 11: Assumptions in the logical framework 2.2.4 Indicators make the plan concrete Indicators describe the project’s overall objectives, purpose, and results in operational and measurable terms. They enable reliable monitoring and evaluation of the project. The joint specification of indicators by key stakeholders helps to analyse what the project means to the stakeholders. Important points often emerge which have not been recognised in the previous planning phases. Indicators are established in the planning phase but they can be revised when the project concept is revised. Additional indicators can be established, for example, to meet the requirements of an evaluation. For most projects, it is necessary to specify both qualitative and quantitative indicators. It is often useful to have several indicators for a single objective. Quantitative and qualitative indicators in a NGO health project Both qualitative and quantitative indicators are used in a Training Health Centre project in Malawi, supported by a Finnish NGO. The quantitative indicators on different levels include number of visits to the out-patient department and antenatal clinic, vaccination coverage, percentage of deliveries assisted by trained personnel, and the number of cases of different diseases and malnutrition divided by sex and in age groups. The qualitative indicators used measure the satisfaction of the people with the different health services given, the involvement of different groups of people in decision-making in the local management committee and the type of decisions made there. Sometimes it may be too difficult, or too costly, to use direct indicators. For example, the quality of roofing or the number of radio sets possessed by a community might serve as an indirect indicator to measure the income level. An indicator must answer questions such as "How much?", "What?", "Who?", "For how long?" and "Where?". To answer these questions, the sources of verification must be established, i.e. it is necessary to define how the indicator will be checked in practise. It is important to define an optimal number of indicators, since too many, or too complex, indicators can be difficult and costly. Indicators are established on different levels of the intervention logic and the logical framework. Some indicators measure progress or changes in the project environment, others are used to monitor impact, effectiveness, efficiency etc. Figure 12: Issues to be monitored using indicators Progress/environment Impact/effectiveness Overall objectives - Relevance of project

- Impact of project

Project purpose

- Extent of achievement of Project Purpose - Effectiveness of project

Results

- Extent of achievement of results

- Sustainability of project

Activities

- Carrying out of activities

- Efficiency of activities

Means

- Delivery of means

Assumptions

- Changes in environment

In order to monitor effectiveness and impact, one requires appropriate indicators. The indicators should give an "early signal" of progress towards the project purpose and the overall objective. As an example, the indicators can measure whether the beneficiaries have access, are using, and are satisfied with the project services. This gives an indication that the project is offering relevant services and the project purpose is likely to be met. Statistics-based indicators such as literacy rates by sex, infant mortality or value of agricultural exports can also be used to monitor effectiveness and impact. Specific indicators are often necessary in monitoring factors ensuring sustainability, presented in the Basic Principles, Chapter 1.3 "Achieving sustainable development" . The indicators are used continuously by the project management. The related data-gathering activities should be a part of the project. The indicators may also be checked by external evaluators. The specification of indicators and their sources of verification provide the basis for project-specific monitoring and evaluation. Baseline studies are often necessary for identifying indicators and in order to establish the actual departure point using them. In the logical framework matrix, indicators form the second vertical column. The third column should be reserved for their sources of verification. A complete logical framework matrix can now be drawn up. Figure 13: Logical framework matrix Overall objectives Objectively verifiable indicators Sources of verification

Project purpose

Objectively verifiable indicators Objectively verifiable indicators Means

Sources of verification

Assumptions

Results

Sources of verification

Assumptions

Activities

Costs

Assumptions

Pre-conditions

For programmes with several sub-projects, or for distinct project components, it can be useful to extend the logical framework across several interlocking frameworks. This makes more detailed planning possible and assists in the division and definition of responsibilities. When using interlocking matrices, the results of the larger programme form separate components or subprojects. The project purpose of the larger programme becomes the overall objective for all the components. Results of the programme framework become the "component purposes". In the following picture, a large programme is first presented in a single logical framework matrix which is then broken down into several interlocking matrices, each of them presenting, in more detail, one result (a component or a sub-project) of the programme. Figure 14: Logical framework matrix for a programme and an interlocking matrix for its result 1 (component 1) Logical framework matrix for the programme (will be availabe online later on) Logical framework matrix for programme component 1 (will be availabe online later on) Logical framework matrices for other programme components 2.2.5 Approach describes how The planning process and the logical framework matrix as its outcome, provide the basic contents for a project document. In addition, the project document format presented in Annex III contains information on how the project will be implemented. The implementation approach refers here to issues such as the planned geographic coverage of the intervention (national, regional, etc.), its general timeframe (short-term, medium-term, long-term), the intended level of participation and e.g. whether it is a pilot project to be extended to other geographic areas at a later stage. The general timeframe is further detailed in a tentative timetable. 2.2.6 Organisation determines roles and responsibilities Implementation organisation is designed based on the logical framework and Finland’s cooperation policies. Partner countries themselves must assume the responsibility for, and ownership of, the intervention. Finnish assistance plays a catalytic role in the development process. An example of an organisational structure for the management and monitoring of a large project is presented in Annex V. An external agency, a consulting company or other institution, can be selected by Finnish and partner country authorities to provide support services in implementation. The services normally include technical assistance personnel, financial services and administrative support. The agency assists project personnel in planning and monitoring throughout the project cycle. However, the agency does not represent the Government of Finland nor the partner country.

2.2.7 Budget details financial framework The budget provides the financial framework for implementation. It reflects internal priorities of the project and different roles and responsibilities of the stakeholders. The budget draws a picture to illustrate which way the intended Finnish contribution is part of the general financial framework, local and foreign, for the sector or a programme. The responsibility and ownership of the partner country are confirmed through its financial contribution to the project. When project implementation is planned, the possibilities of different stakeholders to contribute financially are analysed. The budget specifies the contributions by source, such as local government sources, beneficiaries’ direct contribution, Finnish contribution and other foreign sources. The use of the different contributions and the timing of their availability is planned. Contributions-in-kind such as premises or personnel are determined in financial terms or, if that is not possible, in numeric terms such as person-months, square-metres etc. The budget is prepared jointly by the key stakeholders’ representatives. In implementation, the budget is reviewed annually and revised according to need and possibility. The actual expenditures are regularly compared with the budget figures and the progress made. 2.2.8 Various roles of the project document The project document, which is produced as a result of the planning process described in these guidelines, is used as a planning tool in the preparation of work plans. The original project document is reviewed annually and, if necessary, revised in the annual work plans. The work plans specify the contents of the project document and adjust it to the changing project environment. Preparation of work plans is described in Chapter 3.3.1. The project document is also used as a legal tool in various agreements. It is annexed to the intergovernmental agreement between Finland and the partner country. It also serves as the basis for contracting a consulting company, or other institution, to provide support services to implementation.

3. MONITORING

Monitoring is the continuous assessment of the intervention and its environment. It takes place at all levels of management and uses both formal reporting and informal communications. These guidelines determine the issues to be monitored within a single intervention, be it a programme, project or other kind of cooperation. A minimum common format for progress and monitoring reports is established, based on which, each project will build its own internal monitoring design. The approach aims at developing the partner country´s own monitoring and reporting systems rather than imposing on them parallel systems used only for projects with Finnish contribution. 3.1. How stakeholders monitor Monitoring is the continuous assessment of the intervention and its environment with regard to the planned objectives, results, activities and means. Monitoring enables a stakeholder to review progress and to propose action to be taken in order to achieve the objectives. Monitoring identifies actual or potential successes or failures as early as possible and facilitates timely adjustments to the operations. Monitoring takes place at all levels of management and uses both formal reporting and informal communications. Representatives of different stakeholders are engaged in monitoring: - the local and Finnish government officials, who are not directly involved in the implementation monitor the interventions externally; - the local and expatriate project managers i.e. the management team of the intervention monitor it internally; - the beneficiaries and other affected groups monitor the intervention when they participate in its implementation, use its services etc. An example of a project specific internal management and monitoring set-up is presented in Annex V. The example covers the stakeholders involved in the project but excludes others who may be affected by it. The setup is best applied to large projects in countries where there is a Finnish Embassy. Such a multi-level set-up is not justified for small projects. Design of a project specific monitoring system is essential during the formulation or early implementation phase. Good design is based on the clear setting of measurable objectives and assumptions for the intervention and its components, for which indicators are defined. For the selected set of indicators, sources of verification are established and provisions made for collecting data and managing records. If project external statistical information is used, institutional arrangements for gathering, analysing and reporting data are agreed with local statistics offices. The actual departure point, i.e. benchmarks which describe the situation and trends during a particular period, is established using the defined set of indicators. Baseline studies are often used to identify indicators and give contents to them. Baseline study is a collection of data which describes and analyzes socio-economic and other conditions and trends in a particular site during a particular period. The indicators set through a baseline study are used at a later date as reference points to demonstrate that an objective of the project has been reached and that a change has occurred. Special emphasis will be put on monitoring capacity building within the partner countries and organisations, with the aim of making the monitoring activity sustainable.

3.2. Integrated approach facilitates monitoring The integrated approach to project planning, implementation and evaluation ensures that the same criteria is used all along the project cycle. The integration means that the same elements, e.g. impacts on such factors as gender and environment, are monitored throughout the project. This is realised by using the same basic format for all reports, including the monitoring reports. The project document format used is presented in Annex III. The use of the logical framework is part of the integrated approach. It systemises the analysis and setting of objectives and assumptions. In the logical framework matrix, the project’s overall objectives, purpose, and results are pronounced in operational and measurable terms by using indicators. In the logical framework, indicators are the main tool for monitoring progress towards achieving the objectives. The hierarchy of indicators follows that one of the objectives: some indicators measure the achievement of results, others the achievement of the project purpose. Figure 15: Logical framework and levels of indicators Overall objectives Indicators Sources of verification

Project purpose

Indicators

Sources of verification

Assumptions

Results

Indicators

Sources of verification

Assumptions

Activities

Means

Costs

Assumptions Pre-conditions

Since no project can be planned nor implemented in isolation from its environment, the external factors are equally important to be monitored. In the logical framework, the external factors are expressed as assumptions, the likelihood of which to materialise must be monitored. 3.3. What is monitored and how Issues to be monitored include progress of implementation, environment of the project, and its impact and effectiveness. These, and other issues to be monitored, can be positioned on different levels of the intervention logic as presented below. Figure 16: Issues to be monitored using indicators

Progress/environment

Impact/effectiveness

Overall objectives

- Relevance of project - Impact of project

Project purpose

- Extent of achievement of Project Purpose - Effectiveness of project

Results

- Extent of achievement of results

- Sustainability of project

Activities

- Carrying out of activities

- Efficiency of activities

Means

- Delivery of means

Assumptions

- Changes in environment

In formal reporting, these issues to be monitored are addressed in progress and monitoring reports. 3.3.1 Progress reports The project document normally is not precise in relation to activities and means. It is usually considered rather as a "process document", with emphasis on development options and objectives. During implementation, the project document is specified and adjusted to the changing environment in work plans. The work plan is derived from the logical framework matrix for the intervention. The activities, expressed as processes in the project document, are broken down into more detailed tasks (see page 33) for which timing is specified. Operational responsibilities are further elaborated. The work plan is presented together with a budget which details the delivery of means. An example of a work plan format is presented in Annex VI. The extent of the achievement of results is monitored by comparing the actual results with those planned in the project document and the work plans. The indicators used measure behavioural change such as "involvement of people in local decision-making" or "establishment of water users´ associations", or physical progress, such as "family planning advice units established and in use by the beneficiaries". The carrying out of the activities is monitored by comparing the activities performed with the milestones established in the work plans. The indicators are derived from the project records, e.g. dates of starting and completing a specific activity. The delivery of the means, the human resources and material inputs, is monitored by using indicators that are either physical, e.g. appointment of staff, or financial, e.g. a partner country contribution which has been made available. The data is derived from the project accounts and records. Changes in the project environment may affect both the results, activities and means of the project as well as its impacts and effectiveness (described in Chapter 3.3.2.). The changes are monitored by assessing the likelihood of assumptions and risks (which are defined in project planning) to materialise, and by observing changes in the external environment of the project in general. Especially, other projects within the sector and the possible changes in the sector policies are under observation.

The foregoing issues form the minimum common format for progress reports: 1. Extent of achievement of results 2. Carrying out of activities 3. Delivery of means 4. Possible changes within the sector and in the project environment in general (including likelihood of assumptions and risks to materialise), and their effects on the project. 5. Proposals for changes in the planned activities, their justification and, if necessary, a request to other stakeholders to approve the change formally The emphasis is on reporting results and changes in the environment to other levels of management using appropriate indicators. Activities and means are reported only in a summarised form. The minimum common format may not always be sufficient. As an example, the priority needs of the beneficiaries may change due to unforeseen external factors and the project strategy chosen does not any longer correspond to them. A "check-list" of issues to be monitored is presented in Annex IV "Intervention profile/performance summary" form. Monitoring of all those issues must be continuous although they are included in the progress reports only when there is a specific reason to do so. The progress reports are normally prepared quarterly. In very rapidly progressing projects, the reports can be prepared monthly. The existing reporting systems of the partner country are followed to the extent possible, and developed in view of this minimum common format and other needs. The level of detail in monitoring and reporting is proposed in the project document and agreed between the stakeholders when the implementation starts. Example of a project specific progress reporting system In a large, rural development project to which the monitoring set-up of Annex V is applied, the work plans are drafted by the management team using participatory methods and finalised by the local steering committees. The work plans cover the fiscal year of the partner country. The progress reporting is primarily a task of the management team, composed of partner country project managers and expatriate advisers. In its draft progress reports, the management team compares the actual results, activities etc. with those foreseen in the work plan. The logical framework format is used to list and schedule the activities, and to show any differences between the planned and the realised. The progress reports are drafted quarterly for finalisation and approval by the local steering committee meeting. The beneficiaries´ representatives participate to the local steering committee meetings and their views are taken into account when finalising the quarterly progress reports. The last "quarterly" progress report, prepared before the annual supervisory board meeting, covers the whole year and is included in the annual monitoring report (described in Chapter 3.3.2).

3.3.2 Annual Monitoring Reports Some elements of the project, such as its overall objectives, factors ensuring sustainability or compatibility with the strategic goals of Finnish development cooperation, are not addressed in the progress reports unless there is a specific reason to do so. However, they must be under continuous review and assessed, at least, in annual monitoring reports. Relevance is monitored by assessing the correspondence of the project with beneficiaries´ priority needs in the changing project environment. Overall objective, which describes the long-term development goal towards which the project contributes, and Project purpose, which states the direct effects of the project on the beneficiaries, are subject to impact and effectiveness monitoring. Impact can normally be assessed after the project has been implemented for a few years. Although the assessment can be a costly and complex task, it is the backbone of any evaluation exercise and must be addressed already during planning and implementation. Project purpose is unlikely to materialise until the end of the project. However, the extent to which the project purpose has been (or is likely to be) achieved must be assessed. This cause-effect relation between the results achieved and the fulfilment of the project purpose as a consequence describes the effectiveness of the project. To monitor effectiveness and impact requires appropriate indicators. The indicators should give an "early signal" of progress towards the project purpose and overall objective. For example, the project purpose may describe the beneficiaries´ use of the project services such as preventive health care. The indicators measure whether the beneficiaries have access, are using, and are satisfied with the project services. This gives an indication that the project is offering relevant services and the project purpose is likely to be met. The use of "early signal" indicators means that the beneficiaries´ views are assessed continuously and systematically within the project internal monitoring system. The management team may also organise periodic surveys using local specialists. To monitor effectiveness and impact, the management team can also use statistics-based indicators such as literacy rates by sex, infant mortality or value of agricultural exports. Factors ensuring sustainability are monitored continuously using established indicators and/or direct observation. This includes an assessment of changes in the sector policies and in the institutional framework. Efficiency concerns the relation between the results and means i.e. whether the quantity and quality of results achieved justify the quantity and quality of means used. Efficiency is monitored using cost-based indicators, e.g. the cost per square metre, and/or performance-based indicators, e.g. the time used for a specific task. The issues to be monitored form the minimum common format for monitoring reports: 1. Summary 2. Proposals for changes in the project, if any, and their justification 3.1 Correspondence of the project with the present priority needs of the beneficiaries i.e. relevance 3.2 To the extent possible, assessment of impact of the project towards its long-term development goal 3.3 Extent of achievement of the project purpose as a

consequence of the project results, i.e. effectiveness 3.4 Extent of achievement of results 3.5 Carrying out of activities (and delivery of means, if needed) 4. Possible changes in the project environment, including materialisation of assumptions and risks, and their effects on the project 5. Factors ensuring compatibility and sustainability 5.1 Compatibility with the strategic goals for Finnish development cooperation: reduction of poverty, promotion of equality, democracy and human rights, etc. 5.2 Policy environment 5.3 Economic and financial feasibility 5.4 Institutional capacity 5.5 Socio-cultural aspects 5.6 Participation and ownership 5.7 Gender 5.8 Environment 5.9 Appropriate technology 6. Assessment of the efficiency of the implementation Following the integrated approach to project cycle management, the minimum common format for monitoring reports is compatible with the project document and evaluation report formats, as shown in Annex I. The monitoring reports are normally prepared annually. The last quarterly (or monthly) progress report covers the whole fiscal year and is included in the annual monitoring report (Chapters 2, 3.4, 3.5 and 4 of it). The sustainability and compatibility factors are explained in detail in the Chapter 1.3. "Achieving sustainable development". The reporting, monitoring and evaluation issues and their relationships with the intervention logic are presented in a diagram form in Annex VII. In progress reports, it is the management team reporting their own understanding of the results achieved and the activities performed. In monitoring reports, it should primarily be the beneficiaries’ and other affected groups’ views and perceptions, which are compiled and reported by the management team. Monitoring reports must make a clear distinction between the management team’s and other stakeholders’ views on the monitoring issues. The management team must assess the project objectively and report even the critical views of other stakeholders. The management team should keep in mind that external mid-term reviews, evaluations and audits will address the same issues. Discrepancies between the external assessments and the management team’s monitoring reports are major criteria in evaluating the management team’s performance. At the end of the programme a completion or final report will be prepared according to the instructions in Annex XI.

Example of a project specific monitoring system In the example rural development project, to which the monitoring set-up of annex V is applied, the management team is responsible for compiling the annual monitoring report based on continuous follow-up and periodic informal surveys on beneficiaries’ views. The continuous follow-up is realised through key informants among the beneficiaries, and through informal gatherings and formal meetings of the beneficiaries, from which the management team receives the views. Although beneficiaries’ representatives participate to the steering committee meetings, periodic informal field surveys are conducted to ensure that the views of the different subgroups within the beneficiaries are taken into account. The continuous follow-up and periodic informal surveys measure the beneficiaries´ and other affected groups’ satisfaction with project results and their perceptions of changes in the project environment. Sample groups of beneficiaries are used to avoid too complex and costly surveys. The beneficiaries’ and other affected groups’ views give contents to some of the indicators used. Other impact and effectiveness indicators are based on statistical data provided by the local authorities. When compiling the annual monitoring report, the management team adds their own separate assessment of the monitoring issues. The annual monitoring report is finalised by the local steering committee meeting and then presented for approval by the annual supervisory board meeting.

3.3.3 Other performance monitoring Monitoring takes place at all levels and uses both formal reporting and informal communications. Beneficiaries monitor project performance daily when they participate in project activities and/or use services delivered by the project. Representatives of the partner country and Finnish government visit the project from time to time. Other stakeholders are affected by the project and form their impression of it based on the impacts felt. Personal judgements of different stakeholders are collected using a rating form. The Intervention profile/performance summary-form form is presented in Annex IV. Based on it, project specific forms can be prepared e.g. for the use by the beneficiaries. The use of the summary form provides a structure for judging performance by a set of criteria and a rating scale, which is agreed upon by the stakeholders using the form. The summary form may be used in connection with steering committee meetings, annual supervisory board meetings, mid-term reviews, evaluations or on an ad-hoc basis. If used systematically, it provides stakeholders with a tool to identify successes and weaknesses, to observe progress over time, and to compare different projects. Example of the use of the summary form In the rural development project, to which the monitoring set-up of Annex V is applied, the summary form is used e.g. in connection with the annual supervisory board meeting. The aim of the representatives, from the Ministry for Foreign Affairs of Finland and the Ministry of Cooperation of the partner country is to judge which are the weak and strong points of the project. They also wish to check whether there are differences in views between them, and between

their views and the annual monitoring report. Since the representatives use the form annually, it also provides a track record of progress over years. The programme officer in charge of the project in the Ministry for Foreign Affairs in Finland uses the form e.g. when someone from the consulting company providing support services for the project is visiting the project site. The visitor is requested to fill in, and return, the form signed to the Ministry after the travel. 3.3.4 Financial reports The minimum common format for financial reports is based on annual budgeting and quarterly financial reports. The partner country’s own financial reporting systems are followed to the extent possible, and developed in view of this minimum common format and other needs. The project specific financial reporting system is built on the minimum common format and on the specific requirements of the project and the partner country. The different stakeholders’ needs of information and their capabilities to analyse it determine what is presented and how. Simple standard tables and graphics promote understanding and use of the reports as a planning tool. Both donor and partner country expenditure is reported. Contributions-in-kind such as premises or personnel which may be difficult to report in financial terms, are described in words. The following issues form the minimum common format for financial reports: 1. Executive summary, highlighting any major overexpenditure or underutilisation of funds, reasons for them and action required from the financiers, if any. 2.1 Comparison of the planned budget for the quarter with the actual expenditure during it, by component and, if necessary, by budget line. 2.2 Explanation for the differences between the budget and the actual expenditure, by component and, if necessary, by budget line. 2.3 Cumulative expenditure from the beginning of the fiscal year, by component and, if necessary, by budget line. Annexes: The original annual and quarterly budgets The financial reports are normally prepared quarterly. They should be prepared and submitted together with the quarterly progress reports, so that the two combined provide a complete record of progress to financiers and other stakeholders. The last quarterly financial report covers the whole fiscal year. An example of an expenditure follow-up table and related graphics is presented in Annex VIII. Instructions for preparing the programme completion report or final report are given in Annex XI.

4. EVALUATION 4.1. What evaluation is Evaluation is a systematic and objective assessment of the design, implementation and outcome of an on-going or completed intervention. The two main purposes of evaluation are i) to improve future aid policy and interventions through feedback of lessons learned, and ii) to provide a basis for accountability, including the provision of information to the public. Different types of evaluations comprise project, programme, sectoral, thematic, policy and management evaluations. Other related assessments include audits and reviews. The integrated approach to project cycle management and the logical framework method are used in these guidelines as a basis for designing and carrying out evaluation work. 4.1.1 Evaluation has two main purposes Evaluation generally performs two functions: i) it is a learning tool to improve future aid policy and interventions, and ii) provides a basis for accountability. Although both functions are usually covered by the same exercise, it is important to determine, in advance, the main users of an evaluation. Donor agencies, partner countries, consultants, technical advisers and administrators are interested to improve aid policy, programmes and projects through evaluations. Their emphasis is on lessons to be learned for future rather than on whether projects succeeded or failed. Accountability is wanted by the Parliament, politicians, media, pressure groups etc. Emphasis is on success or failure in light of originally-defined objectives, and the reasons behind the successes or failures. The most important aspect, however, is to realise and remember that all evaluation efforts are wasted if there is no change in organisational behaviour as a result. The guiding principles of the Finnish evaluation work are those determined by the OEDC/DAC and its Expert Group on Aid Evaluation, in whose work Finland seeks active partipation. According to the definition of OEDC/DAC, an evaluation is an assessment, as systematic and objective as possible, of an on-going or completed project, programme or policy, its design, implementation and results. The aim is to determine the relevance and fulfilment of objectives, developmental efficiency, effectiveness, impact and sustainability. The evaluation process must be impartial and independent. It must provide information that is credible and useful, enabling the incorporation of lessons learned into the decision-making process of both partner countries and donors. 4.1.2 Different types of evaluations and other related assessments Project evaluation is the most common evaluation type. In general, it is also the easiest to undertake. If the project design is well done, the evaluator can base his or her work on fixed, clearly defined objectives with relevant and measurable indicators. In practice, the eventual project may finish up looking very different from its original design. Therefore, it is important that records of the changing circumstances are kept throughout the project's life, i.e. monitoring becomes crucially important for effective evaluation. Programme evaluation will become more common with the planned shift away from individual project assistance towards programme assistance. Programme assistance may take the form of general programme assistance,

such as balance of payments support or import support, or sector programme assistance, which is targeted at a particular sector and often linked to specific policy conditions. Programme assistance involves many donors, and the partner country's own investments, and is difficult to evaluate without, at the same time, forming a judgement about the reform effort in general. The objectives of such reforms are often not as clearly stated as they generally are with specific projects, whilst success or failure may be influenced more by a number of external factors than by the actions taken by the partner country. For the above reasons, programme evaluations should be made jointly not only with the partner country but also with other donors. Other non-project evaluations include policy evaluations on issues such as the use of concessional credits or a NGO support programme, and thematic evaluations, which focus on issues such as participation, gender or cost effectiveness. In management evaluations it is the organisational structure and behaviour under scrutiny. Active participation in the evaluations of multilateral aid is necessary to be able to propose improvements in the performance of international organisations, and to judge whether the financing of their operations is justified. Other related assessments include audits, which assess the conformity of the intervention to procedures, norms and criteria established in advance by the financiers. Since issues such as management performance are under scrutiny, audits can be considered part of the external monitoring of a project. Reviews are mid way between monitoring and evaluation, as they involve a fresh look at the objectives, design and performance of a project. Compared to evaluations, reviews are more limited in scope, time and focus. Reviews are used to verify the relevance, sustainability and efficiency of the project, and that the project is going in a direction where the intended effects and impacts will be reached. There is, however, less emphasis on lessons learned or accountability. Project Completion Reports document the status of the project at the time of the completion of the assistance. They can be seen as a link between monitoring and evaluation (last stage of monitoring and first stage of evaluation) and an input to eventual evaluation. The main emphasis is on lessons learnt. See Annex XI. Baseline studies can be seen as a first step in the evaluation process. Baseline study is a collection of data, carried out normally during the project preparation phase, which describes and analyzes the socio-economic and other conditions in the project area. The indicators set through a baseline study are used at a later date as reference points to demonstrate that an objective of the project has been reached and that a change has occurred. 4.1.3 Integrated approach and the Logical Framework The guidelines in this document are based on the integrated approach to project cycle management. This approach consolidates the preparation and evaluation criteria used by the aid agencies represented in OECD/DAC. It helps all those in charge of evaluation to take account of the factors considered most relevant to success of a project. The elements of an intervention are structured using the logical framework. The integrated approach means that the logical framework is used to organise and analyse the information within a single project. It also means that the same elements are applied throughout the life of a project and that the documents used in each phase have the same format as shown in Annex I. The logical framework systematises the setting of objectives and assumptions for a development project and the subsequent analysis of these. It highlights causes and effects between the key elements and provides a tool to present them. The left hand column of the logical framework portrays a project’s objectives and provides a hierarchy according to which the indicators, their verification and assumptions are placed in the other columns. Figure 17: Logical framework matrix

Overall objectives

Objectively verifiable indicators Objectively verifiable indicators Objectively verifiable indicators Means

Sources of verification

Project purpose

Sources of verification

Assumptions

Results

Sources of verification

Assumptions

Activities

Costs

Assumptions Pre-conditions

4.2. Issues to be evaluated The general issues to be evaluated are relevance, impact, effectiveness, efficiency and sustainability. Each of these has a direct relation to a specific level or levels of the intervention logic and the logical framework. The compatibility factors to be evaluated relate to Finland´s development cooperation policy and include poverty reduction and promotion of human rights and democracy. The sustainability factors to be evaluated are policy environment, economic and financial feasibility, institutional capacity, sociocultural aspects, participation and ownership, gender, environment and appropriate technology. Each of the evaluation issues and factors must be addressed in every evaluation. They can be divided in categories according to their importance in a specific evaluation. A number of other evaluation issues, specific to the evaluation in question, may also exist. 4.2.1 General evaluation issues and their relation to the logical framework Definition of issues to be addressed is essential in all evaluation work. The following are the basic groups of questions to be asked: Relevance Does the project make sense within the context of its environment? Relevance concerns whether the results, purpose and overall objectives of the project are in line with the needs and aspirations of the beneficiaries, and with the policy environment of the project. Impact What has happened (or is likely to happen) as a consequence of the project? Impact concerns whether there has been a change towards the achievement of the overall objective(s) as a consequence of the achievement of the project purpose. Both intended and unintended impacts are reviewed. Effectiveness

To what extent has (or is likely to be) the project purpose been achieved, and to what extent is the achievement a result of the project? Effectiveness describes how well the results achieved have furthered the achievement of the project purpose. Efficiency Does the quantity and quality of the results of the project justify the quantity and quality of the means used for achieving them? Efficiency concerns the relation between the results and means i.e. whether the process of transforming the means into results has been cost-effective. Efficiency assessments are normally part of the planning and monitoring process. They may be included also in evaluations, especially if the evaluations cover management performance. Sustainability What has happened (or is likely to happen) to the positive effects of the project after the external assistance has (or will) come to an end? In terms of a single project, sustainability can be described as the degree to which the benefits produced by the project continue after the external assistance has come to an end. It is a central theme in all evaluation work and relates to all elements of the logical framework for a specific project. The foregoing evaluation issues have a direct relation to a specific level or levels of the intervention logic. The relations are illustrated in the Figure 18 on the following page. The same figure is presented in Finnish language in Annex IX. Figure 18: Evaluation issues and the logical framework

SUSTAINABILITY The extent to which the positive effects of the project continue after the external assistance has come to an end.

RELEVANCE The degree to which the project is justified in relation to the needs of the beneficiaries and the policy environment of the project.

IMPACT The intended and unintended impacts of the project.

EFFECTIVENESS The extent to which the project purpose has been achieved as a result of the project.

EFFICIENCY The cost-effectiveness of transforming the means into results.

MEANS

ACTIVITIES

RESULTS

PROJECT PURPOSE

OVERALL OBJECTIVE(S)

INTERVENTION LOGIC 4.2.2 Factors ensuring compatibility and sustainability The following goals which are central to Finland’s development cooperation policy are to be addressed in all evaluations commissioned by the Department of International Development Cooperation of the Ministry for Foreign Affairs: 1. Poverty reduction To reduce poverty, promoting economic growth in isolation is not sufficient. Empowerment of the poor is another important objective, since this can contribute towards a more equal distribution of wealth and power. Finland therefore emphasises the promotion of poor people’s rights and capabilities. 2. Protection of the environment Assessing environmental impacts together with the various parties involved, and the resolution of possible disputes, are important aspects in both the strategic planning of development cooperation and in project preparation, implementation and evaluation. 3. Human rights, equality and democracy Finland is committed to the promotion of equality, human rights and democracy within development cooperation. Finland therefore recognises the essential role that equal participation and good and transparent administration play in national development. Furthermore, there are eight sustainability factors to be addressed in all evaluations: 1. Policy environment Development projects are always planned and implemented within the policy environment of the country of implementation. A project must be in line with the partner country’s political, economic, environmental, social and cultural environment, and this, in turn, must be conducive to the project’s objectives.

2. Economic and financial feasibility Assessment of a development project’s financial and economic feasibility is aimed at making sure that the resources employed are being used efficiently and effectively, and that the benefits will be sustained once external support has been withdrawn. 3. Institutional capacity Adequate institutional capacity enables a country to utilise and manage its resources efficiently and effectively. In the case of a development project, institutional capacity can either hinder or promote the sustainability of the results. 4. Socio-cultural aspects Different subcultures, the relations between them, values, perceptions of ownership and justice, customs, religions, gender roles and beliefs are examples of socio-cultural factors that have a major influence on the success and sustainability of a development project. The behaviour of individuals is determined by social structures and networks of social relationships and obligations, as well as by shared knowledge and values.These factors have the power to mobilise and bind people to a development project or to make them reject it. 5. Participation and ownership Increasing ownership requires that power and control is increasingly transferred to the partner country, and within the latter, from intermediary, implementing organisations towards the intended beneficiaries. Development must be a participatory process, which means that the various stakeholders influence and share control and risk over the development initiatives, decisions and resources which affect their lives. 6. Gender The overall objective of efficient and sustainable development is only attainable when the needs and roles of both women and men are fully recognised in the planning and implementation of projects. 7. Environment All development projects have a direct or indirect effect on their natural or man-made environments. Making development sustainable requires that planners, decision-makers, implementors and evaluators understand these impacts, and operate accordingly. 8. Appropriate technology Technology is part of a society’s products, processes and institutions and is thus a major force in social and political change. In a development project, technology refers to much more than the particular items of equipment or the exact tools which are used and this has far-reaching implications on the sustainability of the benefits produced. For more details on each of the foregoing compatibility and sustainability factors see Basic principles Chapter 1.3 "Achieving sustainable development". Each of the compatibility and sustainability factors relate to all elements of the logical framework. The evaluation model presented in Figure 19 on the next page has three dimensions: the intervention logic, the evaluation issues and the factors ensuring compatibility and sustainability. Indicators describe the general evaluation issues and sustainability and compatibility factors in such terms which can be monitored and evaluated. The indicators, which were established in the project planning phase, are used throughout the project cycle, including mid-term reviews and the final evaluation. Additional indicators may, however, be established to meet the requirements of a final evaluation.

The level of importance of each of the compatibility and sustainability factors varies from one evaluation to another. A clear distinction of the factors into different categories helps the evaluator to focus on the key issues and not to waste time on assessing factors that are not central in a specific evaluation. The following categories can be used: 1. Factors which are directly related to the overall objectives, project purpose or results. 2. Factors which we hope that the intervention can influence positively. 3. Factors which we do not want the intervention to influence in a negative way. The elements of the evaluation model are also listed in the Intervention profile/performance summary-form presented in Annex IV. In evaluations, the form provides a check list and a simplified rating scale which can be developed according to the needs of the evaluation in question. Figure 19: Evaluation model

4.3. How to plan and carry out an evaluation Terms of reference determine the most important aspects of an evaluation. The general evaluation issues and the compatibility and sustainability factors must be included in the terms of reference. In addition, there are normally other evaluation issues relevant to the situation in question. Quality of evaluations depends on the methodology used and the expertise and impartiality of the evaluators chosen. Participation of different stakeholders increase the credibility of findings and usefulness of evaluation results. Demand for local evaluation capacity is increasing. Local evaluation capacity building serves both the donor’s and the partner country’s own efforts to increase transparency. Evaluation report answers the questions presented in the terms of reference.

Conclusions and recommendations must based on the findings which, in turn, should be supported by empirical evidence. 4.3.1 Terms of Reference Each evaluation must be planned and terms of reference drawn up, in order to: - define the purpose and scope of the evaluation, including identification of the users of the findings. - describe the methods to be used during the evaluation. - identify the standards against which project performance is to be assessed. - determine the resources and time required to complete the evaluation. Terms of reference also provide a formal record of agreement between the financing authorities and the evaluators. Terms of reference are usually drafted by the Ministry for Foreign Affairs’ officer-in-charge of the evaluation and representatives from the partner country. Evaluators may participate in the finalisation of the terms of reference e.g. by proposing methodological issues. When defining the issues to be evaluated, the officer-in-charge must include in the terms of reference the general evaluation issues and the compatibility and sustainability factors presented in Chapter 4.2. In addition to these, there are normally issues to be addressed, which are essential for, and relevant to, the specific situation where the evaluation takes place. The specific evaluation issues may relate to the very reason for the evaluation. For example, an evaluation issue specific to the thematic evaluation of the overseas effectiveness of expatriate advisers could be the effect of the culture shock on the advisers’ performance. Other specific evaluation issues may arise from any specific needs or problems experienced in the implementation, such as lack of monitoring information or poor management performance. Carrying out an evaluation is a process where unforeseen issues to be addressed may arise during the exercise. Inclusion of these issues in the scope of an evaluation which is already being carried out, is decided mutually by the different stakeholders, and additional resource allocations are made accordingly. The following are the main items normally to be included in the terms of reference for an evaluation. More details can be found in Annex X "General format for terms of reference for evaluations". 1. Subject of the evaluation 2. Background of the evaluation 3. Evaluation issues 4. Compatibility and sustainability 5. Methodology 6. Evaluation team 7. Reporting In the Department of International Development Cooperation of the Ministry for Foreign Affairs, the terms of reference for all evaluations should be cleared with the Central Evaluation Unit.

4.3.2 Planning of evaluation requires expertise When planning an evaluation, one must take into account the general requirements; impartiality, independence, credibility and usefulness, as well as the purpose of the evaluation and the resources available. Scientific approach improves the quality of information but may not be justified because of time constraint, nature of the evaluation etc. Also, when scientific approach is justified, it must be kept in mind that evaluation is, above all, a tool for decision-making, and decision makers want the findings, conclusions and recommendations in a concise form. Credibility of an evaluation depends on the expertise and independence of the evaluators and the degree of transparency of the evaluation process. The evaluators should have the required expertise to make relevant and realistic conclusions and recommendations based on the findings. The reliability of the findings, in turn, depends on the empirical evidence the evaluators can show to support the findings, including sources of information. Evaluation reports must show the evidence and clearly distinguish between the findings and conclusions/recommendations. Besides the skills of the evaluators, the selection of appropriate methods for data collection and analysis affects the quality. A wide range of relevant methodologies and techniques is presented in evaluation and scientific literature. Only a few examples of issues to be considered in the selection of an appropriate methodology are briefly described here. The data collection for an evaluation can seldom cover the whole range of stakeholders and beneficiaries who are affected by the intervention. Defining how to choose a sample becomes essential; its size and nature determine whether it is representative or not. The sample can be chosen in many different ways, ranging from a random sampling, where every unit has an equal chance to be included in the sample, to a selection made by the evaluator. Identifying the standards against which performance is to be evaluated is important. Without any comparison with a control group or other interventions, the result is quite predictable: the project had some impact on various aspects but could have done better in others. Using a control group which has not been affected by the project and comparing it with a group of project beneficiaries before and/or after the project makes the comparison possible. Since the use of a control group is often costly and difficult, it is more common to study only the situation of the beneficiaries but to do it before and after the intervention. Availability of baseline data and the use of indicators are essential for such evaluation. To complete the evaluation, its results should still be compared with other similar projects and/or with the general developments outside the area of influence of the intervention. For the collection of data, a number of methods can be used. Traditional quantitative methods include sample surveys where quantifiable aspects are measured using standardised questionnaires and interviews. The method is often used in connection with the baseline studies and requires a substantial amount of skilled resources and time. Data collection on qualitative aspects is often carried out using direct observation by the evaluators. Since the evaluator cannot spend as a long time with the beneficiaries as e.g. in anthropological research, the methods used are simplified. Group interviews, key informant or resource person interviews are cheap and quick to undertake. Choosing the ones to be interviewed and interpreting their views correctly, however, requires a lot of knowledge and skills from the evaluator. In evaluations commissioned by the Department of International Development Cooperation, the methodology is normally only outlined in the draft terms of reference. Evaluators are asked to elaborate the methodology in their proposal on how to carry out the evaluation. Evaluators also propose resources required for using the proposed methodology. The team leader of the evaluation prepares a work plan in consultation with partner country and other stakeholders and the management team of the intervention to be evaluated. When the methodology and its resource requirements have been agreed upon, the terms of reference are finalised and signed. The evaluation work plan may be included in the terms of reference or it may be finalised later e.g. in the partner country. 4.3.3 Participation improves quality

To enhance ownership and sustainability, Finland aims at increasingly shifting the responsibility for planning, implementation and evaluation to the partner country and her representatives at different levels. Participatory development stands for a partnership which is built upon the basis of a dialogue among the various stakeholders, during which the ‘agenda’ is set jointly, and local views and indigenous knowledge are deliberately sought and respected. This implies negotiation rather than the dominance of an externally set project agenda. Thus, people become actors instead of being simply beneficiaries (OECD/DAC Expert Group on Aid Evaluation). In participatory evaluation, a variety of beneficiaries and other stakeholders actively take part in the different phases of the evaluation process: the definition of the purpose and scope of the evaluation, the selection of the methods, the gathering and analysis of data, and decisions made based on the evaluation results. In participatory data collection and analysis, techniques such as group discussions, key informant interviews and field workshops are used to gather both quantitative and qualitative information. The external evaluator is a neutral observer who assesses project processes and effects in local communities. Although the main responsibility for the evaluation process remains with the external observer, the above-mentioned and other participatory techniques increase the influence of different stakeholders on the evaluation findings and improve their credibility and usefulness. The conclusions and recommendations, which are made based on the findings, are shared with the different stakeholders. The stakeholders then jointly decide on the measures to be taken as a result of the evaluation. 4.3.4 Demand for local evaluation capacity is increasing As in planning and implementation, participation in evaluation should not be seen only as a means to improve the quality but also as a valuable end in itself. By participating in evaluations of donor interventions, local organisations and individuals gain experience and skills which benefit national programmes. The demand for local evaluation capacity is increasing because of the partner countries’ efforts to make their administration more transparent. The shift from project type of donor interventions towards programme assistance further adds to this demand. In programme assistance, the partner country organisations have increased responsibility not only for planning and implementation but also for evaluation. Local evaluation capacity building is necessary to meet the partner country’s own and the donor community’s requirements. Arranging joint evaluations and coordinating evaluation programmes with partner countries and other donors is essential. Instead of transferring ready-made evaluation systems, which usually have slight chances of being adopted and used, partner countries should be provided with consistent evaluation methodologies and terminologies, based on which they can build their own evaluation systems. 4.3.5 Evaluation report - the first step Evaluation report must, in principle, answer the questions presented in the terms of reference. The format follows that of the terms of reference and differs from it depending on the issues relevant for the evaluation in question. The same format is used for final evaluations, which take place after the donor intervention has already ended, and for mid-term reviews. The scope in mid-term reviews is, however, different (see Chapter 4.1.2) and the evaluators may have to rely more on indicators which give an "early signal" of progress. For example, the project purpose may describe the beneficiaries´ use of the project services such as preventive health care. The indicators measure whether the beneficiaries have access, are using, and are satisfied with the project services. This gives an indication that the project is offering relevant services and the project purpose is likely to be met. In case the project internal monitoring system does not provide this kind information, the mid-term review may carry out a survey using local specialists. 1. Executive summary. 3-5 pages clearly separated from the rest of the report. Especially, conclusions and recommendations.

2. Subject of the evaluation. Brief history of the intervention, changes in the project environment and their effects on the intervention, materialisation of risks and assumptions, etc. 3. Background of the evaluation. Purpose of the evaluation, methodology used, limitations of the evaluation, etc. 4. Evaluation issues 4.1 General evaluation issues: 4.1.1 Correspondence of the project with the priority needs of the beneficiaries i.e. relevance. 4.1.2 Assessment of impact of the project towards its long-term objectives. 4.1.3 Extent of achievement of the project purpose as a consequence of the project, i.e. effectiveness. 4.1.4 Assessment of the efficiency of the implementation. 4.2 Specific evaluation issues 5. Factors ensuring sustainability and compatibility 5.1 Compatibility with the strategic goals for Finnish development cooperation: reduction of poverty, promotion of equality, democracy and human rights, etc. 5.2 Policy environment 5.3 Economic and financial feasibility 5.4 Institutional capacity 5.5 Socio-cultural aspects 5.6 Participation and ownership 5.7 Gender 5.8 Environment 5.9 Appropriate technology 6. Conclusions and recommendations. Suggestions for operational improvements and developmental lessons learnt. Following the integrated approach to project cycle management, the standard format for evaluation reports is compatible with the project document and monitoring report formats, as shown in Annex I. Both positive and negative findings are reported under the Chapters 4 and 5 of the report. To the extent possible, the findings are supported with empirical evidence. Conclusions and recommendations are based on the findings and clearly separated from them. To ensure that a conclusion or recommendation is relevant and realistic, it must be discussed with the stakeholders concerned before it is presented. Operational recommendations which are relevant only for the intervention in question are separated from the lessons learnt, which are relevant also for other interventions or for designing future policies.

4.4. What to do with the evaluation report Evaluation feedback process has two main purposes: i) dissemination of conclusions and recommendations, and ii) operationalisation of the lessons learnt. Different techniques such as seminars, summaries and synthesis reports are used to disseminate information. The operationalisation of the lessons learnt means that there is a change in organisational behaviour as a result of the evaluation. Different institutional mechanisms are used to approve the recommendations, and to take action upon the approval. 4.4.1 Dissemination of conclusions and recommendations Evaluation feedback is a dynamic process which includes both the presentation and dissemination of evaluation results and their application to new or existing development interventions. Presentation of the evaluation report is one of the initial stages in the feedback process. Based on the report, conclusions, recommendations and lessons learnt are disseminated using different techniques. The dissemination techniques used by the Department of International Development Cooperation of the Ministry for Foreign Affairs include organisation of evaluation seminars during, and after, the evaluation exercise. In the seminars, evaluation results are presented by the evaluators and discussed by different stakeholders. If organised during the evaluation exercise, the seminars can be used for planning of evaluation work. Participation of partner country stakeholders requires that seminars are organised also in the partner country. Other dissemination techniques used include printing of evaluation summaries, bibliographies and synthesis reports. While the evaluation seminars are aimed for the stakeholders who have a special interest and/or an involvement in the interventions, the printed material is meant for broader circulation in and outside Finland. This includes the media and the public in partner countries and in Finland as well as other donors. The officer-in-charge of the evaluation must ensure that the terms of reference are clear in who will use the conclusions and recommendations, and for what purpose. This, together with an indication of seminars to be organised or special summaries to be written, gives the evaluators a basis for structuring their conclusions and recommendations. The quality of the evaluation report, both in terms of contents and presentation, determines how easy or difficult the dissemination process will be thereafter. Concise but profound reports with good summaries, and organisation of participatory presentation seminars, contribute to an effective dissemination. 4.4.2 Approval and operationalisation of recommendations The application of the evaluation results to existing or new interventions is in fact one of two main purposes of evaluation. As stated earlier, all evaluation efforts are wasted if there is no change in organisational behaviour as a consequence. Different institutional mechanisms are used by the Ministry for Foreign Affairs of Finland for operationalising the evaluation results. In case of mid-term reviews, where the emphasis is in giving direction to the project, there is instant feedback through different levels of project management, such as local steering committees or annual supervisory board meetings. They study, approve or disapprove, and put into action the recommendations. Responsibility for bringing relevant information from mid-term reviews to the attention of other projects lies with the Ministry´s officer-in-charge of the mid-term review and the partner country authorities. In case of final evaluations, the emphasis is more on lessons learnt for new projects and for future strategies. The Central Evaluation Unit of the Ministry for Foreign Affairs plays a central role: all those involved in planning new projects, or in designing strategies, find relevant evaluation information there. As an example, planners of a new intervention must consult evaluation reports made earlier on similar interventions.

Evaluation results are also used in staff training, both on the level of individual interventions and within organisations in general. As an example, a thematic evaluation on gender aspects is followed by a series of training workshops organised for the staff of the Ministry for Foreign Affairs, partner country organisations and project management and support personnel. Whatever the institutional mechanism for evaluation feedback is, it must make a clear distinction between the recommendations made by the evaluators, the approval of the recommendations by the concerned stakeholders, and the action taken upon the approved recommendations. How to decide on the Ministry for Foreign Affairs’ approval, disapproval or a point of view with regard to a specific recommendation, depends on the nature of the evaluation and decisions to be taken. As a general rule, it is the responsibility of the officer-in-charge of the evaluation to obtain necessary viewpoints on the recommendations within the Ministry, to report these to other stakeholders concerned and to propose to them how joint decisions will be made in practice. In reporting the Ministry’s viewpoints, the officer-in-charge uses a short memorandum containing issues such as: - the Ministry’s general comments on the recommendations. - comments concerning the implementation of the evaluation. - concrete recommendations for action, indicating time and responsibility for their implementation from the Ministry’s point of view. - how the implementation of recommendations should be followed-up.

ANNEX I INTEGRATED APPROACH TO PROJECT CYCLE MANAGEMENT PROJECT DOCUMENT FORMAT
1. 2. Summary Present situation 2.1 Government and sectoral policies 2.2 Background studies 2.3 Problems to be addressed 2.4 Stakeholders and beneficiaries Definition of the intervention 3.1 Overall objectives (and indicators) 3.2 Project Purpose (and indicators) 3.3 Results (and indicators) 3.4 Activities Assumptions and risks Compatibility and sustainability 5.1 Compatibility with the strategic goals 5.2 Policy environment 5.3 Econ./financial feasibility 5.4 Institutional capacity 5.5 Socio-cultural aspects 5.6 Participation and ownership 5.7 Gender 5.8 Environment 5.9 Appropriate technology Implementation 6.1 Approach 6.2 Organisation 6.3 Tentative timetable 6.4 Budget Monitoring Evaluation

MONITORING REPORT FORMAT
1. 2. 3. Summary Proposals for changes in the project, if any, and their justification Assessment of the intervention 3.1 Correspondence with the present needs i.e. relevance 3.2 Presumed impact of the project 3.3 Extent of achievement of project purpose i.e. effectiveness 3.4 Extent of achievement of results 3.5 Carrying out of activities Possible changes in the project environment, including materialisation of assumptions and risks, and their effects on the project Compatibility and sustainability 5.1 Compatibility with the strategic goals 5.2 Policy environment 5.3 Economic and financial feasibility 5.4 Institutional capacity 5.5 Socio-cultural aspects 5.6 Participation and ownership 5.7 Gender 5.8 Environment 5.9 Appropriate technology Assessment of the efficiency of the implementation

EVALUATION REPORT FORMAT
1. 2. Executive summary Subject of the evaluation, including brief history of the intervention, changes in the project environment and their effects on the intervention, etc. Background of the evaluation: its purpose, methodology used, limitations etc. Evaluation issues 4.1 General evaluation issues: 4.1.1 Correspondence with the present needs i.e. relevance. 4.1.2 Impact of the project. 4.1.3 Extent of achievement of project purpose effectiveness 4.1.1 Assessment of the efficiency of the implementation 4.2 Specific evaluation issues Compatibility and sustainability 5.1 Compatibility with the strategic goals 5.2 Policy environment 5.3 Economic and financial feasibility 5.4 Institutional capacity 5.5 Socio-cultural aspects 5.6 Participation and ownership 5.7 Gender 5.8 Environment 5.9 Appropriate technology Conclusions and recommendations: suggestions for operational improvements and developmental lessons learnt

3.

3.

4.

4.

4. 5.

5.

5.

6.

6.

6.

7. 8.

ANNEX II

GENERAL FORMAT FOR TERMS OF REFERENCE
1. INTRODUCTION Explains the reason for the exercise and its general nature. OBJECTIVES OF THE EXERCISE Why the exercise is carried out? What is expected as its result? BACKGROUND OF THE EXERCISE Brief history and present situation of the issue(s) to be studied. ISSUES TO BE STUDIED Scope of work: main issues to be studied, their inter-relationships, any priorities or special emphases. Implications of the logical framework methodology and integrated project cycle management on the issues to be studied. PLAN OF WORK Mode of work: methodology, key information sources to be used, etc. REQUIRED EXPERTISE Personal qualifications required to successfully carry out the exercise. TIMETABLE AND REPORTING Timing of different phases of the exercise, language and number of copies of the report, how the report will be commented upon, and approved, etc. MANDATE Parties with which to cooperate, authorisation to make commitments, etc.

2.

3.

4.

5.

6.

7.

8.

ANNEX III

PROJECT DOCUMENT FORMAT
Programme/Project Fact Sheet 1. 2. Summary Present situation 2.1 Government and sectoral policies 2.2 Background studies 2.3 Problems to be addressed 2.4 Stakeholders and beneficiaries Definition of the intervention 3.1 Overall objectives (and corresponding indicators) 3.2 Project Purpose (and corresponding indicators) 3.3 Results (and corresponding indicators) 3.4 Activities Assumptions and risks Compatibility and sustainability 5.1 Compatibility with the strategic goals for Finnish development cooperation 5.2 Policy environment 5.3 Economic and financial feasibility 5.4 Institutional capacity 5.5 Socio-cultural aspects 5.6 Participation and ownership 5.7 Gender 5.8 Environment 5.9 Appropriate technology Implementation 6.1 Approach 6.2 Organisation 6.3 Tentative timetable 6.4 Budget Monitoring Evaluation

3.

4. 5.

6.

7. 8.

ANNEXES Logical framework matrix (unless presented under section 3.) List of other documentation available Job descriptions

ANNEX IV

INTERVENTION PROFILE/PERFORMANCE SUMMARY
COUNTRY/ TITLE OF INTERVENTION: STATUS (in planning/implementation or completed): Rating scale: -2=highly unsatisfactory, -1= unsatisfactory, +1=satisfactory, +2=highly satisfactory, N/A= not applicable or not known -2 -1 +1 +2 N/A Comments RELEVANCE Correspondence with beneficiaries´ priority needs IMPACT Impacts on beneficiaries and other affected groups EFFECTIVENESS Contribution of results to achieve project purpose EFFICIENCY Cost-effectiveness of converting means into results POLICIES Reduction of poverty Promotion of equality and human rights Promotion of democracy and good governance SUSTAINABILITY Correspondence with local policies Development of local institutional capacity Strengthening of local human resources Economic/financial soundness Suitability to local socio-cultural context Participation of local stakeholders Optimal participation of men and women Impacts on environment Appropriateness of technology ASSUMPTIONS Likelihood of assumptions to materialize IMPLEMENTATION Achievement of planned results Compliance with budget Availability of baseline data Use of indicators MAIN CONCLUSIONS:

ACTION NEEDED, AND BY WHOM:

OTHER COMMENTS:

FILLED IN BY:

DATE AND PLACE:

ANNEX V
INTERNAL MANAGEMENT &MONITORING SETUP SUPERVISORY BOARD MEETING (SVB) GENERAL MANDATE SPECIFIC MONITORING PRACTICES Monitoring review based on the annual monitoring report; rating of the project using the performance summary form. MEMBERS RESPONSIBLE FOR MONITORING Finnish side Representative from the Ministry for Foreign Affairs in Finland who attends the SVB Partner country side Representative(s) from the Ministry of Cooperation and the sectoral ministry, who are not personally involved in the project implementation. Representative(s) from the sectoral ministry who are involved in project implementation; project director and key specialists of the management team; representatives of the beneficiaries. Partner country project personnel TIMING

STEERING COMMITTEE MEETING (SCM)

Policy issues and amendments to the project document; approval of the annual monitoring report; approval of the work plan and budget. Responsible before the SVB for the project management and the achievement of the project purpose.

Monitoring review takes places once per year in connection with the SVB or in case of a specific need 4 times per year or in case of a specific need

Finalises the work plans, quarterly progress reports and annual monitoring reports, and presents them to the SVB; rating of the project using the performance summary form.

Representative from the Embassy of Finland; project coordinator and key specialists of the management team.

MANAGEMENT TEAM

Daily project management and administration

BENEFICIARIES

Implementation of and participation to the project activities; utilisation of the project services.

Design/installation of the internal monitoring system including both quantitative and qualitative indicators. Drafts the work plans and quarterly progress reports, and compiles the annual monitoring reports. Continuous supply of monitoring information to the other levels.

Expatriate project personnel

Weekly meetings or when necessary

If required, project personnel and/or ad hoc monitoring specialists may assist the beneficiaries.

If required, project personnel and/or ad hoc monitoring specialists may assist the beneficiaries.

Continuously and/or ad hoc

ANNEX VI
EXAMPLE OF A WORK PLAN FORMAT

Logical framework for the project:
Overall Objectives Project purpose Result 1 Indicators for result 1 Sources of verification Activities for result 1 Means Costs Indicators for overall objectives Indicators for project purpose Result 2 Indicators for result 2 Sources of verification Activities for result 3 Means Costs Sources of verification Sources of verification Result 3 Indicators for result 3 Sources of verification Activities for result 3 Means Costs Assumptions Assumptions Assumptions

Annual work plan for result 1 of the project:
Result 1: Planned extent of achievement of result 1 in year X:

Indicators:

Planned extent of achievement in year X using indicators: Quantitative: Qualitative:
Jan ----Feb ----xxx Mar ----xxx Apr ----xxx xxx xxx ----xxx xxx ----xxx ----xxx ----xxx ----xxx xxx xxx xxx --------xxx ----xxx ----xxx ----May ----xxx Jun ----xxx xxx Jul -----

Sources of verification:

Quantitative: Qualitative: Activities
Process 1 Task 1.1 Task 1.2 Task 1.3 Task 1.4 Process 2 Task 2.1 Task 2.2 Task 2.3 Process 3 Task 3.1 Task 3.2 Task 3.3 Task 3.4 Task 3.5 Operational responsibility

Aug -----

Sep

Oct

Nov

Dec

xxx xxx ----xxx xxx -----

-----

-----

----xxx xxx -----

xxx -----

xxx -----

----xxx

xxx xxx

xxx xxx

xxx

xxx

xxx

Separate work plans are presented for results 2 and 3 of the project.

ANNEX VII

REPORTING, MONITORING AND EVALUATION ISSUES
Monthly Progress Report Overall Objectives + Assumptions
CHANGE

Quarterly Progress Report

Annual Monitoring Report

Evaluation Report

IMPACT

IMPACT

Project Purpose + Assumptions
UTILIZATION

EXTENT OF ACHIEVEMENT EFFECTIVENESS EFFECTIVENESS

Results + Assumptions
ACTION

EXTENT OF ACHIEVEMENT

EXTENT OF ACHIEVEMENT

Relevance

Activities + Assumptions
ALLOCATION

REALISATION

REALISATION

EFFICIENCY

EFFICIENCY

Means + Pre-conditions

DELIVERY

(DELIVERY) Sustainability

ANNEX VIII

Planned Actual budget expenditure 2nd quarter 2nd quarter TECHNICAL ASSISTANCE (TA) Personnel costs: Long-term Short-term Reimbursable TA costs: SUB TOTAL TA COMPONENT 1 Training, communication Materials SUB TOTAL COMP.1 COMPONENT 2 Construction Training Equipment SUB TOTAL COMP.2 COMPONENT 3 Micro projects Facilitators Credit schemes SUB TOTAL COMP.3 OTHER COSTS Recurrent costs Vehicles, office equipment Financing costs SUB TOTAL OTHER COSTS CONTINGENCIES (5%) TOTAL FINLAND TOTAL PARTNER COUNTRY GRAND TOTAL

Difference budgetactual

Used %

Original budget 1997

Accumulated expenditure 1997

Available for july-dec

Original budget 1997 TA COMP.1 COMP.2 COMP.3 960 000 TA 720 000 COMP.1 1 050 000 COMP.2 650 000 COMP.3 360 000 OTH. COSTS 190 000 CONTING. 3 930 000

Accumulated expenditure 1997 479 365 376 367 524 232 339 280 176 575 0 1 895 819

200 000 20 000 20 000 240 000 25 000 115 000 140 000 125 000 50 000 80 000 255 000 25 000 20 000 80 000 125 000 50 000 35 000 4 500 89 500 42 500 892 000 257 000 1 149 000

200 000 17 000 16 879 233 879 22 600 117 563 140 163 119 821 53 647 79 854 253 322 18 654 16 000 85 000 119 654 49 851 38 921 4 452 93 224 0 840 242 256 389 1 096 631

0 3 000 3 121 6 121 2 400 -2 563 -163 5 179 -3 647 146 1 678 6 346 4 000 -5 000 5 346 149 -3 921 48 -3 724 42 500 51 758 611 52 369

100 % 85 % 84 % 97 % 90 % 102 % 100 % 96 % 107 % 100 % 99 % 75 % 80 % 106 % 96 % 100 % 111 % 99 % 104 % 0% 94 % 100 % 95 %

800 000 80 000 80 000 960 000 120 000 600 000 720 000 500 000 200 000 350 000 1 050 000 220 000 80 000 350 000 650 000 200 000 140 000 20 000 360 000 190 000 3 930 000 1 120 000 5 050 000

400000 39 500 39 865 479 365 57 800 318 567 376 367 239 603 104 775 179 854 524 232 115 280 34 000 190 000 339 280 97 821 68 754 10 000 176 575 0 1 895 819 578 964 2 474 783

400 000 40 500 40 135 480 635 62 200 281 433 343 633 260 397 95 225 170 146 525 768 104 720 46 000 160 000 310 720 102 179 71 246 10 000 183 425 190 000 2 034 181 541 036 2 575 217

OTH. COSTS CONTING. TOTAL

Original budget 1997/ Finnish contribution
OTH. COSTS 9% CONTING. 5% TA 24 %

Accumulated expenditure 1997/Finn. contr.
OTH. COSTS 9% CONTING. 0% TA 25 % COMP.3 18 %

COMP.3 17 %

COMP.1 18 %

COMP.1 20 % COMP.2 28 %

COMP.2 27 %

ANNEX IX

EVALUOINTIKÄSITTEET JA LOOGINEN SUUNNITTELUKEHIKKO
KESTÄVYYS Miten pysyviä hankkeen aikaansaamat tulokset ja muutokset ovat?

TARKOITUKSENMUKAISUUS Miten hyvin hanke vastaa paikallisia tarpeita ja kuinka se on edistänyt tavoiteltua muutosta?

VAIKUTUKSET Mitkä ovat hankkeensuunnitellut ja suunnittelemattomat vaikutukset?

TULOKSELLISUUS Miten hankkeen tulokset ovat edistäneet sitä tarkoitusta jota varten hanke käynnistettiin?

TEHOKKUUS Miten tehokkaasti hanke on lyhyellä tähtäimellä muuntanut voimavaroja tuloksiksi? (ns. panos/tuotos-suhde)

VOIMAVARAT

TOIMINNOT

TULOKSET

TARKOITUS

KEHITYSTAVOITE Minkälaista muutosta halutaan?

Mitä voimavaroja käytetään?

Mitä tehdään?

Mitä saadaan aikaan?

Mikä on hankkeen tarkoitus?

HANKKEEN PÄÄMÄÄRIEN HIERARKIA

ANNEX X

GENERAL FORMAT FOR TERMS OF REFERENCE FOR EVALUATIONS
1. Subject of the evaluation Explains the subject of the evaluation. E.g. a project, programme or a policy. Describes briefly its history and general context e.g. previous phases, sectoral policies, and the institutional framework. Describes the main components of the logical framework. Overall objectives, project purpose, activities, means and assumptions (in matrix form, if available). Describes the availability of monitoring information. Baseline data, indicators, EIA-, gender- etc. analyses, and the findings of possible previous reviews and evaluations. 2. Background of the evaluation Explains why the evaluation is carried out. E.g. there may have been a formal decision to do so after a certain number of years, or the seemingly poor (or excellent) project performance has raised the need for a review, or there is a need for new orientation from the point of view of one or some of the stakeholders. Describes who will use the the evaluation results and for what purpose. E.g. one or some of the stakeholders will use them for accountability, or for decision making to continue/phase down/expand etc., or to provide lessons learnt and recommendations for improvements concerning policies/other interventions/etc. 3. Evaluation issues Describes first the general evaluation questions to be answered: Relevance. E.g. does the intervention make sense within the context of its environment? Has it addressed the needs and aspirations of the beneficiaries? Have changes in the environment affected its relevance? Impacts. E.g what has happened (or is likely to happen) as a consequence of the intervention? What are the positive or negative, intended or unintended impacts of the intervention on the beneficiaries and on the institutional level? What are the impacts of external factors? Effectiveness. E.g. to what extent has (or is likely to be) the purpose of the intervention been achieved, and to what extent is the achievement a result of the intervention? What is the progress from the beginning (baseline) towards the agreed project purpose (as determined in the project document)? Efficiency. E.g. does the quantity and quality of the results justify the quantity and quality of the means used for achieving them? How cost-effectively have the means been converted into results?

1 (3)

ANNEX X
Sustainability. E.g. what has happened (or is likely to happen) to the positive effects of the intervention after the external assistance has (or will) come to an end? Describes then the specific evaluation questions to be answered: E.g. implementation issues such as how well has the project been managed internally and/or externally (management team, local and Finnish government officials, other stakeholders)? Has it been coordinated with other projects? How well is the monitoring organised? Is the utilisation of funds transparent? 4. Compatibility and sustainability Describes the questions to be answered concerning compatibility. E.g. How are the goals which are central to Finland’s development cooperation policy taken into account in the planning and implementation? How have they affected the intervention? Poverty reduction. E.g. How are poor people’s rights and capabilities promoted in the project? Protection of the environment. E.g. Are environmental impacts assessed together with the various parties involved? Human rights, equality and democracy. E.g. How are equality, human rights and democracy promoted in the project? How is transparency of administration increased? Describes the questions to be answered concerning sustainability. E.g. How have the factors ensuring sustainability been taken into account in the planning and implementation? How have they affected the intervention? What have been the effects and impacts of the intervention on the sustainability issues? Policy environment. E.g. Is the project in line with the partner country’s policy environment, and is this in turn conducive to the project’s objectives? Economic and financial feasibility. E.g. Are the resources employed used efficiently and effectively, and are the benefits sustained once external support has been withdrawn? Institutional capacity. E.g. Does the institutional capacity of the partner country enable her to manage the project efficiently? Is the institutional capacity strengthened to promote the sustainability of the results? Socio-cultural aspects. E.g. Are socio-cultural factors taken into account to mobilise and bind people to the project? How does the project affect socio-cultural values? Participation and ownership. E.g. Who has power and control over the project? Who participates in decision-making? Gender. E.g. Are the needs and roles of both women and men fully recognised in the planning and implementation? How does the project affect the relations between women and men? Environment. E.g. What effects does the project have on natural or man-made environments? Is the project managed in environmentally sound way?

2 (3)

ANNEX X
Appropriate technology. E.g. Is the technology used compatible with the available human resources? Is it socially acceptable? Although all the foregoing issues must be addressed in every evaluation, the level of detail with which each one of them is dealt varies from one evaluation to another (see Chapter 4.2.2 “Factors ensuring compatibility and sustainability”, page 68). For more details on each of the factors see Chapter 4.2.2 and Chapter 1.3 “Achieving sustainable development”. 5. Methodology Explains what is the basic methodology for the evaluation. E.g. whether it is based on documentation available, on interviews with key persons, on field work including data collection and analysis, or on all of these. Suggests the degree of participation. E.g. defining whose representatives should be included in the evaluation team and specifying who will name them. Describes a tentative timetable. E.g. the time allocated for preparations, field work and reporting. 6. Evaluation team Describes the general requirements for the evaluation team. E.g. independence, representation of both genders, ability to work as a team. Outlines the special technical expertise required. E.g. the approximate number and type of technical specialists. Outlines the special evaluation expertise required. E.g. the qualifications of the team leader, including his or her methodological knowledge, team leader and evaluation experience. 7. Reporting Explains how and when the conclusions and recommendations are presented. E.g. workshops or seminars to be organised in the partner country and Finland and the types of reports to be prepared (draft, final).

3 (3)

ANNEX XI

PROGRAMME COMPLETION REPORT / FINAL REPORT
GENERAL Support to programme can be discontinued when agreed outputs are achieved, the time frame has expired and the available foreign financing has been fully utilised. Support may also be terminated if there are major implementation problems, and measures to rectify the situation, as laid down in the Agreement, have not led to an acceptable solution for the financing agency and/or the partner country govenrment. In cases where programme enabling environment; politically, economically or socially has been deteriorated and does not provide fruitful climate for continuation of the programme; foreign financing of the programme may be discontinued. In all cases, a Programme Completion Report (PCR) shall be issued, which refer back to the Programme Document and other agreements as appropriate. As a minimum it should provide a record of programme outputs, activities and inputs. It should also discuss impact, relevance and sustainability, as well as fulfilment of obligations and regulations laid down in mutual agreements. The preparation of the PCR should be initiated 6-3 months before the completion of the project. PURPOSE OF THE REPORT The main purpose of Programme Completion Reports is to provide an assessment of the effectiveness of the implementation phase of programmes. The PCR assesses the extent to which the planned outputs have been achieved and whether this has been done with the time scale and cost originally planned. This information enables all parties (financing agency, partner country government, supplier of services or consultant) to draw conclusions and lear lessons from its experience of implementation. The report also contributes to the accumulation and dissemination of experiences and provides a basis for determining whether a terminal or ex post evaluation is warranted. The Programme Completion Report has other purposes too: • it provides an initial opportunity to assess the wider and longer term impact of the project or programme. This means making an initial assessment of the extent to which the wider (development) and immediate objectives identified in the Programme Document are likely to be met. However it is important to recognise that in most cases a reliable impact assessment of this kind can only be made some years after the inputs have been completed, and that the primary tool for assessing impact is the evaluation system. It should be emphasised that a base line study and proper monitoring of the programme by using measurable development indicatores is the basis for any analysis • it provides a chance for the Ministries concerned to identify activities that may be particularly suitable for evaluation at some future date • it can act, alongside the Programme Document and Annual Workplans, as an aid to project monitoring. Content of PCR

1 (5)

ANNEX XI
The PCR shall be based on existing information, i.e. as a rule no special studies or surveys should be carried out. The data base will typically comprise: • programme document and country agreement • project work plans and logical framework where applicable • programme progress and review reports • evaluation reports. As a minimum a PCR must deal with the following: 1. The establishment of the components and activities in relation to those indicated in the Programme Document and implementation plan including reasons for possible divergences. 2. A description of the main budget lines of the original budget, budget revisions and actual expenditure. 3. A review of the partner country financial contribution to the programme as agreed in the programme document, annual consultations and country agreement. 4. An assessment of the fulfilment of the programme objectives, or the probabilities of reaching these objectives, and if relevant the reasons why the objectives are not judged to the fulfilled as foreseen. 5. An assessment of the future sustainability of the programme. 6. Experiences of general nature gained in connection with the implementation of the programme. Standard format The report must be prepared in accordance with the general format attached on the following pages. Please note that the Ministries staff responsible for the project of the Embassy of Finland staff may indicate another, more suitable format for the report.

2 (5)

ANNEX XI

FORMAT FOR PROGRAMME COMPLETION REPORT
Basic programme data (Prgramme / Project Fact Sheet) as background information 1. Summary and lessons learned • The summary should in schematic form address outputs produced, effects and impact as well as programme efficiency, effectiveness and relevance. • Important lessons of general interest and to the design and implementation of similar programmes in future. 2. Programme background • Brief description of the sector concerned focusing on activities and constraints relevant to the programme. Explain further how the programme fitted into the development plans of the Government. 3. Programme design • Briefly discuss the programme strategy and design (internal logic) including the management set-up and institutional framework. 4. Inputs • Provide statement of accounts and relate it to the budgets • Provide information of programme financing from both foreing (Finland and others) and local sources • Provide an overview of the use of Technical Assistance Personnel, institutional partner or consultant where relevant • Provide information on efforts for local institution and capacity building • Supply an overview of the use of equipment where relevant (inventory list as an attachment) • Analyse both the Finnish financed outputs and the partner government financed outputs. Have they been: • Appropriate • Sufficient • Timely • Well-coordinated • Efficiently procured and delivere 5. Activities • List main activities 6. Outputs • List and compare outputs as planned and as produced 7. Efficiency

3 (5)

ANNEX XI
• Assess where annual work plans are realistic and adequately prepared, degree of adherence to action plans, quality of programme organization and management, financial delivery dates of project components, extent of Government interest and policy support, if any major project reformulation took place during the implementation period, and if any external factors (outside the control of project management) have strained project implementation. 8. Fulfilment of objectives • are the development objectives still relevant. • to what extent has the project achieved its development objectives. • For each development and immediate objective show relevant indicators which measure achievements • What other facts - if any - have been observed which can confirm that the development objectives are or are being attained? 9. Sustainability • Assess the prospects and condition for future sustainability of project activities in particular with regard to funding, maintenance and institutional aspects / management and in terms of benefits to the target group. 10. Identification of alternatives • Is there, or would have been, a more efficient, effective or relevant way to approach the problem being addressed by the project? 11. Further analysis Possible need for further analysis, impact study or evaluation 12. Need for further assistance Possible need for further assistance 13. Policy relevance Consistency with policy on poverty reduction: was the programme a targeted poverty programme was the programme focused on poverty issues has there been in the programme some interventions to policy matters did the poverty matter not explicitly appear in the programme plan Consistency with policy on environmental sustainability has the programme increased the capacity to manage local environment has the programme strengthened local capacity to solve environmental problems has the programme contributed towards participating in solving the global environmental problems are the environmental aspects not featuring in the programme at all Consistency with policy on human rights, democracy and good governance has the programme strengthened the role and capacity of civil society has it strengthened the democratic institutions and responsible government

4 (5)

ANNEX XI has it strengthened the competence of public sector has it contributed to the capacity of human rights institutions has it influenced the leaders on all leves to respects rights, rule democratically and govern effectively have these principles not featured in the programme at all

Consistency with the policy on gender equality has the programme increased women’s participation in decision making has the programme improved women’s income levels and economic condition has the programme improved women’s access to basic health and family planning services has the programme improved women’s and girls access to basic educational services and vocational training has the programme been able to protect and promote the human rights of women have the gender guestions not systematically been addressed in the programme at all 14. General experiences gained / lessons learned Positive aspects which may be replicable / negative aspects which may be avoided in future / other general lessons In connection with programme implementation In connection with the fulfilment of obligations and regulations stipulated in the Agreement Problems experienced

Annexes: References List the most relevant reference documents, e.g.: • Programme Document and Agreement • Agreed Minutes of annual consultations • Reports, including statement of accounts • Last annual progress report and audit report • Reports from reviews and evaluations Inventory list Note: Programme Completion Report / Final Report is signed by the programme coordinator and countersigned by the Finnish team-leader if not the same person.

5 (5)

ANNEX XII

PROGRAMME/PROJECT FACT SHEET
Project Title Project Number Sector Sub-Sector Focus

Geographical Coverage and Project Site

Duration Starting Date Project Financing

Total Ministry for Foreign Affairs, Finland Partner Country Other financing

FIM FIM

Intended Beneficiaries

Project Purpose

Overall Project Objective

Institutional Framework

Competent Authorities