Setting up a monitoring and evaluation (M&E) system is essential to evaluate data effectively, to track progress and to take well informed decisions.
What is a monitoring and evaluation (M&E) system?
The definition of an M&E system varies between different organizations. In most cases an M&E system refers to all the indicators and tools that you will use to measure a program that has been implemented according to a plan. M&E is the process of gathering, systematizing and analyzing information based on objectives and results; it is a management and learning tool that provides crucial information about the process and success of the project.
Benefits of using monitoring and evaluation
Monitoring and evaluation (M&E) helps to:
• Have an effective management and decision-making process in order to improve the execution and results of the project.
• Check the effectiveness and limitations of the methodology applied to execute certain projects.
“Monitoring” refers to a continuous process of analysis according to the project operating plan (POP). It is conceived as an internal activity of the project that is fundamental for good administration and management.
The “Follow-up” refers to the system of observations carried out to check and analyze activities and results of every project that have been executed to check if it has contributed to the achievement of the established objectives. The follow-up consists of a technical analysis, an evaluation of the reports (ITF, semi-annual report, final report), visits, meetings and other activities carried out by the UNIP and / or URIP.
The “Evaluation” refers to a more analytical exercise, carried out to determine the degree of success of the project. In addition to measuring effectiveness, the analysis may include other criteria such as relevance, sustainability, etc.
The M&E system will be carried out at two levels: at the program level and at the project level. M&E elements are basically the same. Since the projects are part of a program, the Planning and M&E systems are closely related:
1. The M&E of the project is directly related to the results, objectives and indicators of the Program. So, program planning influences the strategic project plan.
2. Project-level M&E results directly contribute to progress and achievement M&E to program level.
Project Operational Plan (POP)
The Project Operational Plan (POP) consist on planning the project activities in order to achieve results. Any financial modification between the activities or technical modification of the Plan must be formally requested with the respective support to the IICA national office to your evaluation and approval.
Monitoring and Evaluation exercises and baseline development
The M&E system depends on existing needs, capacities and resources. The most important thing is that the process can provide relevant data to measure changes, achievements and effectiveness.
M&E methods
Some M&E methods can be used to monitor indicators and the environment of the projects. For example, if a project tries to improve economic situation or productivity it may be useful to involve some producers to have a better appreciation of all changes over time rather than just do a survey at the end of the project.
There are diverse types of formats for evaluations. Some of them are:
• Participatory evaluations: It is a method useful for understanding how the target group views the project and to discover differences between certain groups (women-men, youth, minority groups, etc.).
• Systematic monitoring (registration in predesigned formats, workbooks) of the activities and results by field staff who is the team in daily contact with the beneficiaries.
• Specific field studies to obtain specific information.
Baseline
The baseline is a set of selected strategic indicators that serves as a point of comparison for future evaluations in order to determine the achievements that have been made to a project during its execution. A baseline is established before starting the planned actions and the exercise is repeated at least at the end of the project (starting line) to be able to compare the changes in the indicator values. In other words, the baseline shows the situation before starting the project and the starting line shows the situation at the end of the project, using the same methodology and the same indicators.
The baseline should focus on the most relevant aspects related to the purpose of the project. Therefore, the variables should be closely related to the indicators in the project’s logical framework, specifically at the purpose and results level to measure project performance.
Establishing a baseline is best suited for pilot projects that seek to improve the situation of the beneficiaries in terms of sustainable access to energy (AEA) and / or higher profitability
Through an analysis before and after the execution of the project, changes can be measured introduced by the project. In the case of research projects or studies, the establishment of a baseline seems less adequate, but as in the pilot projects the base values of the indicators of purpose and results before the execution of the study or investigation.
How to establish a baseline?
The primary reason to establish a performance baseline is to being able to quantify changes in performance and changes in your load or application. The most difficult part of establishing a baseline is determining which operations are critical to the effective use of the system. You want the list of operations to be complete and concise. Too many items increases the amount of work needed to establish reliable baselines.
The basic rules outlined below allow you to narrow down the number of operations that need baselines to a manageable number.
When generating your list of operations to baseline, include the following:
*Tasks that are done many times throughout the day (such as creating orders, process control, and data entry tasks)
*Tasks that are important to the user (such as Web site queries, customer support screens, and order entry tasks)
Methods for data collection
Sometimes, the information necessary to establish the baseline can be obtained from secondary sources, in most cases a field study is needed. In its realization, quantitative and qualitative methods are used in order to optimize the quality of the findings.
The most used methods are surveys and interviews, as well as focus groups, evaluation participatory rural (ERP) and document review (in the case of secondary sources).
For development projects, field studies are generally based on interviews and focus groups because they allow a broader approach to the subject evaluated and individuals are able to provide both quantitative and qualitative information.
The focus groups allow you to know the opinion and collective values of certain groups of the population. Other qualitative methods relevant to a baseline are ERP methods that include various instruments such as mapping, preference ranking, wealth ranking, periodic and historical diagrams, institutional diagrams, and other methods that facilitate the public participation in the evaluation.
In the case of a field study, particularly when it is based on interviews, it is necessary to get a sample that guarantees the representativeness of diverse segments of population.
How to make a sample?
In order to draw valid conclusions about the baseline and start lining results, it is necessary to establish a representative sample of the target population.
1. Selection of areas and respondents
To be a representative sample of the target population, respondents must be distributed over the entire geographic area of the project and represent all relevant groups. It is advisable to select some representative areas of the entire area and select a certain number of respondents based on a random selection. There are several possibilities and sampling statistics techniques (random, stratified, cluster, systematic, and other types) that we cannot discuss in detail in this guide.
In case you want to understand the differences between certain types of beneficiaries (by gender, age or other criteria) it is important to include enough representatives from each group (stratum) in your sample in each selected area, for example proportionally to the size of each population group. Stratified sampling requires that the size of each stratum need to be large enough to be able to compare the differences. In most projects a stratified sampling will not be necessary or applicable.
In the selection of the areas and respondents, it is necessary to take care of certain prejudicial criteria. For example, logistics, access, distance to the road, preference of certain social groups are factors that can influence the representativeness of the sample.
Determine the sample size
The most important considerations in calculating the sample size are related to the budget available to do the study and statistical requirements. A small sample would not allow statistically valid conclusions. On the other hand, a very large sample can be expensive and go more beyond the project resources. However, it is recommended to calculate the sample size based on statistical criteria and, if necessary, lower the margin of error to achieve an acceptable size.
Reports
The progress of a program largely depends on the execution of the projects. Delays or problems in the execution of the projects automatically affect the degree of progress of the program. In order to measure progress
The Financial Technical Report (ITF), is used to calculate the progress of the project and see the difference between the plan and actual progress.
The final report shows the entire project achievements. It is based on information from all previous semi-annual reports, last semester analysis and the results of the starting line. The final report focuses particularly on the analysis of effectiveness. It also include a sustainability analysis which measures environmental effects, externalities, and economics and financing.
DataScope is a platform which allows various industries to streamline, organize and evaluate the work of their field staffs thanks to online forms which provide real time indicators 100% adaptable to any field.
