Evaluation utilizes many of the same methodologies used in traditional social research, but because evaluation takes place within a political and organizational context, it requires group skills, management ability, political dexterity, sensitivity to multiple stakeholders and other skills that social research in general does not rely on as much. Here we introduce the idea of evaluation and some of the major terms and issues in the field.
The 'counterfactual' measures what would have happened to beneficiaries in the absence of the intervention, and impact is estimated by comparing counterfactual outcomes to those observed under the intervention.
The key challenge in impact evaluation is that the counterfactual cannot be directly observed and must be approximated with reference to a comparison group.
There are a range of accepted approaches to determining an appropriate comparison group for counterfactual analysis, using either prospective ex ante or retrospective ex post evaluation design.
Prospective evaluations begin during the design phase of the intervention, involving collection of baseline and end-line data from intervention beneficiaries the 'treatment group' and non-beneficiaries the 'comparison group' ; they may involve selection of individuals or communities into treatment and comparison groups.
Retrospective evaluations are usually conducted after the implementation phase and may exploit existing survey data, although the best evaluations will collect data as close to baseline as possible, to ensure comparability of intervention and comparison groups. There are five key principles relating to internal validity study design and external validity generalizability which rigorous impact evaluations should address: Confounding factors are therefore alternate explanations for an observed possibly spurious relationship between intervention and outcome.
Selection bias, a special case of confounding, occurs where intervention participants are non-randomly drawn from the beneficiary population, and the criteria determining selection are correlated with outcomes. Unobserved factorswhich are associated with access to or participation in the intervention, and are causally related to the outcome of interest, may lead to a spurious relationship between intervention and outcome if unaccounted for.
Self-selection occurs where, for example, more able or organized individuals or communities, who are more likely to have better outcomes of interest, are also more likely to participate in the intervention.
Endogenous program selection occurs where individuals or communities are chosen to participate because they are seen to be more likely to benefit from the intervention. Ignoring confounding factors can lead to a problem of omitted variable bias. In the special case of selection bias, the endogeneity of the selection variables can cause simultaneity bias.
Spillover referred to as contagion in the case of experimental evaluations occurs when members of the comparison control group are affected by the intervention. Impact heterogeneity refers to differences in impact due by beneficiary type and context.
High quality impact evaluations will assess the extent to which different groups e. The degree that results are generalizable will determine the applicability of lessons learned for interventions in other contexts.
Impact evaluation designs are identified by the type of methods used to generate the counterfactual and can be broadly classified into three categories — experimental, quasi-experimental and non-experimental designs — that vary in feasibility, cost, involvement during design or after implementation phase of the intervention, and degree of selection bias.
White  and Ravallion  discuss alternate Impact Evaluation approaches. Experimental design Under experimental evaluations the treatment and comparison groups are selected randomly and isolated both from the intervention, as well as any interventions which may affect the outcome of interest.
These evaluation designs are referred to as randomized control trials RCTs. In experimental evaluations the comparison group is called a control group.
When randomization is implemented over a sufficiently large sample with no contagion by the intervention, the only difference between treatment and control groups on average is that the latter does not receive the intervention. Random sample surveys, in which the sample for the evaluation is chosen randomly, should not be confused with experimental evaluation designs, which require the random assignment of the treatment.
The experimental approach is often held up as the 'gold standard' of evaluation. It is the only evaluation design which can conclusively account for selection bias in demonstrating a causal relationship between intervention and outcomes.
Randomization and isolation from interventions might not be practicable in the realm of social policy and may be ethically difficult to defend,  although there may be opportunities to use natural experiments. Bamberger and White  highlight some of the limitations to applying RCTs to development interventions.
Methodological critiques have been made by Scriven  on account of the biases introduced since social interventions cannot be triple blinded, and Deaton  has pointed out that in practice analysis of RCTs falls back on the regression-based approaches they seek to avoid and so are subject to the same potential biases.
Thus, it is estimated that RCTs are only applicable to 5 percent of development finance.
Quasi-experimental methods include matching, differencing, instrumental variables and the pipeline approach; they are usually carried out by multivariate regression analysis. If selection characteristics are known and observed, they can be controlled for to remove the bias.
Matching involves comparing program participants with non-participants based on observed selection characteristics.Evaluation of the Effects and Consequences of Major Accidents in Industrial Plants - Kindle edition by Joaquim Casal. Download it once and read it on your Kindle device, PC, phones or tablets.
Use features like bookmarks, note taking and highlighting while reading Evaluation of the Effects and Consequences of Major Accidents in Industrial Plants. Evaluating the Effects of Medication One of the best ways to evaluate a medication is a blind evaluation.
A simple way to do this is to start the medication and do NOT tell the teacher at school. If the teacher says "WOW, your son's behavior has improved remarkably," then you know that the medication works. To evaluate a medication, it is. Evaluation is a systematic determination of a subject's merit, worth and significance, using criteria governed by a set of standards.
It can assist an organization, program, Generalized needs & values, effects Judge the relative merits of alternative goods & services. According to the World Bank's Independent Evaluation Group (IEG), impact evaluation is the systematic identification of the effects positive or negative, intended or not on individual households, institutions, and the environment caused by a given development activity such as a program or project.
Outcome Evaluation measures program effects in the target population by assessing the progress in the outcomes that the program is to address. To design an outcome evaluation, begin with a review of the outcome components of your logic.
Overview Ripple Effects Mapping (REM) is a versatile participatory evaluation tool. The intent of REM is to collect the untold stories and behind-the-scene activities that can ripple out from a specific program or activity.