Content and objectives
The objective of this course is to give insight into the various approaches and methods currently used to evaluate the impact of social policies. The course starts with a general overview in which the advantages and limitations of experiments, the difference-in-difference approach, and the cross-sectional or comparative approach are discussed. Part of the course is devoted to the various types of simulation models (empirical micro-simulations versus standard simulations; dynamic versus static, etc.). The course presents findings from empirical evaluation studies and the extent to which these studies conform to predictions provided by theoretical models and simulations. Special attention will go to the fields of labour market policies and tax benefit policies, from which several examples will be studied

Programme

Title: Evaluating the Impact of Social Policies
Karel Van den Bosch

The day consists of a lecture (AM) and a seminar (PM), in which a text will be discussed.

Lecture outline:
The lecture gives an overview of the various approaches to measure the impact of policies, in particular social policies. Discussed are the advantages and limitations of experiments, the difference-in-difference approach, and the cross-sectional or comparative approach. Examples from the various approaches are presented to substantiate the points made. The lecture serves as an introduction to the other lectures in the same week by Gerlinde Verbist, Bea Cantillon and Ive Marx.

This lecture will be (somewhat loosely) based on (text attached):

Van den Bosch, K and Cantillon, B. “Policy Impact”, forthcoming in The Oxford Handbook of Public Policy (Goodin, Moran, Rein, eds.).

Seminar theme:
"In the afternoon, there will be individual talks, or a seminar, whatever the students prefer. The seminar would be based on the following text:

Ravallion, Martin: "The mystery of the vanishing benefits. Ms. Speedy Analyst's Introduction to Evaluation" World Bank Policy Research Working Paper 2153.

Title: The Evaluation of Tax-Benefit Policies
Gerlinde Verbist

Lecture outline:
Tax-benefit policies can be evaluated in various ways. We focus here on the social outcomes of taxation and social benefits. We first present an overview of indicators that can be used to evaluate these policies. Next, we look at the instruments that are increasingly used for such evaluations, namely simulation models. We distinguish the various types of simulation models (empirical microsimulations versus standard simulations; dynamic versus static, etc.). The scope and limitations of these various methods will be illustrated with examples from the literature.

Reading assignment:
The lecture gives an overview of the evaluation of tax-benefit policies, with an emphasis on the use of simulation models.  The reading assignment focuses on international comparative issues in this context. 

Students are required to read two papers:

H. Immervoll, H Levy, C Lietz, D Mantovani, C O’Donoghue, H

Sutherland & G Verbist (2005), Household incomes and redistribution

in the European Union: quantifying the equalising properties of taxes

and benefit” EUROMOD Working Paper EM9/05, The Microsimulation

Unit, Cambridge University, 25p. (also published in D. Papadimitriou

(2006), The Distributional Effects of Government Spending and

Taxation, Palgrave/Macmillan, pp.135-165;

H. Immervoll, P Marianna and M Mira D'Ercole, 2004, "Benefit        Coverage Rates and Household Typologies: Scope and   Limitations of Tax-Benefit Indicators”. OECD Social, Employment and Migration Working Paper, Organisation for Economic Co-Operation and Development, Paris

Title: The Evaluation of Labour Market Policies
Ive Marx

Lecture outline:
In Europe and elsewhere, governments have implemented employment subsidies and/or cuts in employers’ social security contributions in order to improve the labour market prospects of the unemployed and other vulnerable groups. According to OECD figures, such programmes account for a significant share of active labour market spending in the OECD area, on average 24 per cent. Policy makers often justify this strategy by referring to theoretical analyses and simulations which suggest that such measures have strong positive effects on the employment and mobility chances of beneficiaries. This lecture presents findings from recent empirical evaluation studies which have sought to gauge the actual effectiveness of such measures. The most striking overall finding is that the net employment effects of such measures generally turn out to be substantially lower than what theoretical models and simulations predict, even under relatively conservative assumptions. This finding is particularly striking because of its consistency across studies and evaluation methodologies used.

Reading assignment:
Whereas the lecture deals with demand oriented measures, the reading assignment deals with mostly supply oriented measures. 

Students are required to read a paper by Rebecca Blank: Evaluating Welfare Reform in the United States

This paper reviews the economics literature on welfare reform over the 1990s. A brief summary of the policy changes over this period is followed by a discussion of the methodological techniques utilized to analyze the effects of these changes on outcomes. The paper then critically reviews the econometric and experimental literature on caseload changes, labor force changes, poverty and income changes, and family formation changes. A growing body of evidence suggests that the recent policy changes have influenced economic behavior and well-being in a variety of ways. One particular set of 'new-style' welfare programs seems to show especially promising results, with significantly increased work and earnings and reduced poverty.

Optional reading:
OECD (2003), Making Work Pay, Making Work Possible, OECD Employment Outlook 2003.

Title: Fuzzy set Qualitative Comparative Analysis
Natascha Van Mechelen

Lecture outline:
This lecture discusses a comparative approach to measure the impact of policies, fuzzy set Qualitative Comparative Analysis (fs/QCA). fsQCA is a comparative method based on set theoretic relationships.  It is a tool for systematically comparing configurations of explanatory variables for the presence of absence of a particular outcome. The advantages and limitations of this research technique will be illustrated with empirical literature.

This lecture will be based on:
Charles C. Ragin (2006), Set Relations in Social Research: Evaluating their consistency and coverage. In: Policital analysis, Volume 14, Number 3, pp. 291-320.

Charles C. Ragin (forthcoming), Calibration Versus Measurement. In:

David Collier, Henry Brady, and Janet Box-Steffensmeier (eds.),

Methodology volume of Oxford Handbooks of Political Science.

Natascha Van Mechelen & Veerle Demaesschalck (2006) Devolution as a means to adequate social safety nets? Paper presented at the Fifth Annual ESPAnet Conference, September 20-22, University of Vienna, Austria

Reading assignment for seminar:
Jon Kvist (2006) Diversity, Ideal Types and Fuzzy Sets in Comparative Welfare State Research. In: Benoît Rihoux & Heike Grimm (eds.),  Innovative comparative methods for policy analysis: beyond the quantitative-qualitative divide. Berlin : Springer, pp. 167-184.