A discussion of the reasons behind the choice of experimental design
On 25 August 2016 the Finnish Ministry of Social Affairs and Health sent out for comments a legislative proposal on the basic income experiment. As could be expected, initial reactions to it have been highly critical.
As is the case with most matters of public interest, the more details about them are revealed, the more criticism they tend to attract. The public response may be favourable as long as the discussion remains on a theoretical level, but when the focus turns to more concrete issues, the comments become more critical.
We may perhaps safely assume that everyone is in favour of equality, but opinions diverge when it comes time to find practical ways to promote it. Similarly, there is a strong general support for the idea of basic income but it does not have bearing for the specific basic income models. Also taxes needed to finance the system will diminish the support of the model.
Thus, the political and practical problem may be that in principle people are in affirmative but in practice they are not willing to pay the financial costs.
As expected, this is precisely what happened with the basic income experiment. Some thought that the amount that would be paid out as a basic income was outrageously large, while others found it shamefully meagre. Some considered the target population to be completely unsuitable and argued that it should include students and other young persons, those who are partially fit for work, freelance workers, microentrepreneurs, non-regular workers, and low-income individuals.
Criticism from proponents of a basic income
While some of the criticism was expected, some came from unexpected quarters, namely those who wholeheartedly support the idea of a basic income.
Economists pointed out that the model selected was not cost neutral. Some even denounced it as ”outrageously expensive”. However, there were economists who had good words for the experiment, saying that the experiment design can provide valuable information about incentive effects.
The criticism expressed by Social Democrats was expected given their earlier comments on the basic income. Some of them rejected the experiment out of hand. ”Sample too skewed and small, research pointless. How can anyone who calls himself a scientist go along with this travesty?”
Criticism also came from the ranks of the Green Party and the Democratic Alliance, both parties that support the idea of a basic income. They found fault with the decision to focus on the unemployed, with the fact that the basic income would be exempt from tax, and with the high cost of the system. Some argued that the experiment should have been built around voluntary rather than mandatory participation.
Voices from the Left Alliance took up the same line of criticism, calling for the inclusion of various specific population groups in the experiment.
According to the Christian Democrats, the model is unsustainable and should be replaced by something that mimics the British Universal Credit system.
The Centre Party’s youth organization said that the proposal left young people behind. Some Centre representatives admitted the problems but argued that the experimental design suggested is a good starting point and possibly can be complemented larger by a larger experiment.
There are many reasons for this criticism. Much of it is well founded while some stems from misunderstandings, whether accidental or intentional. Some of it is simply politics: Whatever the government does, the opposition will criticise. After all, the logic of opposition politics seems to be that the government never does anything well or to completion.
Compared to the optimal experiment design outlined in the preliminary report, the current proposal is very limited in scope and for many a terrible disappointment. This, too, is behind some of the criticism being heard.
Finally, some of the comments combine all of these elements.
In the following, I will attempt to explain why the criticised model and experiment design were chosen, and why the model diverges from the designs outlined in the preliminary report. I will strive to provide an answer to the question, ”why is such a foolish model being tested in the first place?”
Research design outlined in the preliminary report
In the preliminary report we sketched out an optimal model based on a partial basic income of €550 after taxes. We argued that it would be advisable to test also other basic income levels (€650 and €750) within the parameters allowed by the constitution. The idea was that the basic income would be deducted from existing contingency-based entitlements. Earnings-related benefits would have been retained to the extent that they exceed the basic income level.
In our plan, basic income would be taxable, but an amount of earnings equal to the basic income would be exempted from tax.
We came up with various tax models, including a flat tax, a combination of a flat tax and the existing progressive tax regime, and examined what it would mean if the existing tax system were to be applied to the basic income. The idea was to test different levels of basic income and to apply to them various tax models and rates of tax.
We found that should the existing tax system be used in the basic income experiment, the effective marginal tax rates and participation tax rates of the basic income models would be significantly lower than those seen under existing social policy arrangements. The outcome of the analysis performed under the above conditions would indicate whether stronger incentives would increase the volume of work in the treatment group vis a vis the control group.
Further, we noted that the model would, however, not be cost neutral in terms of its financing were it to be adopted across the entire Finnish population. The criticism outlined above mainly focuses on this point. We had various models for cost-neutral tax systems, too.
The starting point of the preliminary report was that the experiment would focus on the entire non-retired adult population.
However, due to budgetary constraints, the population had to be narrowed. We cautioned that it would perhaps be necessary to define the treatment group more narrowly in terms of age and income, and to focus on a specific population group among which the incentive effects relating to the decision to accept a job offer would be more marked.
We further suggested that it might be possible to generate separate weighted samples for specific population groups of particular research interest. Once the system required to make the basic income payments was in place, increasing the target population for example to the aimed-for level of 10,000 would not be a problem.
The premise of the preliminary report was that the optimal experimental design would consist of two parts that would provide answers to different types of questions. The first would be a nationwide sample that would ensure that the results could be generalised to the population, while the second would be a regional and more intensive sample at the municipality level, which would make it possible to study externalities.
If only one could be implemented, we prioritised the randomised nationwide experiment.
According to the legislative proposal, the experiment would be conducted among persons receiving unemployment benefits from the Finnish Social Insurance Institution Kela.
A treatment group of 2,000 persons would be selected at random from the target group of about 120,000 people. Participation would be mandatory so as not to produce skewed results.
The proposed level of basic income is €560 per month. Because of the mandatory nature of the experiment, the lowest tested level of basic income would have to be set at a level equal to the labour market subsidy and the basic unemployment allowance.
The basic income would be exempt from tax and would not reduce the participants’ earnings. Participants who find work would get to keep all of their earnings that are taxable under the current tax system. The basic income would be paid out by Kela.
The above means that a basic income would not immediately be paid to the entire Finnish population.
Why were the recommendations of the preliminary report not adopted as such?
While the legislative proposal was drafted and formulated by the Ministry of Social Affairs and Health, perhaps I may be allowed to make a few remarks that reflect my personal experience with the issues involved and the constraints we had to take into account. This much-maligned model was selected under several constraints.
The budget for the experiment was €20 million, which had to cover its administrative cost. This means that an amount less than €20 million would be available to be spent on benefits. As of now at least, the government has not made any promises to increase the budget.
As the Finnish Tax Administration was not involved in the drafting of the law, it was not possible to make any changes in the tax provisions, which means that the model had to be based on the existing tax system.
As said above, the preliminary report of the research team describes a number of cost-neutral tax models, but implementing them might have cost as much several million euros. Also, it would have been impossible, both in terms of the technical implementation and the legislative work required, to produce them by 1 January 2017.
For constitutional reasons, it was decided not to test several different levels of basic income. Under the constitution, all individuals are to be treated equally unless there are justified reasons for not doing so. The prospect of producing valuable information to support decision-making can be considered to be such a justified reason.
A experiment of this kind is without precedent in Finnish history. Therefore, the decision was taken to start off with one level of basic income that would correspond to the current level of basic economic security enjoyed by the target group. Participation was made mandatory in order to prevent selection bias. There are precedents for regional experiments with mandatory participation.
It would certainly be possible to carry out the experiment on the basis of voluntary participation, but such a design includes so many potential sources of error that we decided against it. A voluntary experiment is only the second-best experiment!
Why a treatment group of 2,000 unemployed persons?
The reason for the decision to carry out the experiment among 2,000 unemployed persons is that, as current recipients of welfare benefits from Kela, they offer a cost-effective alternative. The adopted research model is such that it allows us to base the research around unemployment benefits being paid out by Kela.
Also, it is less complicated to draft the law with a specific group of people in mind. With a more heterogeneous treatment group, it would have been more laborious and time-consuming to make sure that the research model would fit a variety of different work, family and social policy situations. Inserting a simple model into a complex institutional environment is a difficult task. In this regard, the time constraints placed on the legislative process limited the choice of the target population and research model.
Groups other than those collecting unemployment benefit from Kela had to be excluded for reasons having to do with costs, taxation and various ”long-tail” considerations of social policy.
Taking into account the administrative realities of unemployment benefit provision, the 2,000 (possibly increased later to 3,000) unemployed individuals are the maximum population size that can be accommodated within the specified timeframe. With a significantly larger sample, it would be necessary to devise a more complex payments system, which would have taken longer.
What I said about the legislative process applies here as well. Introducing a new element into the payments system requires all of the other system elements to be adjusted, which requires a substantial effort in terms of systems development. However, Kela’s systems development resources are already reserved for the pressing task of creating a payments system in time for the handover of social assistance payments from the municipalities to Kela, which will take place on 1 January 2017.
Finally, the choice of unemployment benefit recipients was supported by the possibility to make the sampling speedy and efficient. This is because the up-to-date recipient registries maintained by Kela make it easy to draw a sample of unemployment benefit recipients.
If the target population consisted of low-income individuals, as envisaged in the preliminary report, the sampling would have had to be made on the basis of tax data, which, at worst, can be two years old. Thus, the sample would have been based on data that was no longer current.
A treatment group drawn from some other source than Kela’s beneficiary registry would have been too expensive
As for the notion that the target group would consist of students and other young people, those in non-regular or freelance employment, or individuals who are partially fit for work, we must bear in mind the limits imposed by the experiment budget of €20 million minus administrative costs.
One participant who is a freelancer/non-regular worker/self-employed carries a cost of about €14,000 for the experiment budget. Hence, if the entire budget of €20 million (not taking into account the administrative costs) were spent on the above mentioned groups, the sample would only comprise about 1,400 individuals. Taking into account the administrative costs, we would be left with a sample of not much more than 1,000 persons, which would have to be subdivided into smaller groups of a few hundred individuals.
Any information obtained from such a small sample would not be reliable. A sample population of 1,000 students carries a cost of €10 million. Then, if the sample were divided between several municipalities, the sample would be fragmented even further.
If this does not have any effect, what will?
The model outlined in the proposed law has been criticised for being too generous and not cost neutral.
The model is generous because the basic income has been defined as tax-exempt income. No one is proposing that this ”foolish” tax-exempt model would be adopted as such.
The tax-exempt model was defended with the argument that because it was necessary (as described above) to adopt such a small sample size, the treatment effect would have to be significant for us to determine reliably whether the treatment has any effect. According to power analyses, using the model set out in the proposed law and a sample of 2,000 individuals, we can identify this effect.
The model allows us to determine whether financial incentives affect people’s behaviour. If this generous model has no effect, what will? The model can also be used to study how ”bureaucracy traps” affect behaviour.
My hope is that the experiment scheduled to start on 1 January 2017 will be a pilot study followed later by a larger experiment, which would include some of the other groups mentioned above and which would allow us to test a cost-neutral model, either on the basis of the flat-tax models described in the preliminary report or with modified progressive tax models.
But of course, that would require a larger budget, the participation of the tax administration in drafting revised tax laws, and resources for Kela to construct a payments system suitable for use in a relatively large treatment group.
Professor, Research Director