Hybrid Means Testing (HMT) Model Development

The task of formulating, implementing, and applying the hybrid means testing model (HMT) that would be used for assessing total family income was undertaken within Pilot #3, “Introducing uniform households means testing criteria for granting all types of social assistance, and improving the techniques of calculating a family’s aggregate income for all types of social assistance (SA). This model is expected to increase the accuracy of income assessments, to improve SA targeting, and/or to raise the efficiency of the social inspectors. The HMT is an income estimation method in which both the official income records (the “easy to verify” incomes) and indirect estimation methods (calculating the “hard to verify incomes”) are used in order to predict total family income.

Project researchers have analyzed international experiences where HMT and similar approaches were used in many countries, including Georgia, the Russian Federation, and Bosnia and Herzegovina. Also, simulations of HMT using the 2008 Ukraine Housing Budget Survey data (HBS) and the 2009-2010 data collected in the five pilot offices have been carried out. In order to select the best model formulation several criteria were used: the theoretical validity, model simplicity, the goodness of fit indicator (the coefficient of determination) and the significance statistics of explanatory variables (p-values). As a result, from a large number of alternative models, four specifications have been chosen:

• Classical linear model (LM1) which does not include declared income among its explanatory variables
• Classical linear model (LM2) which includes declared income among its explanatory variables
• Heckit model (HM1) which does not include declared income among its explanatory variables
• Heckit model (HM2) which includes declared income among its explanatory variables

LM1 generates highly volatile income predictions across the households/families. HM1 produces close to constant additional income independent of the amount of declared income. On the other hand, LM2 and HM2 use all available information and tend to produce more robust results. The predictions generated by LM2 and HM2 are quite similar, but the latter model is more complex in its formulation and implementation. Therefore LM2 is recommended for model applications.

Simulation with the data collected at the pilot offices shows that total predicted income exceeds the declared income by one-half on average. This suggests that social assistance applicants in the pilot offices tend to underreport their total incomes by at least one-third (the income data collected by HBS are also likely to be underreported, though this underreporting is much smaller). According to the results of our simulations, the level of underreporting is different for different groups of families and different categories of SA. For example, the total income predicted for families applying for fuel and housing subsidies is only about 28% larger than their declared income, while in the cases of applicants for low income benefits and for child benefits, the predicted income is two to three times higher than the declared income (in the pilot dataset many of these applicants declared zero incomes). Our simulations have demonstrated that if, hypothetically, the income figures were used for granting the benefits, under current benefit eligibility thresholds, many families would become ineligible for the SA programs.

One possible solution could be to change the eligibility thresholds and/or account for only a fraction of additional predicted income (rather than for the total amount) in estimating family incomes. In the case of a large difference between declared and predicted incomes, social inspectors would be advised to select such a family for home inspection.