MTW Retrospective Evaluation: A Picture of MTW Agencies Housing Assistance, MTW and Cost Effectiveness

HUD’s final 2020 issue of Cityscape: A Journal of Policy Development and Research (Vol. 22, No. 3, 2020) focused on the Moving to Work Retrospective Evaluation. The February 17 edition of the Advocate included a brief overview of the contents of the Retrospective, and an in-depth look at one of the included studies Examining the Effects of the MTW demonstration on Housing Choice and Self-Sufficiency Outcomes.

In this issue, we review two additional studies, “A Picture of Moving to Work Agencies’ Housing Assistance,” and “Fund More, Serve More, Save More: Moving to Work and Cost Effectiveness.” These reports have substantial overlap in both authors and underlying administrative data used for the analysis. Future issues will review reports on MTW Agencies’ use of Funding Flexibility and MTW Agencies’ use of Project Based Vouchers. Two additional articles on Rent Reform will also be covered in a future Advocate. Three international perspectives and other data-focused regular features can also be reviewed at the HUD User, Periodicals, Cityscape website.

Who MTW Agencies Serve Compared to Similar, non-MTW Agencies

In “A Picture of Moving to Work Agencies’ Housing Assistance,” the authors compare households served by MTW agencies with households served by similar non-MTW agencies to identify changes over time in type of assistance, amount of assistance, and characteristics of households served, and to determine if those changes were a result of MTW status.

In this article, the authors describe MTW agencies, the types of housing assistance they provided, and the households the agencies served as of 2016. The study uses HUD administrative data spanning 2008 to 2016 – including Public and Indian Housing (PIH) Information Center (PIC) data, Voucher Management System (VMS) data, U.S. Census Bureau data, and HUD-provided counts of housing assistance unique to MTW agencies – to contrast the housing assistance provided by MTW agencies to that of similar traditional PHAs.

As with all MTW research, the study was complicated by the wide variation in MTW agency programs and practices. The addition of nine new MTW agencies between 2009 and 2014, and the shift of units from the public housing to the HCV program through RAD or other conversion and redevelopment initiatives must also be accounted for in the study design. Researchers attempted to balance for some of these complications by identifying a subset of all public housing agencies that were most like MTW agencies – tending to be larger, located in the principal city of a Metropolitan Statistical Area, operate both the public housing and the housing choice voucher programs, and operate in slightly more expensive markets and more densely populated counties. The 39 MTW agencies and the 779 comparison agencies represent only about 21 percent of all public housing agencies nationally, but account for three-fourths of all PHA assisted households in 2016.

These reports provide support for the perspective that the MTW program allows for local initiative and innovation without increasing federal obligations or expenses. They also counter prior reports suggesting that fewer households are served, at a higher cost per household, as a result of MTW participation.

The authors conclusion is that “when measures of MTW housing assistance are compared with those of a subset of comparably sized traditional PHAs, the MTW agencies seem to provide a similar mix of housing assistance, serve similar populations, and have households in areas with similar levels of poverty…. Some characteristics, such as the share of single-adult households and the share of work-able households, have changed over time for both MTW agencies and traditional PHAs…. MTW agencies provide more project-based HCV assistance compared with traditional PHAs and seem to have added new households to their assistance portfolios between 2008 and 2016, whereas traditional PHAs did not.” Based on the data available, they were not able to draw any conclusions on the effect of MTW status on these outcomes.

Significantly, this paper is based on larger HUD funded research with a full report that is forthcoming, including a publicly available on-line data tool that will provide agency-level information for selected measures of housing assistance for each of the 39 MTW agencies. PHADA will notify members when that report and data becomes available.

Cost Per Household and Cost-Effectiveness of MTW Agencies

This paper, titled “Fund More, Serve More, Save More: Moving to Work and Cost Effectiveness,” is a re-presentation of data and analysis originally presented in a longer, June 2020 report, “The Impact of the Moving to Work Demonstration on the Per Household Costs of Federal Housing Assistance” (Stacy, et al., 2020)(http://bit.ly/3qv4waO). That report includes a more comprehensive discussion of the methodology, comparison group selection, and additional sensitivity analyses included in their methodology. This version presents the data in the context of overall cost-effectiveness. Both presentations highlight the challenges of calculating cost-per-household and cost-effectiveness using strictly quantitative measures derived from HUD and PHA reported financial data.

Looking at historical data covering 2003 to 2017, the report responds to other literature indicating that MTW agencies have a higher cost per household served than traditional agencies. This study looks at broader trends, pre- and post-MTW data, as well as trends at similar, non-MTW agencies to determine if those analysis are correct. The research attempts to isolate the impacts of MTW participation as compared to impacts that are the result of broader trends, local conditions, or other factors unrelated to the funding and regulatory flexibility of MTW.

The study relies primarily on PHA FDS submissions, PIC data, and VMS, but also had to construct some data to fill in gaps in HUD’s records, especially in the early years of the study period. For this part of the analysis the study also excludes 19 MTW PHAs who either entered the program before 2003 and stayed in the program, as well as agencies with fewer than 750 household served. These exclusions were necessary in order to find adequate data and to try to isolate the impacts of MTW participation.

The authors state that “We find that MTW status has no significant impact on cost per assisted household when compared with traditional PHAs of similar size… PHAs do receive significantly more funding after joining the demonstration…, but they use this money to serve significantly more households…. We also find that MTW status is associated with statistically significant increases in the amount of funding PHAs hold in operating reserves…. This finding suggests that MTW agencies are able to serve the same number of households per dollar of HUD funding while also saving money in reserves for future developments and other uses. We find no evidence that MTW agencies maintain their cost-effectiveness by shifting their program mix, reducing housing quality or affordability, or serving different households.”

The study findings include:

• No statistically significant relationship between MTW status and cost per assisted household. Controlling for baseline characteristics, national trends, and exogenous cost drivers, the statistical analysis shows that MTW status is associated only with a small and statistically insignificant increase in cost per assisted household.
• Accounting for changes in (1) program mix, (2) housing quality and affordability, and (3) household characteristics does not alter the primary finding of no significant relationship between MTW status and cost per assisted household.
• “MTW agencies had higher costs, as measured by HUD funding per assisted household, than traditional PHAs before they joined the demonstration. Thus, the higher costs observed at MTW agencies in prior studies are probably driven by differences in the costs unaffected by MTW status, such as labor and housing costs, and not because of the regulatory or financial flexibility offered by the demonstration.”
• MTW status did allow agencies to significantly increase the amount of money held in reserves. Given that agencies increased their reserves while serving roughly the same number of households per dollar, we should infer that they found some cost efficiencies.
• MTW status is associated with both an increase in HUD funding and an increase in assisted households.

Implications for Federal Policy and the MTW Program

These reports provide support for the perspective that the MTW program allows for local initiative and innovation without increasing federal obligations or expenses. They also counter prior reports suggesting that fewer households are served, at a higher cost per household, as a result of MTW participation. In fact, more households as served. This research also counters allegations that MTW agencies are serving households by shifting to lower-cost assistance programs, offering lower quality or less affordable housing assistance, or providing assistance to households with fewer needs.

It is also important to point out that no research to date has suggested any increase in fraud, abuse, or misuse of funds as a result of participating in the MTW program. Combined with the research presented in the MTW Retrospective Evaluation, it is reasonable to conclude that there are no negative impacts of participation in the MTW program by local PHAs. Furthermore, these factors raise questions about the utility of HUD’s public housing and housing choice voucher program regulatory regimes – there are no negative impacts or increased risk to HUD or households in releasing PHAs from most of the existing regulatory requirements through participation in the MTW program. Thus a program to provide broad regulatory relief to more agencies – and including the broader flexibilities of early MTW adopters and not the research focused and limited version offered through the current MTW expansion – would be an effective policy to advance local and responsible decision-making while supporting PHAs and the communities they serve in their ongoing efforts to respond to local needs.