PHADA comments on the Public Housing Assessment System
RE: Docket No. FR 4497-P-01
To whom it may concern:
PHADA, which represents more than half of all housing authorities, is pleased to submit its comments on the above referenced proposed rule.
These comments are based on a careful reading of all printed materials the Department has released, that which is published on HUD's Internet site, and the experience of many of our members with the initial advisory assessment process.
The proposed rule states that its purpose is to enhance trust among PHAs, residents, HUD and the general public "by providing a comprehensive management tool for effectively and fairly measuring the performance of a public housing agency." PHADA strongly supports this goal. However, what is known and unknown about PHAS, the complexity of the PHAS instrument and the experience of many housing authorities appears to indicate that PHAS currently is neither effective nor fair. While the Department is justifiably proud of its automated approach, it needs to pay more attention to the substantive issues surrounding PHAS than the flashy "bells and whistles" of its information system.
PHAS is a young system. As such, it is obviously in need of maturation. The experience of many of our members to date underscores the importance of the maturation process. While HUD and the industry both hope that PHAS will become a truly objective measure of housing authority conditions and performance, it is clear to the association that it is not yet there.
Especially in the case of the physical inspection (as detailed below), the system appears very subjective. This appearance threatens to call into question the credibility of the instrument, and more, of HUD and housing authorities. None of the interested parties can afford a "PHAS credibility crisis" at a time of serious funding shortfalls and when the needs of affordable housing appear to be largely off Congress's radar scope.
We are concerned that the Department does not, at least publicly, appear to recognize that fundamental and serious problems with PHAS exist. HUD cannot fix the problems if it insists on promoting PHAS as a success, even as the Department continues to engage in rulemaking. No one but the Department, it seems, is insisting on a near perfect system from the start; one that is clearly not well tested. This is unnecessary and unfortunate.
More to the point, PHADA, along with its partners, withdrew its lawsuit based on the inadequacy of the initial rulemaking process (among other reasons), partially because it understood the Secretary to indicate that housing authorities would NOT be judged on the basis of the final rule published on September 1, 1998. We believed that no physical inspections would be conducted which had associated consequences until a revised PHAS became effective. However, since the withdrawal of the litigation, the Department has made it abundantly clear that it intends to keep to the initial PHAS schedule and rule, scheduling inspections for HAs even while the rule is still subject to the notice and comment process. In contrast, PHADA firmly believes that no inspections that will be used for official scores should be conducted in the midst of the rulemaking process.
The systemic problems associated with the current version of PHAS and the ongoing rulemaking process is an extremely important issue for PHADA and its members. The assurance from the Secretary the industry believed it had received was apparently either misunderstood or withdrawn. PHADA finds this approach profoundly troubling.
The proposed rule, while further specifying that those which fail only certain parts of PHAS will be designated such (e.g., management troubled), continues to utilize a minimum threshold of 60 percent on three indicators. If an agency falls below that threshold on any of the three indicators, it will be troubled regardless its overall score.
PHADA understands the philosophy behind such an approach. And, indeed, if PHAS had the time to prove itself as a reliable tool, may support it. It makes sense that, if a housing authority has deplorable stock but claims good management, fiscal responsibility, and positive resident relations, it should not be a high performing agency. However, at least two of the indicators, the physical condition and the financial condition (as detailed below), have very serious flaws.
Moreover, the Department should be mindful that it will be very difficult, if not impossible, for many HAs to pass some or all of the PHAS indicators when those HAs are subject to potentially huge cuts in federal assistance. Indeed, HUD's budget proposal for FY 2000 includes a 15 percent cut in capital funding, and the total shortfall in the pending HUD appropriations bill (H.R. 2684) exceeds $1 billion.
In addition, HUD's intent to use the minimum threshold for these two indicators is flawed and premature. Once those flaws are addressed successfully, the Department could again consider applying minimum thresholds to those indicators. In the meantime, for at least the first one or two years, HUD should revise the rule to indicate that minimum thresholds will not be used, i.e., the performance classification would be determined by the overall PHAS score only. This would compensate to a limited degree for the inadequacies in specific indicators.
All "near troubled" and some "standard" performers would be required to submit an improvement plan to HUD. The details of the improvement plan are rather clear. What is absent, however, is anything approaching a method under which the other standard performers required to submit the plan would be selected. Parameters are needed to protect the objectivity of the system. Without such parameters, HUD could require an agency with a score of 89 to submit an improvement plan based on subjective and, perhaps, punitive reasons.
In addition, there should be a system in place for HAs to correct minor deficiencies while the inspector is on site. This would allow both HUD and HAs to avoid potential problems related to the inspection.
Incentives for High Performers
This concept has been verbally embraced by the Department for a number of years. Yet, little real regulatory relief has been forthcoming. HUD responded to suggestions received from housing authorities in response to the initial proposed rule in the preamble of the September 1 final rule. Rather than commenting specifically on the merits or demerits of each suggestion, HUD simply states that it will continue to consider the incentive issue. In addition, the PIH staff drew up a comprehensive list of possible relief in the "decontrol" initiative of several years ago. That list has helpful suggestions. Yet, nothing has apparently come of it. This is not a credible response. PHADA encourages the Department to revisit those suggestions and to at least comment specifically on them.
Appeals of Performance Classification Designation
First, only situations that would "significantly" change the agency's score may be appealed. "Significant" is defined as a modification that would change the performance classification of the agency. Since the burden is entirely on the agency, it must be able to compute the scoring deductions of each deficiency. While the Department has detailed more of the scoring bases for each indicator, it has not disclosed them all.
Some in the Department continue to talk of modifications for the physical condition in addition to the known weights, criticality, severity judgements, and neighborhood characteristics. In addition to "building type," some at REAC continue to speak of other adjustments. It is, thus, impossible for agencies which wish to appeal to know with any certainty how their score would be affected by a successful appeal.
In the same manner, also under the physical condition, the scoring process that is known is so incredibly complex that it, too, makes knowing if a successful appeal would "significantly" change the score very difficult.
Under the management operations indicator, little information is available which would clearly indicate which components receive an A and which would receive another letter grade. Since each of the six letter grades have a corresponding numerical value which is central to the formula, such detail is required before knowing if a successful appeal would change the performance designation.
The Department was steadfast in its refusal to grant anything other than technical appeals to non-troubled agencies during the many months of discussion. Indeed, HUD only expanded the appeals rights after the industry groups sued the Department. In the view of many of our members, HUD has yet to offer a fair and real appeals system. PHADA encourages HUD to revise the system to indicate that an appeal will be considered on the basis of errors in all areas, including the inspector's judgement of the severity of deficiencies, and to permit appeals regardless of any change in the performance designation.
Physical Conditions Indicator
The standards are defined simply as "decent, safe, and sanitary housing in good repair" and areas and items that must be "structurally sound, secure, habitable, operable, and in good repair." These standards are nowhere clearly defined. In addition, they leave too much to the discretion of the inspector. What, exactly, does "in good repair" mean? Who defines "health and safety hazards?" Experience with the advisory inspections shows differences between local health and safety codes and PHAS which are not taken into account. As for buildings being "structurally sound," neither are the inspectors qualified nor the software equipped to deal with any structural problems.
In short, nowhere in the rule are the standards clearly defined. Accordingly, PHADA urges the Department to clearly define the standards under which housing authorities are judged. In addition, PHADA specifically recommends that, in cases of health and safety, local code be sufficient. Housing authorities are required to meet code requirements. They should also be protected from negative consequences if they meet those requirements.
The PHAS inspection is designed to take a "snapshot" of the physical condition at the time of the inspection. The problem with this is that the snapshot may not be a real indicator of either the condition of the property or the competence of the management. When this writer accompanied an inspector, he indicated that even if a tornado had occurred the day before the inspection, the HA would be penalized for the damage because it is a snapshot. This is but one of many examples of a rigid interpretation of undefined standards that lead to a false judgement of the condition of the property.
There are many other examples of rigidity circulating across the country. An inspector cites a building for having box fans in windows (no ac) because it blocks emergency egress (even though they are completely portable). Another agency is told that the leaves have to be removed from the roof or the building will fail, even though it is fall and leaves have a tendency to collect on roofs. Grass growing through a sidewalk expansion joint is rated as a serious tripping hazard. A family has just been evicted and, in anger, seriously damages the unit. Another family decides not to accept the new appliances the HA is installing for one reason or another and chooses to keep dysfunctional equipment. The list is extensive. The point is, in the interest of "objectivity," housing authorities are faced with, at best, a rigidity that holds them accountable for things over which they have no control.
The objectivity of the inspection appears, too often, to break down at the level of the individual inspector. PHADA is confident that the majority of inspectors have the highest ethics and do not attempt to manipulate the system. However, others apparently take advantage of their independence.
The software which was to assure objectivity is manipulated by them by overstating the number and severity of deficiencies. In this way, further computer screens are opened and the inspector is able to get to a level of specificity that betrays the "safeguards" of the software.
Again, this is clearly not the case with all inspectors. However, the number of reports from agencies which have undergone the inspection, verifies that such a practice is happening all too frequently. Housing authorities with excellent stock and/or management (sometimes verified by disinterested parties such as Standard and Poors), suddenly find themselves troubled. In our view, this practice is one of the biggest threats to the credibility of the entire system and the industry.
The scoring system is extremely complex almost to the point of being unintelligible. Each of the over 800 possible deficiencies are assigned a percentage weight, a criticality factor with separate numerical values, and a severity factor, also with separate numerical values. Each area, sub-area, item, and sub-item is calculated separately. Weights are "normalized" for missing items. While the general areas have specific weights, the number of items in an area can be so numerous as to seriously skew the results (example, the "common areas" sub-area is worth only 15 percent of the "building" score, yet has approximately 72 percent of all the possible deficiencies in the entire inspection).
PHADA wonders if it is really necessary or desirable to have both separate weights and criticality scales. Also, is it really worthwhile to assign a criticality number, e.g., a 4 on the scale, a different value for the purposes of computation, in this case a 3.00?
The Department has steadfastly resisted making the specifics of its methodology available to housing authorities for fear that the agencies would "game the system." In doing so, HUD fails to recognize the value of housing authorities managing to a truly objective and fair system much like numerous public and private endeavors do in other systems. Advancing a nearly unintelligible system under a banner of objectivity serves no one well and actually encourages manipulation on both sides. Since all the manipulative tools are in the hands of the Department, HAs and residents are the immediate losers. Ultimately, however, HUD also loses as the fundamental problems associated with the inspection become common knowledge. This is unfortunate and unnecessary.
PHADA urges the Department to revisit the entire scoring issue, eliminate items where possible and warranted, and reissue a description of the process with more clearly articulated standards and intelligible methods.
The Financial Conditions Indicator
First, HUD determines the peer group by agency size only. This is only one significant factor which determines similarities among housing authorities. Not taking other factors into consideration seriously damages the usefulness and credibility of the measurement system. Such additional factors specifically relate to financial measurement and include differences in allowable expenses levels and tenant-paid vs. agency-paid utilities. These two factors influence the score enormously. There are, of course, other factors including age and types of buildings and geographical locations.
Second, HUD's determination of the peer size groupings is inadequate and misleading. There may be little substantive difference between an agency of 48 units (very small) and one of 220 units (low medium). Conversely, there may be huge differences between an agency of 1260 units and one of 4200 units, both classified as large and compared to each other.
PHADA urges HUD to revisit the peer issue to come up with group definitions that more accurately reflect true peer groups.
A downside of comparison with one's peers, however they are defined, is that there is no fixed, predictable standard under which an agency is assessed. Rather, HAs are faced with floating standards and having to get a sense of them only after some years of experience.
HUD addresses this partially by asserting that there are certain "sound business practices" that sweep across industries and HAs of various sizes. It chooses to enforce these practices on the two components with the highest weight, i.e., the "quick ratio" and "months expendable funds," each worth 9 of 30 points. Yet, although the Department states that it had conducted "extensive research" into these principles, it never clearly articulates what the principles are. Nor does it say why it applies these unidentified principles to these two indicators and not the rest.
Perhaps there is a compromise between real peer group comparison and baseline sound business practices. PHADA urges HUD to work towards a system that takes both into better account and to be more public about what the principles are and how it chooses to apply them.
PHADA also recommends that HUD revisit the graphs and tables attached to the amendment proposal. They are largely incomprehensible to those who are not trained in statistics. An important principle of rulemaking, which the Department has embraced, is that of using plain language. This description falls far short of anything close to plain language. Specifically at least, the Department should define the many technical terms it uses for the majority of those reviewing the rule who are not accountants or statisticians.
In principle, PHADA supports the notion of point deductions for serious audit findings. However, it is concerned that deductions for differences between unaudited and audited reports will hit small agencies disproportionately. Many small agencies do not have either accountants on staff or retainer or staff trained professionally in accounting. Point deductions in these cases unfairly penalize those who have made their best good faith efforts to submit an accurate unaudited report. PHADA recommends that HUD hold agencies with 250 or fewer units harmless from this specific deduction.
PHADA is supportive of deductions for serious audit flags and findings. However, here again the Department has seemed to go out of its way to make the method difficult to understand. Is it really necessary to identify three separate tiers? Is it not possible to more simply define what the deductions are for a list of flags and findings?
The Management Operations Indicator
Thus, these components are largely redundant and yield a skewed and inaccurate rating of performance. The agencies which perform at a high level will actually be unduly rewarded while those which perform poorly will be penalized twice.
PHADA recommends that, because of this redundancy, HUD eliminate these components from the management indicator.
PHADA also recommends that two elements of the self-sufficiency component be eliminated: security and grant program goals. Resident perceptions of both of these are already taken into account in the last indicator, resident services and satisfaction.
In place of these deletions, PHADA recommends giving significantly more weigh to two very important management items, also listed under self-sufficiency: applicant screening and lease enforcement. Each of these is currently given only one point of weight out of thirty. This is simply not reflective of their importance. Indeed, it is not reflective of the priority the administration has placed on these activities in its "one strike and you're out" initiative.
PHADA is also very concerned that the method of assigning letter grades, with their associated numerical formula value, is not clearly defined in the amendments. This is critical and substantive information that belongs in the rule. While we are grateful that the Department has put language in the rule to indicate that the components will be treated the same as they have been treated in PHMAP, it appears odd to us to refer the public to an expired rule for an explanation of a current one. In order to maintain an image of fairness, HUD should provide this information.
The Resident Service and Satisfaction Indicator
The association is also pleased with competent third-party administration. We are concerned, however, with several issues.
Nowhere is allowance made for the established fact that dissatisfied customers are more likely to respond to a survey than satisfied customers. Housing authorities which have appropriately taken a proactive stance in lease enforcement are likely to be penalized for good management practices.
We are also concerned that the survey is available only in English and Spanish. There are many public housing residents from other cultures which will be excluded from participation in the survey. HUD should also make the survey available in Farsi and several Asian languages.
HUD has said that not all of the 20 questions would be scored but has not said which these would be and what the specific questions are worth. HUD should publish the survey itself as an appendix to the rule and specifically indicate the scoring weights of individual questions.
PHADA is supportive of HUD supplying promotional materials to housing authorities. We also support exempting those HAs which score 90 percent in the survey itself from submitting a follow-up plan.
PHADA looks forward to and expects continued dialogue on this more important matter.
Return to PHADA Front