Evaluation Process Models

NOTE: This toolkit was published by Building Changes in 2013 to help counties meet a 2014 state mandate that all counties have a coordinated entry system for clients entering the homeless system. It has not been updated since then and does not necessarily reflect current or best practice.

In Washington state, coordinated entry has been in operation long enough that an evaluation of the process and review of outcomes is possible. County governments are undertaking this process, and for the tri-county area of King, Pierce, and Snohomish counties, an outside evaluator had been contracted using private foundation funds to review the systems in these counties.

The benefit of an outside contractor is that an evaluation of all levels of the system (lead fiscal agency, lead implementing agency, partner agencies, and other associated collaborators) can be performed.

If the lead fiscal agency is the only evaluator, for example, it may not adequately review itself as critically as needed to increase the system’s performance. It is recommended that an outside evaluator be used if funding can be secured. Contracted evaluation services are helpful for an overall review of the system after a milestone has been met but should not supplant regularly scheduled evaluations of the system and services.

A set evaluation process is necessary to:

Reassert the purpose of the coordinated entry process: clarity on the purpose of the system can illuminate efforts for further systems change in prevention, rapid re-housing, tailored programs and services, and linkages to economic opportunities
Capitalize on opportunities to improve the coordinated entry system by using evaluation results to drive decision making
Celebrate successes with partners—funders and grantees want to hear about how coordinated entry is improving access, assessment, and referral processes for housing and services

The following examples cover evaluation processes that are just coming online to those that have been well established. The implementation of coordinated entry and other system changes may affect how deeply evaluation results can be correlated to meet HUD/HEARTH outcomes and a region’s long-term goal of reducing and ending homelessness. But as the following examples illustrate, achievement of coordinated entry goals can lead to logical conclusions that federal outcomes and regional goals will be met.

King County, Washington

King County rolled out coordinated entry in April 2012. The county has been using feedback from the lead agency and providers to make adjustments as needed. The lead agency also used their initial experience with system demand to restructure staffing. A consistent meeting schedule maintains good communication between the lead agency and the provider network. Partners are engaged in the ongoing evaluation process and participate in making improvements.

King County, along with two other Washington state counties, is part of an ongoing evaluation process performed by an outside contractor. Besides taking part in the outside evaluation, King County has identified a three-tiered data approach to its evaluation plan:

Systems-level data
Operations-level data
Client-outcome data

The lead agency for King County Family Housing Connection (a program of Catholic Community Services) is providing data for the coordinated system evaluation. This is the primary question King County seeks to measure: Is there the right mix of housing interventions available based on the families requesting services, housing inventory, and the rate of availability?

Systems-level data

Inventory of system resources for families with children: types (emergency shelter, transitional housing, rental assistance, supportive housing, etc.), size, and number of units
Rate of throughput: how often shelter and transitional units come open over time
Goal is to evaluate the system resources and to understand where to adjust to better meet needs of families seeking assistance

Operations-level data

Evaluate: time from call to housing referral, inappropriate matches, client placement refusals
Goal is to evaluate the speed and efficiency of process

Client-outcome data

Data from client surveys (performed by partner agencies and the lead implementing agency) evaluate housing stability, client services, and the client’s experience
with the process; targeting of high- medium- and low-needs clients to the right resource
Goal is to evaluate the appropriateness of the services matched to the client; providers will measure housing outcomes and lead agency will measure client satisfaction with services and provider

The evaluation data gathered will remain internal to the county, but results of the evaluation and its implications will be available for review through a dashboard that providers and the greater community can access through its website. King County has an evaluation plan in line with their initial goal and will collect information that meets HUD/HEARTH outcome requirements.

Pierce County, Washington

Pierce County has had coordinated entry in place since January 2011. The county used its early experience as a method for evaluating the intake process, and data confirmed that this experience was, in fact, an accurate assessment of the initial system design.

Using that data, the county, along with its lead agency, Associated Ministries, redesigned the model to meet its goals as such:

Goal	Result
Reduce the number of calls for assistance that a household has to make. Result—access streamlined by creating a centralized intake point with the lead agency, Associated Ministries; more households being served than before	Access streamlined by creating a centralized intake point with the lead agency, Associated Ministries; more households being served than before
Decrease in the vacancy rate of beds and units at provider agencies	Vacancy rates are now close to zero, down from 14 percent two years ago
Reduce the number of households an agency would interview to fit into services	Agencies no longer interview three to five households since households are not selected based upon goodness of fit by the agency, but rather on program criteria with each agency tailoring their services to meet the needs of the family
Collect better data to improve understandings of those being served.	More is now known about households requesting housing that informs programming and systems changes going forward
Divert families from entering the shelter system when possible	A small investment produced timely diversion to prevention services for a small number of families, which reduced shelter use on a small scale, and implementation of rapid re-housing is on track and expected to have a significant impact in shortening shelter stays

From this initial evaluation, Pierce County is beginning to realize the system improvements from coordinated entry. The county reports it can make more informed decisions about funding the right programs to address homelessness for the best return on their investment. Their evaluation results show they are meeting their coordinated entry goals and that a system-wide investment will align them to better meet HUD/HEARTH outcomes. Pierce County’s next step of implementing rapid re-housing is moving them toward the expected long-term goal of reducing and ending homelessness.

Clallam County, Washington

Clallam County has actively been implementing various systems changes, including prevention, coordinated entry, rapid re-housing, and discharge planning, since 2005. Mostly rural, the county developed a coordinated system that meets its unique needs while holding onto best-practice principles. The county has a nationally recognized coordinated system that performs very well, which is supported by evaluation data.

Clallam County uses the statewide HMIS reports to evaluate its coordinated entry system and associated services. 2011 data revealed:

Number of households served remain somewhat constant
Term of homelessness dropped dramatically through the year
Steady exits to permanent housing
Steady recidivism rates (about 2/3 lower for prevention and rapid re-housing compared to transitional)
Earned income remained steady
Data-entry completeness was very high (most client records had complete information)

From this information, the county can glean where resources are needed, for housing, employment, and support services especially, for those in transitional programs. It can evaluate the various systems—prevention, rapid re-housing, and coordinated entry—and identify where efforts should be focused to reduce homelessness. High completion rates for entered data indicate that the partner agency network is complying with the system’s policies and procedures. Their evaluation process highlights outcomes for established goals and meets outcome expectations for HUD/HEARTH.

Alameda County, California

Alameda County began assessing the needs of the homeless community starting in 2003. In 2004, the county, Behavioral Health Care Services, the cities of Oakland and Berkeley, and nine other sponsoring agencies initiated the EveryOne Home Plan, which included these key initiatives:

Prevention
Coordinated entry
Rapid re-housing that was anchored by the conversion of transitional housing units to permanent housing units
Tailored services through a program called HOST
Strategic working committees that focused on increasing housing, services, and meeting HUD/HEARTH requirements

Alameda County’s strong planning and implementation processes gave way to an equally strong evaluation process that results in a detailed community report distributed yearly. The county holds a stakeholder community meeting to gather feedback on the report before releasing it as final. Sharing this report before official release with stakeholders gives them an opportunity to celebrate achievements, acknowledge ongoing improvements, and set goals for the next year—not just for the system, but even for the report itself (e.g., evaluation data in numbers as well as percentages). The stakeholder group is able to see how their work intersects and motivates the network toward developing stronger collaborations. The result is an engaged network of providers and funders who work together to continually improve the system—including its evaluation process.

As of 2011, samples of their evaluation findings include:

Increase in exits with permanent housing
Increase in rates of people who entered the system with no income exiting with some
Improved retention rates in permanent supportive housing
Reduced lengths of stay in shelters and transitional housing
Return-to-homelessness rate lower than the national average

For a deeper look, here is the most recent report, Measuring Success: Progress Report on Ending Homelessness in Alameda County.

Next: Evaluation Challenges and Tips