August 2, 2018

A Deeper Look at the Justice Bus

by Jess Temple, Staff Attorney for Northern California Justice Bus, and Pete James, Senior Manager of Research & Evaluation

July 22, 2018

Like many nonprofits and legal aid organizations, OneJustice is exploring how we can use data to improve our programs and services. That sure sounds great – but what does this actually look like in practice? Can data really provide meaningful insight to the complex, evolving and collaborative process of program delivery? And most importantly, can these insights actually lead to changes in how programs work on a day-to-day level?

To shed some light on this process, we’re going to describe an ongoing project to evaluate the Northern California Justice Bus project. Launched over a decade ago in 2007, the Justice Bus project takes teams of attorney and law student volunteers from urban areas to set up free legal clinics for low-income Californians living in rural and geographically isolated communities. The theory behind this project is that transferring resources in this way can help to reduce the “justice gap” that has been documented in rural areas of our state.

To begin evaluating Justice Bus, we started with a basic but important question: what types of communities does the Justice Bus actually serve, and to what extent is this consistent with the goals of the program? As is often the case in program evaluation, we had to find precise measures for concepts that we often use in more intuitive ways on a day-to-day basis. For example, do we define the community served by the physical location of the clinic or the surrounding populations from which the clients were drawn? Should we use a binary distinction between urban-rural places, consistent with the Census, or a more continuous measure that distinguishes large metropolitan cores from suburbs and small towns? As you can imagine, a host of thorny problems quickly emerged and we will be employing a diverse range of methods to understand the full picture.

For the purpose of this post, we’ll just share some analysis that provides one perspective on the geographic reach of the Justice Bus in Northern California over the last year. In this analysis, we summarize clinic locations by county and categorize counties using the CDC’s urban-rural classification scheme. In the map below, counties with a darker shade of green are progressively less urbanized, and the number of Justice Bus clinics in each county is superimposed in white. Hover over the each county to see more details.

Using this kind of map enables a form of comparison that is central to evaluative thinking. Rather than just telling the story of the counties that we did serve, it prompts us to consider why we served some counties and not others. Looking at the map in this way is revealing. The Justice Bus most frequently operates in “medium metro” counties, typified by San Joaquin County. While these counties have large urban areas, such as Stockton, they are distinct from the large metropolitan core of the Bay Area. The Justice Bus also reaches a range of counties with populations clustered in small towns, such as Humboldt and Mendocino, but Glenn is the only county served that falls into the most rural category.

So what does this data mean, in light of Justice Bus’ aim to serve “rural and isolated” communities? This was the question that Pete, as internal evaluator, posed to Jess and her team who run the Northern California Justice Bus. This led to a useful discussion about the logic and logistics that go into planning the Justice Bus. Jess shared that she sees the primary role of Justice Bus as amplifying the work of organizations that already provide services to low-income Californians. Because we partner with local community and legal services organizations to deliver legal clinics, this means that the program is often most successful in areas that already have some of these organizations that can identify local needs that may be well-served through a clinical model. It is therefore challenging to put together a successful mobile legal clinic in some of the most rural regions where there are fewer organizations to act as partners. In practice, it is also difficult to recruit pro bono volunteers to travel to regions that are located beyond a two-hour drive from the Bay Area; commitments at work and at home mean that there is only so far that volunteers can go on a given day. Putting these two factors together helps explain some of the patterns displayed in the map of clinic locations above.

This process illustrates important points about the relationship between internal evaluation and program planning. Using data to reflect on a program can help surface tensions between theory and practice that might otherwise be downplayed as simply the day-to-day challenges of running a program. As we have seen, Justice Bus has focused its efforts on the most achievable objectives; this is pragmatic, but it also means that its geographic reach is more limited than we might hope. A recognition of these limitations is prompting us to focus planning activity within acknowledged constraints and to more directly recognize the importance of non-geographic factors, such as cultivating strong relationships with community partners. It also helps identify opportunities for complementary solutions, for example supporting more regional volunteering networks rather than attempting to provide pro bono services from a distance.

We believe that shifting strategies surrounding our mobile legal service delivery from the intuitive to the data-driven, and uncovering the ways in which the impacts of our programs diverge from our aspirations, are essential to iterating a successful pro bono program at OneJustice. In doing so, we feel it’s important to share our learning early and often: improving access to justice in our state is a collective enterprise, and to make progress we need to communicate not only about our successes but also our challenges.

To subscribe to OneJustice’s quarterly research newsletter, please email research@one-justice.org