NDIS Review Conversation Series: Paper No. 15

Measuring scheme impact

Key points

The NDIS is anchored on the goal of social and economic participation, which can be taken to mean participants are taking up valued roles in mainstream community life, and not just being present in community
Purple Orange refers to these roles collectively as Citizenhood and these can be measured
A framework called the Four Capitals can give the NDIA a mechanism for quantifying the extent NDIS individual budgets are building participant life chances, key to roles of Citizenhood

The NDIS is there for a reason; to provide Scheme participants with an individual budget that is reasonable and necessary to lift and sustain each person into social and economic participation. It is often described as being founded on principles of social insurance, but the word insurance can be confusing for some people, because the general idea of insurance is that it is an arrangement to offer some protection in the event of an unexpected loss or change. That isn’t the case for the NDIS. It is more a scheme of assurance, where eligible persons can be assured that there are resources available to address the consequences of the person’s disability.

As such, the NDIS individual budget represents an investment in the person by the Australian governments, intended to build the person’s social and economic participation, and through mechanisms that give the person authentic choice and control. Measuring outcomes is the way the Australian governments, and importantly the Scheme participants themselves, can assess the return on that investment.

So, measuring outcomes is important.

Currently, and despite the NDIA’s sincere commitment to trying to understand the Scheme’s impact, the outcome measures currently used are patchy and may not be providing the level of insight needed. For example, one measure of social participation is the amount of time the NDIS participant spends in community. However, this measure does not give any detail on how that time is being spent. If it includes travel in an accessible taxi or a support worker’s vehicle, and time spent walking through a park, or sitting in a café, without meaningful contact with others, then the measure is too blunt, counting presence in community rather than participation. The two are very different, and community presence does not necessarily mean the person is meaningfully participating in ways that are typical for most non-disabled people.

Measuring the right thing poorly is better than measuring the wrong thing really well, and in this case, measuring time spent in the community is measuring the wrong thing. This is because community presence by itself does not resolve the barriers that result in exclusion. It is entirely possible for a person to be present in community in ways that reinforce separateness, otherness, exclusion. In an inclusion context, the phrase ‘community tourism’ which sounds entirely wholesome, is anything but.

So if measuring community presence is a red herring, the question moves to one of how we might best understand if a person is truly participating. A useful distinction between community presence and authentic social participation is to consider the extent to which the person is engaging with other people in community life, and in ordinary valued ways.

In a recent visit to a beautiful wetland in SA, we saw a man living with disability walking. The wetlands have a loop path and we passed this man several times. Each time was the same. He walked several paces behind his support worker, who was scrolling on their mobile phone. Currently, in the way the NDIS outcomes are being measured, this counts as a success, because it is time spent in the community. But it is no success; the man was entirely alone.

We have seen similar instances of people being taken to cafes, to shopping malls, and the like, with the same result. When this is counted as a Scheme success, it is because there is a perceived transactional benefit. A worker showed up at the person’s house and took them out. They were with them, and then they brought them back.

That is not the Scheme outcome people fought for. People fought for a Scheme that not only would mark an end to the rationing, to the waitlists for help, but also for a Scheme that would achieve the opposite of Shut Out; a Scheme that would help ensure each person living with disability is a valued member of mainstream community life.

Moving forward, we have to find better ways to measure authentic outcomes, and we must stand alongside the NDIA in this quest, because measuring authentic outcomes is hard. And as we’ve said above, it is better to measure the right thing poorly than the wrong thing really well.

When we published the Model of Citizenhood Support, it contemplated what it meant to be truly helpful in people’s lives. First, we described something called Citizenhood, which we assert is what lies within the Scheme’s goal of social and economic participation. Citizenhood is defined as where:

“…a person is actively involved as a valued member of their local community, contributing to community life.”

This is what it means to not be shut out. It is what it means to belong.

As such the entire NDIS market – the disability support providers, the early childhood professionals, the employment service providers, the SDA providers, the support coordinators, the LACS, and everyone else providing NDIS-funded services – are in the business of assisting people into these valued roles in mainstream community life; into Citizenhood.

Therefore, a key measure of the Scheme’s success is the extent to which it is assisting people into valued roles, and not the extent to which a person is driven to a wetland to walk alone.

This can be measured tangibly. Paid mainstream employment is a role of Citizenhood and therefore a key Scheme outcome. It is relatively easy to count, and Australia is used to counting employment statistics, so long as this outcome is not confused with employment mechanisms like Australian Disability Enterprises, where the work involves congregation of people living with disability for pocket money wages.

Other roles that can be tangibly measured include mainstream volunteering roles, mainstream community club memberships, diverse friendship networks, neighbour connections, and the like. What qualifies them all is that they are ordinary; that they bear all the hallmarks of the valued roles that non-disabled people have in their lives.

Therefore, we assert the NDIA can explore outcome measures that help reveal the extent to which a Scheme participant is taking up, and holding, valued roles in community life.

This can be taken further. We might argue that it is not the job of disability service providers to deliver people into valued roles. There are two reasons for this. First, each of us builds our life journey based on discovering the things we care about, the choices we make, and the opportunities we take. In other words, it is not a disability support provider’s job to deliver a person into roles of Citizenhood. That is for the person themselves to craft, based on their choices about what’s important to them. The NDIA can measure the extent this is happening for people as a result of their individual budget.

The second reason is that when disability support providers seek to deliver valued roles directly to the person, albeit with hopefully the best of intentions, the choice-making about those roles tends to shift to the service provider, on the basis of what they think they can offer. Also, the world of roles a disability service provider creates can become a facsimile world, different to the real world; housing that is different to what most people have, employment this is different to what most people have, social lives that are different to what most people have, and decision-making that is different to what most people have.

So what then is the role of the disability support provider? How is that agency to be truly helpful? Our Citizenhood model argues the disability agent/agency’s job is to build the person’s life chances, so the person is then able to take up, or remain in, valued roles that are meaningful and fulfilling. Citizenhood sets this out in four main ways, called the Four Capitals, and we argue these are an example of a framework the NDIA can use to build more meaningful outcome measures, that hold NDIS suppliers more accountable for the impact of their work, and which give a more authentic way for the NDIA to assess return on investment.

Below, we give a brief summary of each of the Four Capitals

Personal capital

The first of the Four Capitals refers to the person’s belief in their own value, their gifts, their capacity to grow, to take up valued roles, to see hope in their future, to have jurisdiction over their own decisions, and take purposeful actions. It is Personal Capital that gives you the belief to apply for a job, to ask someone out on a date, to create a sense of home, to take care of your health, and to take a chance on the things that are important to you.

We argue this is a central outcome for the NDIS. Given the tyranny of low expectations that have dogged the disability community for generations, it is surely meaningful if the Scheme supports a participant to reclaim their right to a fair go at what life has to offer, to imagine their valued place in mainstream community life, to see themself for their strengths and gifts and not for their deficits.

This can be measured. Easily.

Knowledge capital

The second of the Four Capitals refers to the person’s knowledge and skills. It contemplates how the person is supported to make the best use of the skills and knowledge they have, and how they are supported to grow new skills and knowledge.

The NDIS currently includes a ‘capacity-building’ element in the individual budgets of many participants living with disability. Often this capacity-building has a therapeutic character, be it speech pathology, occupational therapy, and the like. These can be very important investments, but only if we contemplate how the benefit might be understood. Therefore, we argue that investment in such endeavours can best be measured by the extent to which it grows authentic Knowledge Capital that moves people closer to the take-up of valued roles. Otherwise, the therapeutic pathway is at risk of becoming engrossed in trying to fix the person’s disability instead of trying to fix the consequences of disability. An outcome measure based on ideas around Knowledge Capital in support of Citizenhood, can provide clarity on this.

Meanwhile, it is not unusual for people living with disability in service provision to lose skill and knowledge. For all of us, the retention of our skill and knowledge is supported by us using our skills and knowledge. Unfortunately, it is not unusual in disability services for the service staff to do things for the person rather than with the person. This is often because it is quicker and more convenient. So, again hopefully with the best of intentions, the service provider inadvertently erodes what the person knows and can do by not giving the time and attention to supporting the person to be centrally involved in those things as part of the routine of daily supports.

Translating this into outcome measures, the NDIS might contemplate how to measure the extent individual budgets are being converted into the growth in, and defence of, participant knowledge and skills that can take them into valued roles.

Material capital

The third of the Four Capitals refers to the tangible things in a person’s life. It includes the things the person owns or has control of, and also the public things the person can access, like buses, the shopping mall, the beach, community clubs, employment spaces, education spaces, and so on.

There are two important things the NDIA might measure to assess success. The first is the extent to which the participant’s individual budget is used in a way that defends and advances their personal Material Capital. For example, does a support provider take good care of the person’s stuff? Does a support provider assist the person to move away from poverty (the relative absence of personal Material Capital) into waged employment where the person has disposable income on the same basis as most non-disabled Australians?

The second is the extent to which the participant’s individual budget is used to assist the person use mainstream community resources – public Material Capital –on the same basis as most non-disabled Australians. This includes things like buses, libraries and the like, and not as a ‘tourist group’, and not in aloneness.

These are all elements that can be measured as Scheme outcomes, in support of social and economic participation.

Social capital

The fourth of the Four Capitals refers to the people in our lives. As humans we are interdependent, we give and we take, we live in community where we take up roles that bring value to others, and in turn we gain value from the roles others take up.

But Social Capital isn’t just a marketplace of mutual utility. Social Capital is about the relationships that have importance in our lives. In the many workshops we have run over the years exploring the nature of a good life, themes like family and friends always feature prominently. This taps the importance of what it means to belong, and this sense of belonging is at the heart of the Scheme’s goal of social participation.

The NDIA could contemplate setting outcome measures that explore the extent an individual budget assists a participant to retain connection with the people in their life important to them, and at the same time the extent the participant is assisted to enter new social connections, particularly if the participant has low Social Capital. This is important because many Scheme participants will likely have levels of Social Capital where the only people in their lives, other than core family, might be other people living with disability and people paid to be there. It is a cliché of otherness to assume that the only friends a disabled person can have are other disabled people. And it is a fallacy to say a disability support worker is a person’s friend, because they are not. Friends are not paid to be there.

Outside of the family each of us was born into or raised in, the most meaningful relationships in our lives – partners, best friends, close friends, sincere acquaintanceships – begin with meeting each of these people for the first time. If that first encounter does not happen, nothing else can follow.

This can be a key outcome measure for the Scheme; the extent to which individual budgets are being used in ways that assist participants into new connections and thereby growth in their Social Capital.

Conclusion

Based on the above, we argue the NDIA can build a fresh approach to measuring Scheme outcomes, based on quantifying the extent to which an individual budget is lifting the participant into valued roles in community and economy by advancing the participant’s Four Capitals.

The same can apply to measuring the impact of Information, Linkage and Capacity-Building (ILC) program funds, the Specialist Disability Accommodation (SDA) framework, the use of assistive technology, and the impact of the roles of Local Area Coordinators and other intermediaries.

A set of example measures are set out in the 2013 edition of the Model of Citizenhood Support.

As with the NDIA’s previous selection of standardised assessment tools, there are standardised outcome measure tools that the NDIA might consider, but the problem with these is they weren’t designed with this Scheme in mind. Also, a number of these tools still include considerations of impairment.

As per our argument in an earlier paper about assessment tools, we assert the Scheme, with over 600,000 participants, can grow its own valid and reliable outcome measures, and that such measures should be anchored on the take up, and defence, of valued roles in mainstream community life.

► Join the conversation at our To The POint webinar

Tell us what you think about measuring scheme impact, share your ideas for reform, and help raise expectations about what the NDIS Review can deliver at our final To The POint webinar on Wednesday, 11 October at 1pm ACDT (that is, 1:30pm AEST and 11:30am in the West). The webinar will run for 1 hour and feature Robbi Williams, CEO of JFA Purple Orange, discussing this Paper. Attendees will have the opportunity to ask questions and provide their feedback on issues in the NDIS market and how they can be solved.

Register to join the To The POint webinar here.

Join the conversation on Facebook: www.facebook.com/groups/tothepointpurpleorange/

Plus, follow us on Linkedin, Facebook, or Twitter: @JFAPurpleOrange

► Our NDIS Review Conversation Series

With the NDIS Review underway, JFA Purple Orange is publishing a series of papers to help stimulate conversations about the future of the Scheme. Each fortnight, we will tackle a different topic of reform that we think is critical to the work of the NDIS Review. We strongly believe that the NDIS is an essential component of ensuring that Australians living with disability get a fair go at what life has to offer, but it must be strengthened and sustained. We are committed to playing a constructive role in developing ideas for reform that ensure the Scheme delivers on its original promise.

► About us

JFA Purple Orange is an independent social profit organisation based in South Australia that undertakes systemic policy analysis and advocacy across a range of issues affecting people living with disability and their families. We also host a range of peer networks for people living with disability including people living with intellectual disability, physical and sensory disability, younger people, people from culturally and linguistically diverse backgrounds, and people in regional South Australia. Our work is characterised by co-design and informed by a model called Citizenhood.

► Media enquiries

If you would like to republish this paper or interview Robbi Williams, CEO of JFA Purple Orange, about it, please contact Marissa on (08) 8373 3833 or email marissab@juliafarr.org.au.