NWX-CFPB HQ

CFPB FinEx Webinar - Financial Well-Being Scale
January 28, 2016
1:00 pm CT


Good afternoon and thank you all for holding. All participants will be able to listen only until the question and answer session of today's call at which time if you'd like to ask a question, you can press star followed by one on your touchtone phone. 

You will be prompted to record your name prior to asking a question. Also today's call is being recorded. If you have any objections, you may disconnect at this time. I would now like the turn the call over to your first speaker today Ms. (Irene Skricki). You may begin.

Great. Thank you very much and thank you everyone for joining us on our FINX webinar today on measuring financial well-being. We're very excited to have you all here today. We had a big number of people RSVP. So we're very excited about that. A lot of energy around and interest in this topic today. Our speakers will be Genevieve Melford from the Office of Financial Education and Hector Ortiz from the Office of Older Americans. I'm going to do a few intro slides to get us started and then we'll go into their presentations.

So and also thank you everyone for your patience with our webinar platform. Hopefully those of you who haven't been able to log in did get the slides that I sent so you can follow along. We are planning to change to a system that works better very shortly but not quite in time for this call. So thank you for that. So I will do our standard disclaimer. 

This call is not legal interpretation, guidance or advice and the opinions stated are the presenters and may not represent the Bureau's views. A quick overview for those of you who've been on CFPB FINX webinars before, I always do the first couple of slides, the same few slides just to get everyone comfortable so you know you're in the right place.

And so just a quick overview about the Bureau, new federal - a relatively new federal agency that helps consumer finance markets work by making rules who are effective, enforcing those rules and empowering consumers to take more control of their lives. And obviously it's on the empowering consumer side that we're kind of talking about today in terms of helping consumers manage their financial lives.

And then just quickly within the consumer facing side of the Bureau, there are six offices. The Official of Financial Education is one of those offices but we work closely with all the others dealing with special population such as older Americans, service member affairs, students, financial empowerment, which is our office working with economically vulnerable clients. And in particular today's topic was done jointly with the Office of Older Americans.

And our standard word about FINX. Presumably most of you are in it since you have been getting its emails. But if not, you can email the box - email box cfpb_finex@cfpb.gov and say that you want to join. We have regular webinars, regular newsletters, some in person convenings that are - we're doing around the country and we're also doing surveys and trying to learn back from all of you what you're doing. So we're very happy that so many of you have joined our - and are participating in FINX.

And also our standard slide, we do have the resource inventory on our Web site that has all of the tools and resources from the Bureau for financial educators. You can get to it at the URL at the top of this slide consumerfinance.gov\adults-financial-education. And actually this is being updated. It will be out the next week or two as a revised resource inventory with all the new things that have come out since the first version of this six months ago.

Lastly, we encourage you to join our financial education discussion group on LinkedIn where we both post FINX and other Bureau materials and urge all of you to post as well. I know a lot of you are in that and I hope more will join. And I'll just note, we do have webinars the fourth Thursday of every month. Our next one will be February 25. We are close to nailing down the topic. So I don't have an announcement about that yet but there will be. We will continue with our monthly webinars. All right, with that we will turn to our presenters.

I will just say one thing. We'll have the presentation first. If you have questions that are kind of urgent or clarifying questions you can feel free to email them. There's a little Q&A box at the top of your webinar screen if you are in the webinar and I will be monitoring those throughout the call and will pose those to the presenters as needed. And then when the presentation is finished, we will open it up for voice questions. So even those of you who are on audio only will be able to ask questions at the end as well. So with that, that's our housekeeping. We will turn it to our presenters. First Genevieve Melford and to be followed by Hector. Go ahead Genevieve...

Okay thank you very much. Thank you all for joining us to learn more about the CFPB's new tool to measure financial wellbeing. Before I say anything else, I did just want to flag for you all that pretty much everything Hector and I will say today is covered in the publication that you see on your screen there if you're looking at the webinar. It's called Measuring Financial Wellbeing, a Guide to Using the CFPB Financial Wellbeing Scale. So don't feel the need to take any furious notes. We'll present this all to you now and we're delighted that you joined us. But if you want to refer to any of the material that we talked about, you can see this user guide on our Web site at consumerfinance.gov/financialwellbeing.

So with that as (Irene) said I'm Genevieve Melford from the Bureau's Office of Financial Education and I'm so happy to be here with my colleague Hector Ortiz. Our ongoing research to better understand how financial education and empowerment efforts can improve the financial wellbeing of consumers is a joint effort between our two offices in partnership with the Bureau's Office of Research. So I just want to mention this is part of an ongoing research partnership between Financial Education, Older Americans, and Office of Research so.

We are now on slide nine for those of you following at home.

So before we go into the main topic of today's presentation, which is a tool to measure financial wellbeing, I want to just take a minute to give the background of the project for people who are entirely new to this with us today. The bottom line is that in order to advance our financial education mission we need to understand what effectiveness means for financial education initiatives. That is what success looks like. And we've taken as our guide for this work the growing consensus that the ultimate and common metric of financial literacy and capability efforts, diverse as they may be in practice, is improvement in financial wellbeing.

That is an ultimate and common goal regardless of the diversity of approaches. And so we have engaged in a multi-phase effort to define financial wellbeing from the consumer perspective and then create a way to measure it and that measurement tool is something that we would intend to be able to be broadly used by both practitioners and researchers. So next slide please.

Slide ten.

Yes thanks. So more precisely, what is this new scale intended to measure? In the first phase of our research, which we published in January of 2015 if anybody wants to read that in more detail. That's a publication called Financial Wellbeing: The Goal of Financial Education. That is also on our Web site and talks about that phase one research in much greater detail. Here I will just tell you that in that phase of research we conducted open ended, one on one interviews with a broadly diverse group of consumers around the United States as well as a wide range of financial professionals like you all, financial educators, advisors, planners, coaches, tax preparers, credit counselors.

And all of those interviews were intended to help us learn what financial wellbeing means to consumers, what it really means in their lives and to ensure that the factors we identified were broadly relevant across adult consumers in the United States across life stage. We interviewed consumers aged 18 to 95. And in short, what we learned from that work was that financial wellbeing can be defined as a state of being where in a person can fully meet current and ongoing obligations, they can feel secure in their financial future and they're able to make choices that allow them to enjoy life. We also learned that that state of being is not strictly aligned with income levels. It's not just about having money people told us. It's also about managing it.

The definition has four elements. First, having control over day to day, month to month finances which really just means being able to comfortably meet your ongoing obligations. Second it means having the capability to absorb a financial shock and we learned that that can be achieved via a combination of savings, credit and the safety net of family or friends. Third it means being on track to meet financial goals whatever they are for an individual and people in different life stages or circumstances have different financial goals. And fourth, having the financial freedom to make choices that allow someone to enjoy life.

And the specifics of what that freedom of choice means for individuals are notable in their variety. And I want to emphasize this here because in coming up with the measurement scale, we had to figure out a way to measure a construct where the specifics can vary a lot by the individual. So I'll just emphasize that individuals we learned value very different things. That freedom of choice can mean being able to be generous with family, friends, and community.

It can mean having the ability to go back to school or leave a job to seek a better one. Or it could mean being able to work less to spend more time with your family. So that sort of variety of what people want to be doing means that traditional measures such as income or net worth while very important cannot fully capture all of the elements of financial wellbeing as we heard about them from consumers. Okay next slide please.

Another way to describe what we learned about financial wellbeing is that it means that your financial situation provides you with both security and freedom of choice in the present and for the future. And this concept, while grounded in a person's financial reality, is inherently subjective because we learned that a sense of financial wellbeing is related to an individual's personal goals and preferences. And this subjectivity is a purposeful and important part of the construct of the concept of financial wellbeing and you'll see that reflected in the design of the measurement tool. 

Okay slide 12.

So once we had a clear definition of financial wellbeing -- that's the definition I just provided -- we turn to developing a tool to measure it. And we had a number of goals for the measurement tool and these included that it would reflect the consumer perspective on financial wellbeing, that it would be a highly reliable and valid measure of this construct. That means that it should provide consistent results if what's being measured doesn't change. And also so that's the reliable part means that it should provide the same kind of score again and again if the underlying thing you're measuring doesn't change. And valid means that it actually measures the construct it's supposed to be measuring.

And finally, we wanted this tool to provide the ability to produce a financial wellbeing score that is relevant across adult ages and contexts and directly comparable across consumers in different circumstances. For any of you who have tried to think about standard metrics you'll know that this is a particularly fundamental and challenging thing to do and it requires that this measurement tool be able to account for variation in individual preferences and circumstances.

You know, our office, the Office of Financial Education, here at the Bureau is charged with engaging in financial education activities and being concerned about the financial decision making capability of all American adults. So we really came at this from a perspective of wanting to make sure that it worked for everyone not to mention the fact that we have a particular interest due to our partnership with Older Americans to make sure that in particular what we come up with is appropriate and relevant for older adults as well.

So we learned from our consumer interviews that financial wellbeing is a multifaceted concept. You may remember there were those four sub elements. And that due to the inherent subjectivity of the concept, a given person's financial wellbeing is not directly observable to anyone else, to practitioners or to researchers or anybody other than the person themselves.

Now a scale is a term that you all may not be familiar with in the sense that we're using it today. By scale we mean an instrument that is used to measure something that is not directly observable such as an attitude or ability. So for us it was really the natural choice to use to measure financial wellbeing once we learned more about just what the consumer perspective on financial wellbeing was. A scale is made up of multiple questions. So part of the definition of a scale in this sense is that it's always more than one question. And it also incorporates a scoring procedure, a procedure to produce one score that incorporates answers to all of the questions. Next slide.

Okay. Slide 13.

Okay so I feel like I'm moving along nicely. So I hope you don’t mind if I pause here for a minute. (Irene) warned me that the audience for today's webinar may not really want to get into the weeds of the methodology. So I'll ask you to bear with me for a minute here. I want to at least give you a sense of the care and the scientific methods that went into creating this measurement tool. Hopefully you'll never need to explain to somebody else everything that we did here but I do want to give you some confidence in it before we describe what it is.

So first of all we worked with the same terrific team of subject matter and scale development experts that helped us create the consumer driven definition of financial wellbeing. So the team for phase one and the team for phase two were one in the same and they were very purposefully linked in their inquiry. 

That contractor team was led by CEFD and included the UW Madison Center for Financial Security. Vector Psychometric Group led the development of the scale model and scoring procedures using methods based on item response theory or IRT. IRT is primarily used in educational testing and psychology and it provides a scientific method of creating especially precise estimates of unobservable characteristics which is why we were thrilled to be able to work with experts in that for our scale development.

And Gen the SAT and the PSAT are basically scale item response theory right?...

Right all of the stand. Right yes absolutely...

Just to give people a sense of.

hankTs (Irene).

Just something they're familiar with even if they don't realize it.

Right, right. So I would say yes the IRT methods which allow individual responses to individual questions to contribute differentially to an overall assessment. That's the hallmark of item response theory and that's what's used in most standardized educational testing, absolutely. So before stage process was used to select the ten questions that make up the scale and I promise you those questions are on the next slide. I won't keep you from them much longer.

So first a large pool of almost 50 potential questions was developed based on the consumer interviews and a review of related questions used in other surveys to date. So that means that we really thought about how did people talk about financial wellbeing in their own words. What are ways that we might be able to ask questions back to consumers related to what they initially told us and what we understand financial wellbeing to mean to them? So we wrote a bunch of items. We pulled items from existing surveys on related topics. That's how we put together the first big pool of potential questions.

Then we tested those questions through a process called cognitive interviewing which means that we asked people who were totally, you know, unfamiliar with the process, not connected to the project and a diverse range of people, to read the questions, explain what they thought the questions meant and explain how they would answer the questions and if they found any of the language confusing. So some of the questions from the initial pool were then revised based on the findings of those cognitive interviews. So that was kind of the qualitative part of the scale development process.

Then we moved to the quantitative analysis. The large pool of candidate items was then fielded through an online survey of thousands of respondents and analyzed using factor analysis to determine which questions best measured the financial wellbeing construct and several more rounds of surveys and item response theory analysis were then conducted to further refine the selection of questions to the ten that provided the most reliable financial wellbeing scores and also to figure out how each response to each question contributes to the overall financial wellbeing score.

And as a final check once the scale had been put together and the scoring procedure developed, the scores were then compared through other data that we collected in the same survey to theoretically related concepts like income, credit score, self-assessed financial situation and experience of material hardship to make sure that our scale was actually measuring the intended concept. 

So we made sure that, you know, for example people experiencing a lot of material hardship would not score highly on our scale. You know, income is no means the full story but certainly people with higher credit scores and higher incomes do rate higher on our financial wellbeing scale. So that's just kind of a validity check to make sure that the scale score is performing in the way you would expect it to. Okay.

Are we ready?

We're ready. Thank you for bearing with me. Next slide.

We are on Slide 14 

Okay so without further ado here are the ten questions that make up the scale. I know some of you are looking at them either on the webinar or on the PowerPoint but I - it's a short scale and I'm going to read it in case anyone is just listening on the phone. So for the first six statements, the respondent is asked how well does this statement describe you or your situation. And their response options are completely, very well, somewhat, very little and not at all. The statements are one, I could handle a major unexpected expense. Two, I am securing my financial future. Three, because of my money situation I feel like I will never have the things I want in life. Four, I can enjoy life because of the way I'm managing my money. Five, I am just getting by financially. And six, I am concerned that the money I have or will save won't last. So those are the first six statements.

For the last four statements - so together this makes up ten. The last four statements the respondent is asked how often does this statement apply to you. And the response options are always, often, sometimes, rarely and never. These statements are one, giving a gift for a wedding, birthday, or other occasion would put a strain on my finances for the month. Two, I have money left over at the end of the month. 

Three, I am behind with my finances. And four, my finances control my life. So the respondents answers to all ten of these statements can then be used to produce one numerical score and Hector will walk you through the process for doing that in just a few minutes. Okay next slide.

Okay. So now let's talk about putting the scale to use. You know how we developed it. You know what the questions are. You know what a scale is. So this scale to remind you it's designed to allow practitioners and researchers to quantify and therefore observe something that the respondent already has a sense of but no one else can directly observe it about them. So the purpose of the score produced by this scale is to help professionals quantify and compare scores across time or across individuals. 

By that I mean see how somebody is doing over time and how their score might change or to compare to individuals, to groups of individuals, etcetera, and also to study the relationship between financial wellbeing and other factors, which is a question of great interest to us and from my conversations with a lot of colleagues outside the Bureau is of interest to a lot of others as well.

So the scale can be used by practitioners to take an initial assessment of a client or to track their progress over time. In addition to assessing a person's financial wellbeing at intake so the first time you meet with someone you could have them fill out this questionnaire. You could produce a score. You could keep a record of that for yourself and you could, you know, think about what that means in the context of your serving the client. But also reviewing the individual questions that make up the scale with a person that you serve can also help guide a conversation about their financial situation. Both their strengths and their needs in terms that resonate with and motivate consumers.

Remember the wording of all of these items originally comes from our open ended interviews with a very diverse group of consumers. So the hope is that in addition to the quantifiable nature of the scale, the individual questions may be a sort of a useful and resonant way to open some conversations that may be relevant in your practice. And of course the scale and the score produced by the scale can be used to track changes in an individual's wellbeing over time.

And so that then relates to the next point which is that the financial wellbeing scale provides a tool to measure the extent to which programs or any other financial capability initiatives is improving the financial wellbeing of the individuals that they serve. To that end, the scale could be used as part of reports on the effectiveness of programs and services such as financial education and capability programs. 

It could also be used to compare different populations in one program. For example how a particular intervention differentially affects different people or to compare changes in financial wellbeing across programs.

So that's what neat, to go back to this point about the way in which our sort of large and diverse field serves consumers to support their financial decision-making and financial outcomes vary so much. One program might be explicitly working on credit building, someone else on savings, you know, someone else on, you know, bill payment or any number of other things. 

And this financial wellbeing score is something that can be used as kind of a common metric across programs that may have a very different kind of internal performance metrics depending on the goal of the program. And then of course finally it can also be used by researchers in survey research to study the relationship between financial wellbeing and other factors or in their program evaluation efforts.

I'm going to go to the next slide in just a second because Gen is giving me that look that says go to the next slide. But I just want to note based on things I've heard from people before, this is a subjective scale. It is not measuring credit score and to all the things you guys may be measuring in your programs. You may say hey it's not there. This is not meant to be used in and of itself. I know you're going to be saying okay. All right I jumped the gun. I'm sorry but I know people look at the questions there's like but where are the numbers.

Right.

Okay.

Thank you for that nice setup (Irene)...

Oh look okay. Sixteen, slide 16.

So as my final note - this will be my last slide and then I'll turn it over to Hector. I wanted to emphasize precisely (Irene's) point that this financial wellbeing scale is a compliment to not a substitute for existing program metrics or direct outcome measures relevant to a program's goals, which again could be credit score, could be debt reduction, could be savings increases or any number of other things.

And so every program has other things that it needs to track for their operations, for their performance measurement, for their reporting purposes. And again this is really meant to be a compliment to that. And so what I want to emphasize about the complementary nature are the following three points.

So this scale really adds three things in a very complementary way to existing measures of program performance that you are likely already using. The first is that it's a holistic outcome metric. So that's unusual. It's sort of this overall sense of success and a holistic outcome metric that reflects success in a consumer's own terms and allows for variation in individual preferences and goals. And that's a huge one. That's very, very hard to do. And as (Irene) pointed out, which is completely true, we're not asking people hard numbers. We're not asking their credit score. We're not asking if they have three months of savings.

We're not saying that these things don't matter. We wanted to have questions that people could apply to their own situation and could be equally relevant across different adult situations. So that's why purposefully you don’t see any kind of hard numbers in here. And I think the value here is that because you don't see those hard numbers this allows for variation in preferences, goals, and circumstances and it's why it's such a nice complement to, you know, the hard data that we all traditionally like to have for good reasons.

Secondly, it's a rigorous and simple way to measure important but traditionally hard to quantify success factors like feelings of empowerment, confidence, and satisfaction. I think we all value those things. We all know that we care about them for clients and consumers broadly. But traditionally it's been very hard to kind of have a rigorous and simple way to quantify those in a way that you can then use in a sort of a reporting or a more rigorous quantitative analysis way. So this does contribute that as well.

And finally I think I mentioned this before but this measure can be used as a common metric across very different types of financial capability programs and interventions. So those are all the reasons why I would make a pitch that this is deeply complementary with what folks are already doing and add some important new information that I think all of us are going to care about. So with that, I will turn it over to Hector.

Great. And I just want to note there are a couple of emailed questions that I've gotten but I'm going to hold - I’m going to wait until after Hector because I think they'll - they may be either partially answered or they'll fit better. So those of you who are emailing questions keep doing so but I will get to them in a minute. Okay we are now on slide 18.

No you want 17.

Oh sorry, slide 17.

Good afternoon everyone and thank you Gen and (Irene). I'm Hector Ortiz and I am with the Office for Older Americans and we're part of this project. And one of the things that we're hearing to see us - to see Gem mention is this idea of understanding how financial wellbeing connects to other factors. In our particular population how it relates to many of the financial capability programs that are being adopted for older consumers and retirees, certainly issues related to fraud, cognitive decline, health issues, fixed incomes. So a lot of opportunities with this scale.

I will be discussing today how the financial wellbeing scale scoring works but also how to interpret the scale. As Gen mentioned, the score that we use for the financial wellbeing scale it's based on an item response theory analysis. And this method very similar again to the SATs and other standardized tests. What it does it gives us a more precise individual estimate because what it does it allows different items. So each one of those respondent - responses to contribute differently to the final score.

The IRT based scales are typically scored automatically and, you know, virtually through a software but in this case what we did was we created a look up table to make the scoring process available to users and practitioners like you who may not have access to that specialized software. In order to do this, it requires what we call a two-step process and I will be demonstrating how that two steps work. The first step generally consists of adding up what we will call the raw or total response value of the - to the different questions. And the second step will use the look up system to find the corresponding final wellbeing score on the scoring sheet.

And one of the key ingredients of that process is to also look at that score and in terms of its difference based on the respondents age as well as whether the questions were administered by the individual to himself or herself or if it was administered by someone else. And so I will show an example of that. Next slide.

Okay and this is slide 18.

So this scoring sample in step one it's for a consumer between the ages of 18 to 61. These are consumers that we generally call working age Americans but again this example could have been done also for the older consumers. What you will find in both the user guide as well as, you know, our Web site a set of scoring sheets. And the first sheet contains the scale with the spaces where you can record the consumer's answer. Or the consumer can administer the scale to himself or herself and censor the different questions.

The second step of the scoring process once you have recorded those answers, those answers have a numeric value that needs to be added up. One important thing to note here as you can see some of the answers are reverse coded. So you will have places where the value of four belongs to completely while the value of - well some completely value is zero. So be mindful of the reverse coding when doing this step one. Next step. Next slide (Irene).

Yes, slide 19.

Thank you. And the second step is where once you have those answers and you have recorded the total score, the next step is adding up all of those responses -- in this case it's a total response value of 12 -- and using the lookup table based on the age of the consumer as well as how the questionnaire was administered. In this case there's two modes. One is self administer and the other one is administer by someone else. 

The way we tested this was through the phone. In this case we're - it's a younger consumer, again 18 to 61, who self-administered. The total response value of 12 corresponds in the lookup table to a total final financial wellbeing score of 40. If that individual had been an older consumer that value would have been 42. Next step. So when it...

Sorry.

Slide 20.

When it comes to interpreting the score -- no worries -- the financial wellbeing scale score is a standardized number between 0 and 100 that represents the respondent's underlying level of financial wellbeing. The number itself doesn't have a meaning on its own and most people's scores certainly will fall in the middle to like typical curve with extreme low values and extreme high scores being very uncommon. A higher score in our scale certainly means an indicate - indicates a higher level of measure of financial wellbeing but there is no specific cutoff for good or bad financial wellbeing score.

The scores are new so they have not been around long enough for research to establish a meaningful range for different levels. But also there is an opportunity for professionals like you to establish your own benchmarks as you use the scale with your clients and analyze those scores in relation to other data that they may have in their financial situation but also as an aggregate for clients. Next.

And so generally it's important to discuss what broadly this scale does. Generally it - first it's unique because it provides consumer driven measure of financial wellbeing that it draws from the insights from both consumers and experts. Secondly, it's a measurement tool that was developed using state of the art techniques including cognitive interviewing, testing to ensure the accurate comprehension of the questions. And the scale is highly reliable and valid measure of financial wellbeing construct based on multiple ways of quantitative testing.

Third, it's a common metric that will allow us to do apples to apples comparison of scores across consumers. And as you can see not just across consumers of different ages but also consumers depending on the mode that you administer the scale. The scale can be used to assess a current state of financial wellbeing but also can be used to track progress over time and to understand how other factors, program interventions affect financial wellbeing. Lastly and it's important to mention this, it is free. It's a publicly available instrument and measurement scale. The guide, questionnaire and scoring sheets are in our Web site. And you're welcome to use it. Contact us again if you have questions.

Great. Thank you so much Hector and Gen and I just want to add to again that there - the scale there's no sort of right answer to it. Higher is better but each of you in your own practice may see that gee in my experience people who have a 60 are pretty good or maybe they need an 80 or maybe people of certain other characteristics may do better with a certain score. So it's really it's a tool you can kind of use in your own way but at least you can compare across your own populations and those of others. But it's really kind of requires you to do some work to figure out how to really make it work in your own setting until we have more kind of national data on that.

So what I'm going to do is we have a bunch of emailed questions which are very interesting. But first I’m going to have the operator give instructions for voice questions so you can all start to tee up voice questions as well. And once she does that, I will start to read some of the webinar questions for Gen and Hector to answer while you are also phoning in your voice questions. Operator can you give us instructions?

Sure. Thank you. At this time if you'd like to ask a question, please press star followed by one on your touchtone phone. You will be prompted to record your name prior to asking your question. To withdraw your question, you can press star two. Again that's star one to ask a question.

So we have several questions which are fascinating and they range from I'm concerned about this, this is weird, to this is fantastic, which is great, which makes a great discussion. But I will - I'll start reading a couple of them while others of you may be teeing up your voice questions. The first one and this I think is the one we kind of expect. Can you compare and contrast this scale to the financial capability scale from the group at University of Wisconsin Center for Financial Security? Genevieve?

Thank you. I will take that great, great question. I really appreciate it. The simplest way to answer it is that they are measuring different constructs although theoretically related for sure. So our scale let me start with is meant to be measuring someone's experience inside of kind of current state of being. And the idea in theory is that people with greater levels of financial capability would then be able to, you know, take actions and order their financial lives such that they would be experiencing greater levels of financial wellbeing.

So our financial wellbeing measurement is definitely not the same thing as the concept of financial capability. It's more the state of being that's meant to result from higher levels of financial capability. So the financial capability scale is a terrific resource from University of Wisconsin. It was also quite thoughtfully and iteratively developed which is meant to be a tool to measure that construct of financial capability by which I think they mean the behaviors and attitudes that would support having higher levels of financial wellbeing.

Great. Thank you. And actually because I know there may be some, can you also tell us how it differs from the Center for Financial Services Innovations, Financial Health scale or survey...

Absolutely. Another great question (Irene). And I would say that our construct is deeply complementary with that one. Those of you who are familiar with the financial health metrics work that CFSI is doing they are really looking at those nuts and bolts kind of financial ratios and more traditional objective financial metrics. So that's incredibly complementary. They're focused on these kind of objective metrics of financial health and, you know, debt to income ratio, total amounts of savings that financial and their work is geared at financial institutions to help them understand the sort of objective financial health of their consumers.

And again the difference is that our work is focused on this more subjective construct of, you know, if people are experiencing security and freedom of choice. And so the value of that objective information is huge and we're adding how, you know, people are experiencing their lives. And, you know, ours allows for I think a broader range of variation again in preferences and goals because it's non-objective and there's adds that, you know, extremely valuable information of being the more traditional objective kind of financial planning metrics.

Great. Let me just see if there's a phone question. I have a bunch more emailed ones but let me ask about phone.

At this time, I'm showing no questions.

Okay. Can you again say it's star?

It's star one to ask a question.

Okay great. All right let me take - let's see. There's so many to choose from here. All right I'll do the - I'll do it in the order in which it came in. So someone said I looked at the question and they were great but I found the choices for the answers to be very weird. I'm concerned. How do you answer the question - how do you answer questions well? I'm not quite sure what.

Yes I'm not sure what well means but the answer categories essentially it's not a yes, no. It's sort of not a binary I am concerned or I am not concerned. It's kind of like how much of an issue is this for you. How well does this statement describe you? So I mean we went through a lot of testing the answer choices and the framing of the, you know, these kind of how well does it describe and how often is this your experience. 

You know, we pretty carefully tested those, the option responses as well to make sure that these were the ones that were contributing the best value. So I guess all I can say is there's a range of options. People can choose the answer that they think most represents their experience and their situation.

Great.

And we have very much tested the individual responses and response options for their reliable contribution to the scoring.

And then on the flip side we have another question saying these questions are fantastic. Do you have a national benchmark number to track against regularly?

Well a national benchmark number is one thing, regularly is another. What I can tell you is that this is, you know, brand new. We didn't say this and some of you may not realize this but we just published this scale and scoring procedure about six weeks ago. So it is brand new. That's why Hector was saying that data has not been collected that would allow us to kind of establish levels and benchmarks. 

That being said, the Bureau is actually - we are in the midst of designing a nationally representative survey to get for the first time benchmark population metrics of financial wellbeing using this scale and scoring procedure. So stay tuned. I don't think that data will actually. I don't know if you know the pace of government research but we are starting that project and, you know, information on that probably will be available in another couple years.

Right. And I will say there are also others who are planning to use these questions in other survey research. So we may get also numbers from other...

That's right. We very much welcome that and to the extent that other people are farther down the road with their survey projects and have the ability to gather data using the scale more quickly. We know some people are considering that and we, you know, welcome that and think that's fantastic.

Okay. Next question - actually let me pause again. Any phone questions operator?

Yes we do have a couple of questions. (Kathy Swiedler) your line is open.

Oh (Kathy).

(Kathy Swiedler) of University of Illinois extension, your line is open.

Thank you. I'd like to go back and ask for a little more clarification between the financial capability scale and this financial wellbeing scale. As an educator in the field can you give me maybe an example of when I might want to use one versus the other?

That's a great question. I'll start out by saying that again they're measuring different constructs. So I think I would have to say I think that the financial wellbeing scale is very broadly relevant across a range of any financial literacy or capability service or intervention serving any adult. And I do want to emphasize that this is an adult construct and it's one that was defined by and the scale tested on adults. So I would not recommend the use of this financial wellbeing scale with young people who are not kind of managing their own finances. 

But really any financial literacy capability or other type of intervention serving any adults in the U.S. generally I would say this scale is relevant for in terms of, you know, understanding the connection between understanding the financial wellbeing of the folks being served and understanding the connection between the intervention and improvements in financial wellbeing.

So I feel like I can speak much more confidentially about when I think it would be appropriate to use our scale. I don't really want to put words in the mouth of the folks at UW about the circumstances in which they think it's appropriate to use their scale. But it was created with a specific set of kind of empowerment type programs, financial coaching and other things with an eye towards kind of the intermediate program performance metrics that would be relevant for those programs. So I think I'll just have to leave it there and say it just depends. I really would think it depends on which construct you're looking to measure in terms of program outcomes. Does that make sense?

It helps some. Thank you.

Yes I mean one of them is specifically looking at behaviors and attitudes and the other is looking at your experience of your financial situation. So I think often they would be very complementary and it would be very interesting to know if you can improve on the financial capability scale. Does that also improve your financial wellbeing? Certainly we would theoretically think so but we haven't tested that in the field.

Right and I mean I'll just add something because we've certainly heard this question before which is that the financial capability scale asks some sort of self-reported but objective behavioral questions. Have you paid late in the last two months or something to that effect? So that in some ways is going to measure a set of financial stress perhaps if someone's not paying their bills on time but it's not going to measure the subjective sense of I'm managing my finances well. I mean again the financial wellbeing scale is a subjective measure of someone's own sense of wellbeing. The financial capability scale is saying if you're paying your bills late that's a problem behavior.

Right.

You may interpret it and probably you're stressed and you're going - your wellbeing will be lower. But it's there are...

Yes. I would just guess in most - I mean I don't think that there - that these two scales are really ever a substitute for each other because they're not measuring the same thing. Often I think it would make sense to use both if the behaviors and attitudes that are measured in the capability scale are something that you think should be an outcome of your program intervention.

That helps a lot. Thank you.

All right. We've got - let me take another - here's a - it's very straightforward one. I'm curious about the scoring difference for someone who had the questions read to them versus did this on their own. What is the thinking behind scoring these things on a different scale?

Okay. So I'll answer that in two stages. The first is that it's, you know, a pretty well-known phenomenon in survey methodology that there is kind of a systematically different way that people answer questions if they read it to themselves versus if it's read to them by someone for else for two reasons. The first is this sort of maybe you want to please the interviewer and you're more likely to have your answers, you know, seem more social acceptable or something. So there's the possibility of that type of bias. And the other is just literally a kind of memory thing related to whether you hear something or see it written.

So, for example, if I gave you a series of options verbally you'd be much more likely to remember the final option I gave you whereas if you were ready all of them at once, you know, you'd be just as likely to realize that you could answer all five of those options. So there are just some kind of well documented systematic reasons why people may kind of systematically respond to questions slightly differently depending on if they read it to themselves or someone else read it to them. 

So that's the theory behind why we investigated if that would be the case here. And so in order to investigate that we did test the questions both online where people self-administered and in phone surveys and we did find that as expected there was a slightly systematically different - slightly different way that people responded. And so the scoring accounts for that.

Just out of curiosity is it always higher if it's in person or is it - it does not align in such a simple way?

I don't think it's so simple.

Okay. Got it. Yes...

It's just that you - there is good reason to think it might be different. So you should check for it.

Right...

But we checked for it and it was there and that's where our scoring corrects for it.

Okay. All right. Let...

I say that as not actually a survey methodologist. So that is my answer.

All right I still have more email questions but let me do another check for phone questions. Operator do we have any?

We do have another question. (Alan Joseph) your line is open. (Alan Joseph) of Gesher your line is open. Can you please check your mute button?

All right his was one of the emailed questions. So maybe he felt he was - it's been answered. Okay let me ask another one of the questions through the Q&A function here. How do you recommend that organizations use this scale to track aggregate changes in financial wellbeing among clients? Can you provide an example of how those results might be conveyed to stakeholders?

That's a great question. Hector do you want to jump in on that or should I?

Yes go ahead Gen and I will have other thoughts...

Okay. So that's a great question and actually in - I have a pretty simple and straightforward answer to it. Because these scores can be directly compared, as long as you properly calculate each individual score, and by that I mean, you know, as long as for each individual you get their score by using the proper age category and proper mode category. And for practitioners I don't know if you're likely to be using both modes.

So it may not even be an issue. But if you are the point is every time you assign an individual financial wellbeing score, just make sure that you're using the proper age category and the proper mode category. And then once you've done that, the scores are all directly comparable to each other which means that they can be aggregated completely. There's no problem with doing that. So in the simplest way, you can just, you know, do any simple summary of statistics you want aggregating or analyzing all of those individual scores together.

And the question of how would you report on them it's a great question. So this hasn't been used in the field yet. The things that would come to me top of mind are, you know, you probably would want to talk about aggregate changes in raw score and aggregate changes, you know, percentage wise, any kind of distributional differences that you notice in scores changing for different groups of people if you want to.

And the other thing I'll mention for the people that are interested and thinking about it in this kind of way is that, you know, based on the many thousands of people that were part of the original testing of this scale, which was a large and generally diverse pool of American adults, although not technically representative because we weren't using a perfectly representative sample. The mean score is around 50 and a standard deviation is about 15.

So if you were interested in thinking about did someone improve, you know, how much of a standard deviation, here's it about 15. So for people who would actually want to get into that in their reporting, I'll tell you that. But otherwise, you know, some simple my group, you know, is improving this many points or this percentage of points might be fine too.

Right. And then I'll actually answer a different version of that question which is if you consider stakeholders to be your clients in discussions with some practitioners we've already had, people have kind of brainstormed as a group and say we don't think giving the number to clients is a good idea. Again this is up to whoever uses it however they want to use it. 

But the idea that because the number there's no benchmark yet, which someone raised earlier. There's no benchmark. There's no sense of whether 50 may be average but is that good or bad. The practitioners that I've talked to have said we would tell consumers you're improving or, you know, use it in other ways or give it, you know, color codes or something other than saying, you got a 50 which...

Right.

...if you are used to an academic world 50 is failing and that's not what this is meant to say. So that's just a thought about, you know, if and how - I mean...

Right.

...if and how you want to communicate it to your clients obviously that's a discussion...

Right. That's a really important point. An individual score is totally meaningless in isolation right now especially because we don't have any nationally benchmarked data. So exactly. I think talking with a client you may say, you know, okay this is how you answered the questions. Would you like to be able to answer any of them differently? 

Is that a goal you have? So we can now talk about steps you could take and then the next time you see them if you administer it again you can say oh, you know, it looks like your financial wellbeing has improved. It looks like you're answering these questions differently. So yes if the point is to talk about it with the client I would do it in that way and not focus on the number.

Right and just the other thing practitioners have said so far is that separate from all the cool research stuff and all the things Gen talked about, just as a coaching tool or a conversation starter to say gee you said buying a wedding present would be - would put a strain on your finances. Let's talk about that. Or, you know, don't feel secure - your finances control you. What does that mean to you? Let's talk about that and start the conversation. 

You know, whatever the numbers are the questions themselves I think could be helpful in that context. So let's see if we can squeeze. We just have a minute left. If we can squeeze any of these other questions in. There was one about whether we're going to do a large scale - a larger scale assessment and I think for your survey we are. And then...

We are and other people might be and that'll be terrific...

Right. Right. And then when using it as follow-up in a group setting is it necessary to match up before and after individual scores? Are we okay to compare the average before or after scores to the group?

So that's what I wanted to make a quick contribution (Irene) that we say this in our Q&As in the actual guide where we note that it's important to know that small changes in the scores due to answering the questions it's - it happens but at the same time there is typically a margin of error associated with the testing of this scale. So sometimes those small changes are not meaningful on themselves. So there's again in our user guide we provide other key tips about considering - individuals who are considering and professionals considering using the scale.

Great. Okay we're just about out of time. Out of curiosity, operator do we have any other phone questions? I hope the answer is no.

I'm showing there's no questions at this time.

Excellent.

Can I actually answer the question about averages?

Yes.

So thank you for that question. Let me say this. I think that as long as the same people were part of the group at both times that you surveyed them, there's nothing wrong with taking the group - a change in the group average. You don't know what happened to any of the individuals but it's reasonable to say the group overall was doing better than they were at the beginning as long as all of the exact same people were surveyed before and after if that makes sense. Because otherwise changes in the group composition could be driving the effect. Thanks.

Great. Well we are right at time, just a minute after. So I want to thank everybody. Again this is a new thing. There's a lot more learning to be done about it. We'd like to have all of you kind of participate in that learning by trying it out in practice. If you are interested, it is all available on our Web site as Genevieve said. 

And again all of you who are part of FINX already, the link was also included in that little newsletter that went out earlier this week. So thanks again. We really appreciate the speakers and all the participants and we look forward to having you hopefully on our next FINX webinar. Thank you very much.
That concludes today's conference call. You may disconnect at this time.


END