FICO: Not a Great Default Predictor
Joe Nocera has an interesting article in the NYT about FICO credit scores. Banks rely almost exclusively on the FICO scores when making loan decisions — despite the many flaws in the FICO scoring process:
“FICO scores are not the best predictor. The amount of equity a person has in his home, his debt-to-income ratio, his job stability and his cash reserves are all better predictors than credit scores, according to Dave Zitting, the chief executive of Primary Residential Mortgage, a leading mortgage lender. And yet, he said, “The credit score has become the line in the sand for the banks.” (emphasis added)
Interesting stuff . . .
>
Source:
Credit Score Is the Tyrant in Lending
JOE NOCERA
NYT, July 23, 2010
http://www.nytimes.com/2010/07/24/business/24nocera.html
Tweet
Facebook
Reddit
Digg this!
July 24th, 2010 at 9:41 am
I disagree with the article. FICO scores are just about the only reliable tool in underwriting actually. Statistically speaking. Mortgage lenders and brokers like to think they know how to judgmentally underwrite loans. Show me some data. The lender here claims that there are other things more reliable than FICO such as equity, job history, etc. Where is the data for this claim??? Anyone???? There isn’t any. FICO models were created by starting from the back end (delinquency and default) and working backwards. It’s all data based.
Mortgage brokers are sales people who get paid a fee only if the loan closes. They know as much about credit risk as I know about flying a space shuttle.
July 24th, 2010 at 9:49 am
There are algorthms for FICO —
There are apparently no standardized algos for home equity owned, debt-to-income ratio, job stability or cash reserves.
Sounds like a business model waiting to be acted upon — someone should build that and sell it to Fair Isaacs . . .
July 24th, 2010 at 10:00 am
“The credit score has become the line in the sand for the banks.”
Weren’t credit scores available for purchase (well lease) during the credit boom?
July 24th, 2010 at 10:34 am
@BR: You are correct. You would be AMAZED at how primitive and undeveloped mortgage credit underwriting is. Amazed.
July 24th, 2010 at 11:04 am
am i the only one who thinks that people might be even more disappointed if the other metrics were used?
first, i would guess that the VAST majority of people applying for mortgages have zero home equity – that’s why you apply for a mortgage, because you’re buying a home… so cross that one off the list (Nocera spent most of the article talking about mortgages, and i’m assuming that most mortgages are first mortgages).
then, job stability? how the heck do you quantify that? i can’t imagine that factor wouldn’t be over-estimated. cash reserves? again – i’d guess that this is another area where most loan borrowers are relatively light, and might not want it factored in.
the real point is that FICO, like all other metrics, should be used as an INPUT, and not a DECISION MAKER. i agree completely with Nocera that it’s asinine to reject someone with a 619 while accepting a 620 – but that’s not the fault of the FICO algo, it’s the fault of the lender!
July 24th, 2010 at 11:07 am
Tanta of Calculated Risk wrote an entire series providing a detailed insider’s look on the ins and outs of real estate finance. She discusses the specific issue of the failure to develop a mortgage scoring system here based on a previous discussion of FICO problems WRT real estate here.
At the serious risk of bowdlerizing her excellent writing (may she RIP) the short version is the use of FICO made it easier to automate loan origination making the process cheaper and faster (and easier for brokers to game) by removing both the need for skilled underwriters and the time it took them to conduct due diligence WRT buyer fundamentals so development of a scalable mortgage scoring system never got off the ground. Folks tend to forget how local and in many cases how personal all aspects of real estate including financing were until the securitization/derivative explosion this past decade.
July 24th, 2010 at 11:17 am
I agree with Rescission. When properly used, the FICO credit scores filter credit history repository data into a meaningful spectrum of credit risk. It’s a relative spectrum and needs to be adjusted for other relevant facts that are either not available in the credit history repositories or “formally” available but not reliably (like income total debt ratio, borrower liquid reserves after closing, and the other factors mentioned by Dave Zitting. Lenders and credit guarantors can and do develop their own enhanced credit spectrums, based on their own experience and experience accumulated from other providers of such archival evidence.
FICO scores are presented as predictors of relative default likelihood over the next 24 months, and demonstrably have predictive value beyond that time horizon, but with diminishing strength (call it the “score shelf life”). When the macro economy (or the regional economy, for those who underwrite in smaller regions) gets whacked, the ‘shelf life’ may be seriously reduced. We call this phenomenon the “compression effect”, and note that the “strongest” scores tend to become “strong smelling” (pass their ‘use by’ date) faster in such situations. The private mortgage insurance industry commented on this effect in early 2008,
http://www.federalreserve.gov/SECRS/2008/October/20081031/R-1318/R-1318_4_1.pdf
This was one of many comments to the regulators over several preceding years, not that they listened very hard, when it might have made a difference: http://www.privatemi.com/news/media/20100202.cfm )
July 24th, 2010 at 12:07 pm
I can’t help but detest the FICO score system with a passion. I appreciate that they came up with an algorithm that makes it efficient for lenders to judge my credit worthiness without really having to know me, but it really needs more studying and tweaking. Example, I closed on a home April 2nd, 20% down, at 4.875% through an online discount broker. We’re going to be here a decade or more and I decided I wanted to refi at 4.375%. The only thing material that changed about my credit profile is a modest increase on one credit card that I requested. My wifes credit profile is unchanged. To any rational person, our credit profile is identical to what it was three months ago. At that time her average score was 765 and mine was 753. The report that was ran last week now has her average score at 735 and mine at 725. Fortunately, my discount broker went half with me on his yield spread for what would be the additional fee he’d have to charge me for our scores being below 740. It pushes the ROI on this endeavor out a few months, not a big deal but it’s money I could have spent somewhere else into the economy rather than to line the pockets of the financial industry. That really pisses me off.
July 24th, 2010 at 12:46 pm
With significant equity one would naturally try harder to keep the house, rather than defaulting whenever rate resets, or the income stream slows down, or just because house is worth so much less than the loan. It’s a whole different ballgame when people/investors have their own money/home equity “in the game.”
Whether or not you like the use of of statistics and math formulas in finance, I think you don’t need to consult NN Taleb’s books to see that a lot of them failed to be accurate predictors. If soemone ahs an accident and is in a hospital for a week, and doesn’t pay 4 bills on time, his credit will be destroyed, but he would still be a lower credit risk than someone who have “shakier” job and lower down payment.
July 24th, 2010 at 1:32 pm
Then there’s the issue of employers using FICO scores to decide who to hire.
http://www.calwatchdog.com/2010/06/11/new-bill-to-ban-employer-fico-use/
July 24th, 2010 at 1:53 pm
Here’s how dumb banks are. At a former job one of my duties was to respond to banks needing to document the salaries of our employees. Two of the questions on the form were the date and amount of the next raise. Any response to those questions except “unknown” generated three more identical forms which had to be filled out almost identically in ink (photocopying the unchanging info was not acceptable to the banks for reasons of ‘policy’.) The first form was to show the salary of the employee just prior to the raise. (Hello- it was of course the same as the salary I had reported on the first form.) The second form, due at the next pay date, was to show that the raise had actually been given. The third form, due at the following pay date, was to show that ‘the raise had not been rescinded’.
I asked one of the bankers one time why the three forms were required. He said three forms would prevent me from lying. I responded, “If I’ll lie to you once, I’ll lie to you three times.”
Reliance on FICO or any other number is simply CYA for the bankers. If you set up a simple rule, then no one can blame you for a bad loan so long as the rule was followed, even if anyone with a pulse should have known the loan should never have been made. Bankers are being trained to have all the freedom of judgement of workers at McDonalds.
July 24th, 2010 at 2:04 pm
I’ve kind of promised Barry a “Why I Hate Banks” article (though in truth it could be a whole series). In the meantime, a snippet that’s somewhat relevent…
I recently joined a Credit Union. I could have done this years ago. What prompted this action was the fledgling movement to move away from the big banks to local ones. I’ll save the unbelievable “thrills and spills” that ensued, save this…
I get a letter recently from them saying that because of a credit check, they won’t honor any checks written against an account with insufficient funds.
Nobody asked them to honor bounced checks. They took it upon themselves to do a “hard” credit check on a checking account. We have no credit with them. We have never applied for credit, and have no intention to ever ask for credit.
So what happens? The FICO scores drop because somebody did a “hard” credit check.
Great system.
Next time: The Great Vanishing Money Trick
July 24th, 2010 at 2:13 pm
Despite the shortcomings credit scores are excellent predictors of default in every statistical model we have seen at UFA (www.ufanet.com). The problem in this cycle was not credit so much as geography, especially local economic conditions. Most of the losses this credit cycle are explained by the change in local economic conditions not by changes in credit scores.
For a comprehensive geographic score that addresses these risks see http://ufanet.com/FSLocation.htm
July 24th, 2010 at 3:01 pm
bergsten’s comment reminds me, I was in the market for a mortgage a year ago and applied to two reputable lenders to see which one would offer the better deal (what is advertised is not always the case when the required disclosure docs are produced pending approval of application).
Both lenders naturally ran a credit check and, lo and behold, my FICO score dropped as a partial result. I say partial because along with the ‘hard’ credit check the score was apparently also reduced somewhat when it was revealed I only had a couple lines of credit open, had not opened any recently and had retained one too long (my Visa card for crying out loud).
To add some additional weirdness one of the loan officers seemed genuinely bothered that I had applied to another lender so I had to (gently I assure you) remind her that she was in the business of selling money and it was nothing personal: I was only doing what any rational consumer would do, reading the product label including the fine print and inspecting the contents.
July 24th, 2010 at 3:02 pm
Turns out the Nationally Recognized Statistical Rating Organizations (S&P, Moody’s and Fitch) were not good predictors of default either …. and they had models with sophisticated algorithms.
July 24th, 2010 at 7:01 pm
Underwriting? Don’t make me laugh. Underwriting is dead. Our anecdote is fun. I have a lower FICO because I have one (1) credit card and mostly pay cash for things. This makes me a ‘thin file’ which the models consider risky. So when we apply for mortgages, we mostly use my wife’s income alone. Sure, we double the PITI-to-income ratio, but the only input at which banks look is FICO, and we get better rates.
Sad.
July 24th, 2010 at 8:23 pm
FICO is the de facto 3-digit number tatooed to each person’s forehead.
The *evil* big 3 – try as they may- to impose their Vantage scores are not tickling mortgage lenders who refuse to have nothing to do with (disad)Vantage scores.
Valid indicator?
Probably not.
But it is what it is. It’s laughable to think job, income, assets, etc have (or should have) anything to do with credit worthiness. It’s pretty easy to BK debt away…even today.
Instead of ruminating about FICO’s accuracy, I’m more interested in Fair, Isaac’s recent marquee release last week. Surely, everyone one of you must have caught the headline:
25.5 percent of consumers — nearly 43.4 million people — now have a credit score of 599 or below!
Twenty-five percent….
Wow!
Now, that’s newsworthy. FICO as the “best” indicator is NOT newsworthy.
On top of 43.4 million are the 26 million reported as un- or underemployed, reports the Labor Department, as millions more face foreclosure – which alone can chop 150 points off a person’s score.
In and around Sarasota, Florida, I meet daily with homeowners facing foreclosure. No longer is credit on their minds. Guess what happens to an economic recovery with MILLIONS on the sidelines, rejected from financing purchases requiring credit?
Maybe The Big Picture readers feel this is a good thing to push people to the sidelines. Perhaps it’s a case of “you made your bed, now you sleep in it” sort of situation.
I glanced at comments with a few comments of my own:
@wcw In mortgage underwriting, there is no mixing-and-matching credit scores and income. In other words, if one spouse/partner does NOT have the minimum required MIDDLE credit score (not average), then the spouse/partner with the credit must have the income to carry the purchase.
Regarding your other point, isn’t it funny (in a sense) that cash carries NO weight today to creditors. Good luck getting a mortgage banker to finance you without minimum 3 active, seasoned tradelines.
Push that credit card to 60-100% utilization & watch your score plummet up to 50 points (or higher).
Fair?
Doesn’t matter. It is what it is.
@RW Every 3-5 hard pulls hit a score 12-27 points. For all you mortgage shoppers, keep in mind Fair, Isaac’s algo is supposed to overlook multiple hard pulls during 30 day period and 14 days prior, treating all as 1.
DUH…you’re shopping for the best rate/terms on a house or car or other big purchase.
However, it doesn’t work as advertised. I have MULTIPLE fact-based anecdotes showing buyers at the closing table & not able to get funded. Underwriting pulled credit before funding & discovered scores had DROPPED below minimum acceptable.
Sorry, no cigar – regardless of how loud buyer’s mortgage person screamed to loan rep for exceptions.
@bergsten What is Credit Union’s “permissible use” for pulling your credit? When you opened your account, did the app. give them permission to pull your credit?
Now, I realize Florida’s housing issues do not reflect economic conditions in the rest of the country. However, it seems as through more “distress” is hitting luxury market as well.
Cheers,
Mike
July 24th, 2010 at 10:03 pm
Just like many MNC’s use audit data, which they know isn’t accurate in many parts of the world, to measure performance because “We have to have a benchmark”. It’s Institutionalised stupidity.
July 25th, 2010 at 7:18 am
@MtSREg:
“However, it doesn’t work as advertised. I have MULTIPLE fact-based anecdotes showing buyers at the closing table & not able to get funded. Underwriting pulled credit before funding & discovered scores had DROPPED below minimum acceptable.”
“Sorry, no cigar – regardless of how loud buyer’s mortgage person screamed to loan rep for exceptions.”
Scores can change between two readings for a variety of reasons, not just ‘number of hard pulls’, but for other activities which frequently accompany mortgage applications, like fudging the LTV by arranging loans elsewhere to pay for some or all of the down payment, or reports from previous use of credit updating the credit history between the two readings. Some of the possible changes in the credit history that feeds the score algorithm are innocuous and unrelated to the current transaction (though the new information does affect the score value), others are indicators of possible misrepresentation of source of closing funds which trigger alerts beyond mere change of score (although the underwriter’s response may be to simply refuse exceptions based on the scores and existing score range guidelines rather than to explain all of this to the applicant).
As for the greater distress hitting the high end price scale in Florida, see previous remarks about the compression effect on the predictive power of credit scores in stressed markets in an earlier comment. The discussion in Appendix A of this reference is particularly illuminating:
http://www.federalreserve.gov/SECRS/2008/October/20081031/R-1318/R-1318_4_1.pdf
July 25th, 2010 at 7:29 am
A year or more ago I warned here that the housing crisis and unemployment would crush credit scores and make the recovery (by way of obtaining consumer credit) slower. Why do we suppose GM recently bought a subprime car lender? Ans: so they can control the situation.
Some points:
- Other seemingly irrational things influence FICO. If you get a $2,500 personal loan from a bank, no problem. Get the same loan from a finance company and your score will go down 50-75 points. There is no recognition of those paying their credit card balances off monthly. The credit balance is reported monthly (generally) by the credit grantor and whatever your balance is at that moment in time is what is reported. If the person pulling the credit report asks that payments be verified by the credit bureau then the credit bureau locks that payment (seemingly because it has been verified?)- even if it is associated with a revolving credit (such as a HELOC) in which you later reduce or increase the balance. Many bank automated underwriting systems take this garbage in as gospel.
- Aside from highlighting the potential of errors, most of the information in the article, such as current address and employer, are irrelevant to the FICO score. The source of that is not lenders reporting credit histories. It is those who are pulling credit reports and so they are ignored.
- The bank I recently retired from made a (partial) living doing used car loans to people in the B+ to C- range. We manually underwrote everything and factored in debt to income, time on the job, etc. Our delinquency and charge-off rates were well below peer. Yet, regulators gave us a wrist slap for making too many “sub prime” loans, which they arbitrarily decided were anything under a credit score of 660. It did not really deter us or change our business plan but it caused stepped up internal reporting.
- Same bank ran an in-house mortgage program for non-conforming mortages (those which we could not or made no sense to sell to Freddie for whatever reason.) One non-conforming issue was credit score. Again, manually underwritten using all available information. The Federal Home Loan Bank decided that because we had so many “sub-prime” loans in our mortgage portfolio (I think they used 640 as the cut-off), they reduced the amount of credit secured by that portfolio they would make available. Understandable, but presto = reduced liquidity for the bank. BTW delinquencies in that portfolio were below industry for “prime.”
- As inflexible as the GSEs have become, the PMI companies are even worse. FHA is still out there in sub-prime land. Incidentally, I am puzzled about the allegation that the GSEs rely exclusively on credit scores. They have their own in-house automated underwriting systems which has available to it debt-to-income, loan to value, time on the job, etc.
- On the point of using FICO scores to set (tier) loan interest rates, aside from the obvious income producing aspect of that to banks and the logic that those with better credit should pay lower rates than those with poorer credit since they constitute less risk, the compliance regulators (under Fair Lending) look at this closely to make certain that 1. you are using an objective measure and 2. that you are not applying it in a disparate fashion. Do otherwise and you are in a WORLD of trouble or, at least, extensive justification and statistical support for your method. So, the only universally recognized number to use for this purpose is FICO which is why practically 100% of the lenders use it for pricing purposes.
July 25th, 2010 at 10:25 am
not sure that banks take FICO as is, unless they are doing it that way because its cheaper. most take it and then change it to fit their needs.
July 25th, 2010 at 12:33 pm
FICO scores are “virtually meaningless” to quote an authority (Wikipedia). And the range between payment and default has decreased very significantly.
IMHO, the FICO algos are a meaningless pile of crap tuned to find reasons to increase insurance premiums for as many as possible. IMHO they can’t be demonstrated as meaningful other than as major signals requiring major scoring differences.
FICO scores are most definitely discriminatory. Surprised no one mentioned how insurance companies use them to justify premium cost dings. Those that can least afford to make a larger payment than the elites. So if they were a risk before, it just made matters worse.
And they are an insult to everyone struggling to make ends meet.
” We are very sorry, but according to the experts, because you are not deemed sufficiently creditworthy, we have to charge you more for insurance than Joe Elite. ” “State law requires us to disclose this.” “Have a nice day and we really appreciate your business.”
Nothing like a slap in the face. With little recourse. Prove us wrong using your time and energy. .
July 25th, 2010 at 2:12 pm
@ alfred e (neumann?):
“IMHO, the FICO algos are a meaningless pile of crap tuned to find reasons to increase insurance premiums for as many as possible.”
You are entitled to your opinion (like a nose and anus, almost everyone has one), but not entitled to have it taken seriously without credible evidence. Wikipedia is often a useful source of links to good references, and sometimes has good information (at least on non-contentious topics), but given its semi-anarchic editing process, which strongly privileges “true believers” with too much time on their hands and professional axe grinders who are paid to patrol certain topics and keep them reliably negative-content-free, wiki articles always have zero ‘authority’ as a reference.
The easily provoked, the perpetually resentful, and financially profligate (anyone whose emotional buttons are easily pushed by marketers, salesmen, or amateur provacateurs) tend to be poor credit, auto liability, and general liability risks. That’s a mere opinion which, together with a couple of dollars, can get you a cup of coffee but not a premium rate filing approval. However, a person’s general attitude towards risk , person responsibility, and social relations can be captured in part through credit histories and other risk factors like driving records, and default/insurance claim records. Assembled aggregates of such experience are the evidence state insurance regulators require insurers to present in support of their rate filings.
“Prove us wrong using your time and energy”.
Indeed; that or pay the quoted prices, or do without the products offered by these enterprises. No financial guarantor willing to co-sign your loan for a fee you consider reasonable? Find someone with your own high opinion of yourself (and a better credit record and demonstrated financial capacity than you) to co-sign instead. Otherwise, save for the desired purchase and make do without the credit. That’s often not easy, but it’s simple enough.
July 25th, 2010 at 2:57 pm
“” There is no recognition of those paying their credit card balances off monthly. The credit balance is reported monthly (generally) by the credit grantor and whatever your balance is at that moment in time is what is reported. If the person pulling the credit report asks that payments be verified by the credit bureau then the credit bureau locks that payment (seemingly because it has been verified?)- even if it is associated with a revolving credit (such as a HELOC) in which you later reduce or increase the balance. Many bank automated underwriting systems take this garbage in as gospel.”
Ah yes, the other thing I forgot to mention. I’m one of those uses my card for everything guys. I closed out a Walkallovaya card that had a 35k limit on it because they started screwing with the online statement system (probably in the change over to Hells Fargo) and caused me to miss a payment. I noticed on this most recent credit pull where my scores have dropped I have balances of ~1400 on a 1500 card and ~800 on a 2000 card, certainly making the debt/credit ratio look wonderful and no doubt helping my score immensely.
Anyone that uses their cards this way that have low limit cards, if you start doing the mortgage search, be sure to pay your card off in full the day before the statement period closes to make your ratios look better.
July 25th, 2010 at 10:44 pm
Youth in Asia:
“be sure to pay your card off in full the day before the statement period closes…” and keep it at a low balance until you get your mortgage loan. The lender reports the balance at the time when they drop the report from their mainframe to the credit reporting agency (such as Trans Union.) You might pay if off in full on the 28th (end of your statement period) but if the bank reports the following 15th, the balance then is what gets reported.
Good luck.