Demand for housing has been suppressed in the four-year period from 2007 to 2010. A review of household formation shows an annual increase of 500,000 to 600,000 over these years. A more normal gain would be 1 to 1.2 million each year. (As a quick clarification on households, one household corresponds to one housing unit. A single person living in an apartment is considered one household. A family of five people living under one roof is also considered one household.) Population growth has not slowed, rising consistently by around 3 million each year, but household formation has.

That is due to an increasing number of people deciding, or being forced by circumstance, to live with others. Rather than one roommate, many now have two. Some recent college graduates have returned home to live with their parents. Painful foreclosures have also forced people to find temporary arrangements with friends and relatives.

Living in tight spaces is not sustainable. More people cannot be comfortably shoved into existing households. Aside from the desire to be independent and to move away from temporary living situations, there is the issue of “familiarity breeding contempt,” as the saying goes. It is just a matter of time before household formation returns to its historic normal growth of 1 to 1.2 million each year. There could even be more-than-normal household formation for a few years from both normal population growth and from people leaving temporary arrangements. A stronger economy and job prospects will help in restoring normal household formation.

This suppressed housing demand is like a coiled spring. But when will it pop? This year? Next?

When it does pop there will be a rush of home buyers and renters into the market. Inventory will fall, home prices will rise. Rents will rise. Home builders will need to quickly ramp up production and hire construction workers. Home builders are expected to add only 770,000 new units this year, which is well below the one million new demand from household formation, but still up from the 500,000 range of the past three years.

By Shawn Tully, senior editor-at-large March 28, 2011: 5:00 AM ET

Forget stocks. Don’t bet on gold. After four years of plunging home prices, the most attractive asset class in America is housing.

A home under construction in Austin. The number of new homes in the pipeline nationwide is quite low.

From his wide-rimmed cowboy hat to his roper boots, Mike Castleman fits moviedom’s image of the lanky Texas rancher. On a recent March evening, Castleman is feeding cattle biscuits to his two pet longhorn steers, Big Buddy and Little Buddy, on his 460-acre Bar Ten Creek Ranch in Dripping Springs, a hamlet outside Austin in the Texas Hill Country. The spread is a medley of meandering streams, craggy cliffs, and centuries-old oaks. But even in this pastoral setting, his mind keeps returning to a subject he knows as well as any expert around: the housing market. “I’m a dirt-road economist who sees what’s happening on the ground, and in 35 years I’ve never seen a shortage of new construction like the one I’m seeing today,” declares Castleman, 70, now offering a biscuit to his miniature donkey Thumper. “The talking heads who are down on real estate will hate to hear this, but America needs to build a lot more houses. And in most markets the price of new homes is fixin’ to rise, not fall.”

Castleman is in a unique position to know. As the founder and CEO of a company called Metrostudy, he’s spent more than three decades tracking real-time data on the country’s inventory of new homes. Each quarter he dispatches 500 inspectors to literally drive through 45,000 subdivisions from Baltimore to Sacramento. The inspectors examine 5 million finished lots, one at a time, and record whether they contain a house that’s under construction, one that’s finished and for sale, or a home that’s sold. Metrostudy covers 19 states, or around 65% of the U.S. housing market, including all the ones hardest hit by the crash: Florida, California, Arizona, and Nevada. The company’s client list includes virtually every major homebuilder and bank — from Pulte (PHM) and KB Home (KBH) to Bank of America (BAC) and Wells Fargo (WFC).

The key figures that Metrostudy collects, and that those clients prize, are the number of homes that are vacant and for sale in each city, and the number of months it takes to sell all of them. Together those figures measure inventory — the key metric in determining whether a market has a surplus or a shortage of new housing.

Today Castleman is witnessing an extraordinary reversal of the new-home glut that helped sink prices just a few years ago. In the 41 cities Metrostudy covers, a total of 78,000 houses are now either vacant and for sale, or under construction. That’s less than one-fourth of the 343,000 units in those two categories at the peak of the frenzy in mid-2006, and well below the level of a decade ago. “If we had anything like normal levels of buying, those houses would sell in 2½ months,” says Castleman. “We’d see an incredible shortage. And that’s where we’re heading.”

If all the noise you’re hearing about housing has you totally confused, join the crowd. One day you’ll read that owning a home has never been more affordable. The next day you’ll see news that housing starts have plunged to nearly their lowest level in half a century, as headlines announced in March. After four years of falling prices and surging foreclosures, it’s hard to know what to think. Even Robert Shiller and Karl Case can’t agree. The two economists, who together created the widely followed S&P/Case-Shiller Home Price indices, are right now offering sharply contrasting views of housing’s future. Shiller recently warned that the chances were high for a further double-digit drop in U.S. home prices. But in an interview with Fortune, Case took a far brighter view: “The lack of new home building is a huge help that a lot of people are ignoring,” says Case. “People think I’m crazy to be optimistic, but housing is looking like the little engine that could.”

To see where real estate is truly headed, it’s critical to keep your eye firmly on the fundamentals that, over time, always determine the course of prices and construction. During the last decade’s historic run-up in prices, Fortune repeatedly warned that things were moving too fast. In a cover story titled “Is the Housing Boom Over?,” this writer’s analysis found that the basic forces that govern the market — the cost of owning vs. renting and the level of new construction — were in bubble territory. Eventually reality set in, and prices plummeted. Our current view focuses on those same fundamentals — only now they’re pointing in the opposite direction.

So let’s state it simply and forcibly: Housing is back.

Two basic factors are laying the foundation for dramatic recovery in residential real estate. The first is the historic drop in new construction that so amazes Castleman. The second is a steep decline in prices, on the order of 30% nationwide since 2006, and as much as 55% in the hardest-hit markets. The story of this downturn has been an astonishing flight from the traditional American approach of buying new houses to an embrace of renting. But the new affordability will gradually lure Americans back to buying homes. And the return of the homeowner will start raising prices in many markets this year.

ln a long-awaited shift, Wells Fargo is providing FHA mortgages to borrowers with credit
scores as low as 500. The move comes after the National Association of Realtors and
FHA Commissioner David Stevens, among others late last year, criticized the country’s
major banks for requiring credit scores as high as 650 in some cases before making
loans.

At NAR’s annual conference last year in New 0rleans, Stevens said banks’ credit
policies were out of sync with the FHA and artificially restraining home sales by as much
as 20 percent. Under its new policy, Wells Fargo will accept borrowers with credit scores
of 500 to 579 if those borrowers can make a down payment of at least 10 percent; gifted
funds or other down paymeni assistance is not allowed.

For borrowers with credit scores of 580 to 599, borrowers must put down 5 percent, with
the same restriction on gifts and assistance funds. Borrowers with credit scores of 600 or
higher can make a 3.5 percent down payment. The new policy took effect January 15, 2011.

Mortgage rates are back in the 4 percent range, taken there by the 10-year Treasury note’s drop to 3.42 percent, which has wiped out the whole early-February spike. We have Libya, oil, stocks, housing and the economy to thank, but how those pieces interact is not obvious.

Arriving economic data is obscured by incessant “recovery” cheerleading among business media. Some news is pretty good: both consumer-confidence surveys found three-year highs, which may indicate some hiring.

New claims for unemployment insurance are holding down near 400,000. Every measure of manufacturing is doing well, although scratching in at 15 percent of our economy.

The “wait-a-minutes” are led by today’s downward revision in supposedly corner-turning fourth-quarter 2010 U.S. gross domestic product (from 3.2 percent to 2.8 percent).

Growth? Sure. Self-sustaining, accelerating … not so much.

Wal-Mart’s sales in the last 90-days fell 1.8 percent. Orders for durable goods in January (excluding super-volatile transportation and defense) tanked 6.9 percent. The Chicago Federal Reserve’s January national index rolled to a minus sign.

If you’re in doubt, consider housing. New-home sales dumped 12.6 percent in January. But, the National Association of REALTORS® says sales of existing homes rose 2.7 percent.

A story cooking for months: NAR may have overreported those sales for 10 years or more, high by perhaps a half-million each year vs. CoreLogic (its parent is a title company using real data), and since 2008 resulting in as many as 1.5 million imaginary sales each year.

If existing-home sales are running below 4 million annually instead of above 5 million, then we have rather more trouble with distressed-inventory absorption than we thought. And market buyers and sellers have left the field or been elbowed from it.

On that subject, NAR said only 37 percent of January sales were distressed. CampbellSurveys.com, meanwhile, says the distressed fraction was 44 percent in November, and shot up to 49.6 percent in January (California at 66 percent, Florida at 63 percent, Arizona and Nevada at a whopping 72 percent).

Now, two things not to worry about: inflation and the Middle East.

Inflation comes in a few well-defined forms. The deadly one is the wage-price spiral, which plagued us in the 1970s and can be stopped only by recession.

The U.S. today is impervious to such a spiral because we have near-zero wage growth. In fact, rising oil prices will slow this economy by crowding out other spending. Note that commodity prices have fallen — not following oil, instead anticipating slowdown. Same for stocks.

Other inflation forms include the classic money-printing of Zimbabwe. The Fed’s “quantitative easing” (QE) program does print monetary electrons, but the money cannot make it into wallets because the credit system is still broken. No credit, no money.

The last form, cost-pushed inflation, is temporary — it is not a structural ramp-up in general prices. That’s what this is. Oil is driven by speculators, just as it was in 2008, and all should be reassured that this spike has stopped far short of that run to $150 per barrel.

Middle East … it takes some serious chutzpah to be relaxed about the place. But, concept No. 1: the Middle East nations need to sell oil worse than we need to buy it, no matter who is in charge over there.

Non-oil economies in the Middle East have never developed, and 10-fold growth in population has made ’70s-style embargoes impossible today.

No. 2: Despite U.S. politicans talking about radical Islam, these brave Tunisians, Egyptians, and Libyans are in their streets waving their national flags, not pictures of Osama.

No. 3: Post-revolutionary peoples’ governments, no matter how they may roil and rock, are steadier by far than rotten autocracies, especially in dealings with neighbors. Some go sour (Iran), some are in doubt (Iraq), but dictatorships breed anger and extremism, and dictators need to pick external fights with pretend enemies.

Refounded nations, even while struggling to establish representative government, have stabilizing advantages. The greatest of these: pride at self-liberation won hard.

Here’s some unexpected good news for anybody looking to buy a home in this market, especially first-time home buyers who don’t have much down payment cash on hand: The door to an FHA-insured mortgage just opened a little wider.

With no fanfare or public announcements, two of the largest FHA-approved lenders have backed off their controversial “overlay” requirements on FICO scores (lender overlays are qualification requirements that can be more stringent than FHA’s own requirements). Both Wells Fargo and Quicken Loans confirmed last week that they will now lend to applicants with 580 FICOs and 3.5 percent down payments.

Their revised standards conform in most respects to FHA’s own minimums, and open the agency’s financing to large numbers of buyers whose credit scores have sagged during the recession. Wells Fargo is the largest originator of FHA-insured mortgages; Quicken ranks third, according to industry data.

Along with most other major lenders, both companies previously had insisted on minimum FICOs of 620 for otherwise qualified borrowers seeking 3.5 percent down payment loans. If your score came in even slightly lower, they wouldn’t even look at your application. An estimated one third of Americans now have FICO scores below 620, according to one consumer group’s estimate.