Skip to comments.Misconceptions about the CPI (FR Thread to Prove or Disprove CPI and Shadow Stats)(17page PDF file)
Posted on 04/15/2011 9:08:02 AM PDT by Uncle Miltie
Addressing misconceptions about the Consumer Price Index
A number of longstanding myths regarding the Consumer Price Index and its methods of construction continue to circulate; this article attempts to address some of the misconceptions, with an eye toward increasing public understanding of this key economic indicator.
The Consumer Price Index (CPI), published by the Bureau of Labor Statistics (BLS), has generated controversy throughout its history. A soon-to-be-published article by Marshall Reinsdorf and Jack Triplett discusses the many past reviews of the methods and data used in the CPIs construction.1 Beginning with an advisory committee appointed by the American Statistical Association in 1933,2 and continuing through the recent National Research Council panel chaired by Charles Schultze,3 panels and commissions have identified and discussed what is now a well-known set of issues affecting the measurement of consumer prices: consumer substitution behavior, change in the quality of products, the introduction of new types of goods and services, and the appearance of new categories of stores and new channels of product distribution. Given the large number of private and public uses of the CPI, and especially its important role in determining Federal Government revenues and payments, it is natural that each of those issues has been the subject of intense public attention.
Within the past several years....... (Etc.)
(Excerpt) Read more at bls.gov ...
I think we know the major proponents and opponents of the CPI here on FR. What I want to do here is to accumulate the Third Party, academic, philosophical, and economic records of arguments in favor or opposing the CPI as currently constructed, and especially whether Shadow Statistics is a more relevant data set.
Let's leave the emotions and personal attacks in the locker room, guys. We know how you point / counter-point each time this comes up. Let's have third parties weigh in.
It is critical to my personal and corporate decision making to determine the validity of the CPI and Shadow Stats. I don't have a dog in this fight, other than to truly understand the relative validity of each approach.
So, let's treat this as our Phd Econ class, and bring facts and arguments. Let's take the conversation up a level, and let's have this string stand as a representative of each camp's best data, proofs and arguments.
I'll moderate, if you don't mind, asking questions of both sides if I personally need clarification. Like I said, I don't have a dog in this hunt.
BusinessWeak opened a thread for discussion of this topic here. Feel free to bring this to the thread if it supports your position.
I don’t need the freakin’ government to tell me what inflation is doing. I see it every time I go to the gas pimp or grocery store.
Bring your friends, family, and arguments over here, dudes. Keep it FRiendly!
The BLS has attempted to defend against all the Shadow Stats arguments in the linked .pdf. Be sure in your posts that you can overcome their defenses if you wish to defend Shadow Stats.
Dude: We’ve seen that post a dozen times. Plus which, the BLS report to which the thread is linked attempts (and does a fair job) of debunking each of your points.
Please go read the BLS arguments, and bring something new.
So basically they’ve cooked the books. They’ve built their statistics on a foundation of shifting sand which makes them meaningless except in the context to provide political cover in case we need to print more money.
Hey look! We printed more money to pay for excessive government, but your money has the same value!- If you don’t mind eating hamburger instead of steak.
Oh, and ignore the rising food and gas prices, it doesn’t matter.
“September 04, 2008
I’ve yet to find someone who has been able to reproduce the claims made by Shadow Government Statistics about the extent to which government agencies are grossly misreporting the U.S. inflation rate. Apparently, neither has the Bureau of Labor Statistics, as detailed in an article by BLS economists John Greenlees and Robert McClelland in the latest issue of Monthly Labor Review.
First, some of the bolder claims by Shadowstats:
The Boskin/Greenspan argument was that when steak got too expensive, the consumer would substitute hamburger for the steak, and that the inflation measure should reflect the costs tied to buying hamburger versus steak, instead of steak versus steak. Of course, replacing hamburger for steak in the calculations would reduce the inflation rate, but it represented the rate of inflation in terms of maintaining a declining standard of living. Cost of living was being replaced by the cost of survival. The old system told you how much you had to increase your income in order to keep buying steak. The new system promised you hamburger, and then dog food, perhaps, after that....
The BLS initially did not institute a new CPI measurement using a variable-basket of goods that allowed substitution of hamburger for steak, but rather tried to approximate the effect by changing the weighting of goods in the CPI fixed basket. Over a period of several years, straight arithmetic weighting of the CPI components was shifted to a geometric weighting. The Boskin/Greenspan benefit of a geometric weighting was that it automatically gave a lower weighting to CPI components that were rising in price, and a higher weighting to those items dropping in price.
Once the system had been shifted fully to geometric weighting, the net effect was to reduce reported CPI on an annual, or year-over-year basis, by 2.7% from what it would have been based on the traditional weighting methodology. The results have been dramatic. The compounding effect since the early-1990s has reduced annual cost of living adjustments in social security by more than a third.
And here’s the response by Greenlees and McClelland:
To begin, it must be stated unequivocally that the BLS does not assume that consumers substitute hamburger for steak. Neither the CPI-U, nor the CPI-W used for wage and benefit indexation, allows for substitution between steak and hamburger, which are in different CPI item categories. Instead, the BLS uses a formula that implicitly assumes a degree of substitution among the close substitutes within an item-area component of the index. As an example, consumers are assumed to respond to price variations among the different items found within the category “apples in Chicago.” Other examples are “ground beef in Chicago,” “beefsteaks in Chicago,” and “eggs in Boston”....
The quantitative impact of the CPI’s use of the geometric mean formula also has been grossly overstated by some, with one estimate exceeding 3 percent per year. It is difficult to identify real-world circumstances under which geometric mean and Laspeyres indexes could differ by such a large amount. The two index formulas will give the same answer whenever the prices used in an index all change by the same percentage. The bigger the differences in price changes, the more the Laspeyres index will tend to exceed the geometric mean. For the growth rate of the Laspeyres index to exceed the growth rate of a geometric mean index by 3 percentage points, however, the differences in individual price changes have to be quite large.
To see this point, consider another very simplified example. Suppose that the CPI sample for ice cream and related products in Boston consisted only of an equal number of prices for ice cream and frozen yogurt and that, between one year and the next, all the prices of ice cream in Boston rose by 8.6 percent while all the frozen yogurt prices fell by 4.2 percent. In that case, the geometric mean estimate of overall annual price change would be 2.0 percent, only slightly less than the Laspeyres estimate of about 2.2%. In order to come up with a difference of 3 index points, one has to assume a much more dramatic divergence between ice cream and frozen yogurt prices than the one hypothesized. For example, if ice cream prices rose 30 percent in one year, while frozen yogurt prices fell by 20 percent, the overall geometric mean index would still rise by 2 percent, but the Laspeyres index would rise 5 percent, for a difference of 3 index points. However, such a large annual divergence would be quite uncommon within CPI basic indexes— between ice cream and yogurt, between types of candy and gum, between types of noncarbonated juices, or between varieties of ground beef. Moreover, for a 3-percentage-point divergence to continue year after year, the divergence between the individual component prices would have to continue to widen. For example, if, by contrast, during the next year ice cream prices increased by the same amount as frozen yogurt prices, then the two index formulas would give the same inflation estimate for that year. Although such a divergence might plausibly occur in one component for 1 year, it is beyond belief that such sharply divergent price behavior would continue year after year across the whole range of CPI item-area components.
Finally, and most importantly, there is rigorous empirical evidence on the actual quantitative impact of the geometric mean formula, because the BLS has continued to calculate Laspeyres indexes for all CPI basic indexes on an experimental basis for comparison with the official index. These experimental indexes show that the geometric mean led to an overall decrease in CPI growth of about 0.28 percentage point per year over the period from December 1999 to December 2004, close to the original BLS prediction that the impact would be approximately 0.20 percentage point per year.
There’s much more in the BLS article on this and related questions such as hedonic price adjustment and owner’s equivalent rent.
Why do people continue to give credibility to an operation like Shadowstats? Now that’s something that I’d like to hear explained.”
|Chronology of changes in the Consumer Price Index
Began publication of separate indexes for 32 cities (1919)
Collected prices in central cities periodically.
Developed weights from a study that BLS conducted in 1917-19 of family expenditures in 92 industrial centers
Reflected the relative importance of goods and services purchased by consumers.
Collected prices for major groups: Food, clothing, rent, fuels, house furnishings, and miscellaneous
Limited pricing to items selected in advance to represent their categories
Began regular publication of a national index, the U.S. city average (1921):
Based index on an unweighted average of the city indexes.
Estimated U.S. city average back to 1913, using food prices only.
The 1940 CPI revision: the first comprehensive revision
Used weights based on 1934-36 study of consumer expenditures
Collected prices in the 34 largest cities
Implemented a weighted average of cities for the U.S. city average CPI
Improvements made between the 1940 and 1953 revisions
During World War II:
Discontinued the pricing of unavailable items, such as new cars and household appliances
Increased the weight of other items, including automobile repair and public transportation
Adjusted weights in seven cities using 1947 and 1949 survey of consumer expenditures
Adjusted weights for the 1950 census
Adjusted rent index to remove new unit bias caused by rent control
Added new items to the list of covered items, including frozen foods and televisions
The 1953 CPI revision: the second comprehensive revision
Used weights from a 1950 expenditure survey conducted in central cities and attached urbanized areas
Refined the target population to include urban wage earner and clerical worker families
Added a sample of medium and small cities
Updated the list of items that the index covered, adding restaurant meals
Added new sources of price data
Improved pricing and calculation methods
The 1964 CPI revision: the third comprehensive revision
Based weights on 1960-61 expenditure patterns in metropolitan areas
Added single-person households to target population: urban wage earner and clerical worker households
Extended pricing to the suburbs of sampled metropolitan areas
Updated the sample of cities, goods and services, and retail stores and service establishments
Improvements made between the 1964 and 1978 revisions
Made quality adjustments for new vehicles at model changeover
Improved treatment of seasonal items
The 1978 CPI revision: the fourth comprehensive revision
Added a new Consumer Price Index: the CPI for All Urban Consumers, or the CPI-U
Renamed the older CPI as the CPI for Urban Wage Earners and Clerical Workers, or the CPI-W
Used weights from a 1972-73 survey of consumer expenditures and the 1970 census
Expanded the sample to 85 areas
Increased minimum pricing frequency from quarterly to bimonthly
Implemented monthly pricing in the five largest areas
Introduced probability sampling methods at all stages of CPI sampling:
Used probability selection methods to select the CPI sample items within stores
Eliminated the list of eligible items as virtually all consumer items became eligible for pricing
Introduced checklists that define each category of spending
Developed estimates of the CPIs sampling error and optimal sample allocation to minimize that error
Improvements made between the 1978 and 1987 revisions
Began outlet and item sample rotation (1981):
Began systematic replacement of outlets and their item samples between major revisions
Implemented new Point-of-Purchase Survey (POPS)
Selected retail outlets with probability proportional to consumer spending therein
Eliminated reliance on outdated secondary-source sampling frames
Began rotating outlet and item samples every 5 years
Began rotating one-fifth of the CPI pricing areas each year
Introduced rental equivalence concept (January 1983 for the CPI-U; January 1985 for the CPI-W):
Introduced the flow-of-services method, which removes the investment component from homeowner indexes
Discontinued the asset-price approach, which treated the purchase of a home as a consumer good
The 1987 CPI revision: the fifth comprehensive revision
Used weights from the 1982-84 Consumer Expenditure Survey and the 1980 census
Updated samples of items, outlets, and areas
Redesigned the CPI housing survey
Improved sampling, data collection, data processing, and statistical estimation methods
Initiated more efficient sample design and sample allocation
Introduced techniques to make CPI production and calculation more efficient
Improvements made between the 1987 and 1998 revisions
Improved the housing estimator to account for the aging of the sample housing units
Improved the handling of new models of vehicles and other goods
Implemented new sample procedures to prevent overweighting items whose prices are likely to rise
Improved seasonal adjustment methods
Initiated a single hospital services item stratum with a treatment-oriented item definition:
Discontinued pricing of the inputs to the index for hospital services
The 1998 CPI revision: the sixth comprehensive revision
Weights from the 1993-95 Consumer Expenditure Survey and the 1990 census
Updated geographic and housing samples
Extensively revised item classification system
Implemented new housing index estimation system
Used computer-assisted data collection
Added the Telephone Point-of-Purchase Survey (TPOPS):
Allows rotation of outlet and item samples by item category and geographic area, rather than by area alone
Initiated a new housing survey based on the 1990 census (January 1999):
Estimated price change for owners equivalent rent directly from rents
Began using a geometric mean formula for most basic indexes (January 1999):
Mitigates lower-level substitution bias
Reflects shifts in consumer spending within item categories as relative prices change
Published the CPI-U Research Series
Featured backcastings of all CPI method changes to 1978
Provided revision in cases of methodology change
Improvements since the 1998 revision
Extended the use of hedonic regression to estimate the value of items changing in quality
Directed replacement of sample items in the personal computer and other categories, to keep samples current
Implemented 4-year outlet rotation to replace the 5-year scheme
Began within-outlet item rotation for prescription drugs and other item categories
Implemented biennial weight updates (January 2002):
Separated weight updates from major revisions to keep weights as current as possible
(Weights used in 2002-2003 were based on the 1999-2000 Consumer Expenditure Survey; weights used in
2004-05 were based on the 2001-02 Consumer Expenditure Survey; weights used in 2006-07 were based on
the 2003-04 Consumer Expenditure Survey.)
Increased sample size of the Consumer Expenditure Survey, so that CPI weights can be based on just 2 years of data from expenditure surveys 2 and 3 years previous
Added the Chained Consumer Price Index for All Urban Consumers (C-CPI-U) (August 2002):
Uses more advanced superlative index formula (the Tornqvist formula)
Corrects upper-level substitution bias
Began computer-assisted data collection for the Commodities and Services Survey (20022003)
Expanded collection of price data to all business days of the month
Began publishing indexes to three decimal places (Jan. 2007)
You can perform a ineternet search on “Shadow Stats”+”Hedonic” and get some get hits.
What Williams presents on his website and what he espouses in the media are not the same, he ALWAYS advocates his political ideology in the media.
One of the first truly comprehensive critiques of Williams CPI method is here:
September 04, 2008
Williams presented his repsonse to his critics (and specifically the econobrowser critique) a week later in regards to the topic of your post here:
Response to BLS Article on CPI Misconceptions
JOHN WILLIAMS SHADOW GOVERNMENT STATISTICS
September 10, 2008
ShadowStats.com Response to BLS Article on CPI Misconceptions
Reading about the Boskin Commission can make your brain fry and kill your soul, so be warned.
Here is one link regarding the little talked about CPI-RS, one of the effects of the Boskin Commission that turns Williams face red when asked about it:
Men on the Boskin Commission had some doubts about the CPI a decade after the Commission’s final report and implementation,
former member Robert Gordon of Northwestern
The Boskin Commission Report:
A Retrospective One Decade Later*
Robert J. Gordon
"Now you can believe there was a housing bubble, or you can believe that Shadow Stats is trustworthy, but if you believe both you're delusional. Personally, I believe there was a housing bubble and Shadow Stats is full of B.S."
Not intended as a thread hijack but do you know why Toddsterpatriot got banned? I was trying to pin him here for input.
Staying Neutral on Inflation vs. Deflaton
By James Kostohryz Sep 16, 2009 4:30 pm
Use the core CPI as your guide.
Theres been a heated debate in the financial press regarding the prospects for hyperinflation or deflation, with most pundits usually siding strongly with one camp or the other. Nowhere has this debate been more intense than within Minyanville.
I want to go on record saying that I definitely side with neither.
Ive been on record several times debunking the notion of hyperinflation or a dollar collapse anytime in the near future. However, Ive also been on record many times rejecting the doomsday scenarios of the deflationists.
Framing the Debate
Part of the problem with this debate rests in defining what these various terms mean. For example, I cant recall a single prophet of dollar doom or hyperinflation maven make a specific prediction citing a number or numerical range.
Thus, if the CPI were to rise above 10% per annum, I have little doubt that the mavens of hyperinflation will be declaring victory. And if the US dollar index were to decline another 10% or so, I have little doubt that the prophets of dollar doom with be loudly celebrating.
Similarly, the doomsday deflationists are equally vague in their forecasts. As long as the headline CPI stays below 0%, the doomsday deflationists will likely continue to declare victory.
This will not do. Forecasters need to define what theyre forecasting and be specific about their predictions. For example, the term hyperinflation doesnt have a specific numeric definition. However, various textbooks and other authoritative sources state that the minimum inflation rate that would qualify as hyperinflation would be over 100% per annum. Indeed, most sources that attempt to define hyperinflation cite monthly rates of 20% at minimum.
Similarly, to speak of a 10% or even 20% decline in the value of the US dollar — relative to a particular currency or a basket of currencies — as a collapse is nonsense.
Because the vast majority of the US economy is made up services and other non-tradeables, such a decline in the value of the US dollar would hardly be felt at all at the consumer price level.
Furthermore, such a decline would actually be quite positive for the US economy, as it would contribute to the lowering of the current account deficit — a problem that is at the heart of many important economic problems that the US faces, including excess indebtedness, sluggish job growth, and income inequality.
So a 10% to 20% decline in the foreign exchange value of the US dollar is hardly something that should generate the sort of panic that prophets of dollar doom have been attempting to incite — rather it would be something to be welcomed.
What about the doomsday deflationists? Theyre similarly vague in their predictions.
Admittedly, Ive heard few speak of hyper-deflation. So presumably, any inflation rate below 0% would satisfy their criteria.
However, this will also not do. Predictions of deflation need to be specific. For example, many in the deflation camp seem to view todays CPI figure of -1.5% as confirmation of their views regarding deflation.
However, what should we make of the fact that the core CPI — which encompasses the vast majority of goods and services — is still rising at a +1.4% per annum pace?
The Ideological Mindset
The truth of the matter is that many of the most ardent participants in the debate on inflation versus deflation tend to exhibit an ideological mindset.
There are several traits that tend to characterize such a mindset.
1. They tend to see things in terms of extremes — as black or white; all or nothing.
2. They tend to disregard empirical data, or to explain away the data thats not favorable to their point of view.
Let me cite some examples: A few years ago, the ideologues of hyperinflation were ardently arguing that despite the fact that the CPI was not rising, there really was inflation. One statistic they liked to cite was the growth in the money supply.
The funny thing is that ever since the growth in the various money-supply aggregates started to decelerate — and even contract in some cases — the ideologues of inflation stopped citing this metric.
Similarly, the ideologues of inflation constantly used to cite asset price inflation in property values and stock prices as evidence that there really was inflation, despite the fact that there was little or none reflected in the CPI.
However, strangely enough, in the last couple of years since property values and the stock market have been falling, the ideologues of hyperinflation have apparently suffered a case of amnesia and no longer seem to think those metrics are of any importance.
Indeed, when all else fails and the ideologues of hyperinflation are unable to find any empirical support for their claims, they tend to resort to citing highly questionable alternative statistics such as those concocted by Shadowstats.
Despite clear evidence of flat to falling consumer prices and collapsing asset prices, the ideologues of hyperinflation adamantly insist that there really is significant inflation, but the evil government is hiding it from us.
How do you have a constructive argument with people that either refuse to argue on the basis of empirical statistical data or that simply make up their own data?
Similarly, the advocates of deflation have had a bonanza in the past year citing falling stock prices and property values as evidence of massive deflation. Yet, I havent been hearing any of these very pundits talking about inflation rather than deflation now that stock prices are exploding and property values are rebounding.
And again, I dont see many of the ideologies of deflation acknowledging that the core CPI has never once reflected deflation during the recent crisis.
My Own Position
So, let me make my own position clear.
I believe that the core CPI is, in general, the best (although not the only) metric of inflation to monitor given the fact that food and energy prices tend to be volatile. Furthermore, food and energy prices are often driven more by localized supply-side resource constraints than the sort of generalized demand-pull dynamics characteristic of a truly inflationary environment.
Its my expectation that price levels, as defined by the core CPI, will remain within a range of 1.00% to 5.00% for the foreseeable future. Furthermore, its my expectation that disinflationary momentum is probably bottoming out, and that in the short to medium term, core inflation will tend to revert toward its recent norm between 2.0% and 3.5%.
Exactly how much core inflation rebounds from its current level of +1.4% depends entirely on the strength of the rebound in global growth. Liquidity is relatively abundant; therefore a stronger-than-expected global growth spike that puts pressure on certain short-term-supply-constrained goods and services such as natural resource commodities is certainly possible.
However, even in such a scenario, massive overcapacity in most industries, coupled with widespread unemployment, will tend to put an upper lid on any generalized inflation that would propel the core CPI above 5%.
Hyperinflation, although a theoretical possibility that cant be ruled out in the long term, is an extremely remote possibility in the short to medium term (defined as the next two years).
In terms that might be familiar to fans of the Austrian school, under current circumstances, excess liquidity combined with a surge in aggregate GDP growth (characterized by growth differentials among sectors of the economy) may cause relative price distortions and various concomitant ills. However, its unlikely to cause aggregate level hyperinflation.
Disinflation is the term I use most often to describe the economic environment with respect to aggregate prices during the past three decades, including the past year. And further disinflation is certainly a possibility in an environment of weaker-than-expected global growth.
However, massive fiscal and monetary stimulus in the US and elsewhere around the world will also put a floor on the extent of any deflation.
Governments and central banks in the US and around the world have considerable ability through a whole arsenal of measures to prevent deflation and its evident from the policy responses of the past year that officials around the world are determined to employ whatever monetary and/or fiscal mechanisms necessary to prevent a severe deflationary scenario analogous to the Great Depression.
Thus, for this reason, and the stickiness of modest inflationary expectations that have been ingrained in the psychology of the US population, core CPI inflation is unlikely to dip far below 0% in the foreseeable future.
It’s my expectation that disinflationary and deflationary pressures are subsiding and are quite likely to have bottomed out for the foreseeable future.
In particular, positive global growth surprises and concomitantly recovering asset prices are likely to create pockets of modest price increases in some sectors of the economy, which should be reflected in a reversion of aggregate core CPI inflation to recent historical norms between 2.5 and 3.5%.
However, because of massive overcapacity, unemployment, and the stickiness of inflationary expectations, a spike in core inflation above a modest 5.00% is unlikely, even if for no other reason than the fact that bond vigilante-ism would cause a spike in interest rates that would kill the incipient economic recovery, and with it, any major momentum in demand-side inflationary pressures.
In the end, the extremely heated debate between the mavens of hyperinflation and the doomsday deflationists is probably nothing more than a great deal of sound and fury signifying nothing.
Prce the housing bubble in physical gold and you come to the same conclusion, “”Now you can believe there was a housing bubble, or you can believe that market pricing of gold denominated in US Dollars is trustworthy, but if you believe both you’re delusional.”
I can’t find the US property priced in Gold blog post, so here’s a UK property priced in gold blog post and a Aussie property priced in gold blog post:
Property Priced in Gold
Posted on 13 April 2011 by BenM
When priced in gold property prices are down 69% from their peak in 2007
Property prices may only be down 14% when priced in British Pounds, however when priced in real money i.e. gold, we see that property prices are actually down 69% from their peak in the summer of 2007.
Good graphs and arguments here:
M3 is back
We did some sleuthing and data extraction and put M3 back together from various weekly Federal Reserve reports that are still available.
1.The formula we’re using has five 9s correlation to the original data back to 1980.
2.There is only one missing element that is apparently no longer available (Eurodollars) and an adjustment has been applied to generate it. Its only about 3% of total M3 so should not have a material effect on the total.
Here is our article on M3b, which details our work and notes the sources for the data. Note that as of Nov. 10, 2006 the Eurodollar estimation formula has changed - see the article for details.
John Williams monthly reconstruction of M3 is here. Ours tends to be more volatile than his, partly because it’s weekly and partly because of our differences in calculating the repo and Eurodollar component of M3.
Finally and to put M3 into proper perspective with inflation (as measured by CPI without lies), the M3 and M2 strong inflation link is virtually unquestionable. The longer term inflation picture is clear, although M2 shows a pause and likely temporary disinflation as of 2008. Certain bloggers are incorrect and have continually avoided these facts and the linked chart.
Disclaimer: Opinions posted on Free Republic are those of the individual posters and do not necessarily represent the opinion of Free Republic or its management. All materials posted herein are protected by copyright law and the exemption for fair use of copyrighted works.