A Long-Overdue Reckoning For The Bureau Of Labor Statistics

Forbes

18 hours ago

Executive Summary
The 'jobs number' – officially, 'nonfarm payroll employment'– is released monthly by the Bureau of Labor Statistics (BLS). It is said to have a greater impact on financial markets than any other economic indicator, and it is a critical input to the formation of monetary policy at the Federal Reserve.
It is also quite flawed. The initial estimate is released in a hurry each month - earlier than any other major metric – and it is always incomplete. The collection period is very short and typically one third of the desired inputs are missing from the first report. The BLS relies on survey procedures that are outdated, and response rates are dropping rapidly. In the most important poll, nearly 60% failed to respond. The initial estimate is revised at least four times over the following two years as further information trickles in, but the revisions are highly volatile and fail standard statistical tests for precision.
These facts have been quite well-known among insiders. The BLS' methodology is in need of an overhaul. As recently as July, a bipartisan group of 88 prominent economists and policy leaders addressed an open letter to Congress that called for major changes –highlighting the need to adopt more modern, technologically sophisticated approaches to data collection and analysis.
This month the problem burst into our mainstream consciousness. The Bureau issued an unexpected 90% downward revision in its most recent jobs estimates, erasing hundreds of thousands of jobs from its own previous count, which shook the financial markets and significantly altered the calculus on prospective monetary policy moves by the Federal Reserve.
In response, the President fired the head of the BLS and ignited a political furor.
The immediate political controversy is less important than the question of how to address the institutional and methodological shortcomings of the BLS. The agency's product has been considered as the 'gold standard' for economic data, allegedly the best in the world, trusted by investors, and closely attended by the Federal Reserve. But its deficiencies are increasingly evident, which puts that trust at risk. The real problem on the table now is what should be done to shore up this institution, improve its deteriorating performance, and restore its reputation.Jobs Shock
On August 1, the Bureau of Labor Statistics (BLS) issued a large downward revision of its estimate for the number of new jobs created in May and June. The original estimate for May was cut by 87%. The June estimate was cut by 90%.
These revisions erased 258,000 jobs from the count. It was the largest two-month reduction since 1979 (except for the anomalous pandemic shock of March/April 2020).
So far in 2025, the initial estimate has been revised downward for every month, wiping out a total of 461,000 job gains originally reported.
You're Fired
In response, President Trump fired Erika McEntarfer, a career bureaucrat who headed the BLS, which created a storm of controversy. The story has continued to occupy a prominent position in the news cycle for the past two weeks. Pro and con positions are starkly drawn: alleged manipulation, bias and incompetence at BLS on the one hand vs charges of political interference and attempted intimidation by the Administration on the other.
The BLS claims for itself 'a well-earned reputation for producing gold standard data' – which these developments could seem to put at risk. Many economists, politicians and pundits were outraged, fearing that the mission and reputation of the BLS will now be compromised, with dire consequences for American financial markets — and even for democracy itself.
The markets reacted with predictable signs of anxiety. Stock indexes fell, volatility jumped, and investors sought safety in bonds, pushing yields down. Much hand-holding was required.
The calculus on potential Fed policy moves was reset.Why The Jobs Number Matters
First of all, it moves markets.
Second, it probably has the biggest impact on monetary policy — especially as inflation has moderated and the focus of the Federal Reserve's attention has shifted to the labor market. The abrupt and large-scale downward revision in the jobs number will inevitably push the Fed towards lower interest rates sooner.
And of course 'job creation' is a key political goal for all administrations. The jobs number is a report card on the government's economic performance.
The trustworthiness of this number is critical. But trust depends on how reliable the numbers are. And that is where the real issue lies, where a reckoning is needed. The jobs report too often fails the reliability test.
The problem is 'precision' — the lack of it. There are too many revisions, and they vary far too much.Too Many Revisions
The BLS generates at least five versions of each month's jobs number.
The first estimate released each month is always preliminary. It is revised the following month, and then revised again the month after that. The average adjustment after two months is about 50,000 jobs either added or subtracted from the original estimate, but often the change is much larger and may flip from positive to negative unexpectedly.
In addition to the initial figure and the two monthly revisions, the BLS releases an 'annual benchmark revision' which revises the entire 12-month period ending in March. Actually, the BLS first releases a preliminary version of the benchmark in August, and then a final version the following January or February. So, the estimates for each month from April 2023 through March 2024 (the latest complete sequence) went through three estimates in the monthly cycle, and were then benchmark-revised twice more – preliminarily in August 2024, and finalized in February 2025.The Revisions Jump Around Too Much
One might expect the revisions to converge towards the 'true number.' But this does not seem to happen most of the time. For the most recent annual series from April 2023 through March 2024, the initial three monthly estimates were revised downward about 16% on average. Then the preliminary annual benchmark subtracted an additional 818,000 jobs – 'huge,' said The New York Times. Finally, in February of this year, the benchmark revision was itself revised one last time. The new number erased 'only' 589,000 jobs from the monthly estimates.
Two observations emerge.
This extended revision process itself – the fact that so many adjustments are needed, spread out over such a long period – certainly calls into question the integrity of the initial estimate at least – and it is precisely that first estimate that matters most to investors and policy-makers.
The Precision Problem
However, the more important flaw in the process emerges from the extreme variability or volatility of the revisions.
Any measurement process strives for both accuracy and precision. They are not the same. Wikipedia puts it nicely:
The jobs number revisions jump around a lot. The wide dispersion of the series of revised estimates for each particular month indicates a lack of precision in the measurement process. This can be quantified.
Measuring Precision
A common measure of the precision of a series of measurements is relative standard deviation or RSD (the standard deviation of the measurements as a percentage of the overall average of the measurements – also called the coefficient of variation). It is essentially the degree to which the repeated measurements differ from the mean. A standard statistical text describes it this way:What Quantified 'Precision' Means
Statisticians have developed different benchmarks – expressed as maximum permissible RSD values – for different contexts.
[Sources: General Mftg;Pharmaceutical Mftg;EPA; FAO; EURL; FDA;Covid]
Exceeding these benchmarks triggers a range of characteristic concerns.
[Sources: 1,4; 2; 3; 5,6]
One expert, reflecting on personal experience, put it more colorfully.
Modern industrial quality control processes are generally successful in conforming to these standards.Precision Levels for the BLS Jobs Numbers
The BLS numbers don't make the grade. If we treat these revisions as repeated attempts to measure the same thing, we can apply the RSD as a heuristic device. If we compare the initial estimate with the third estimate two months later, the deviations are far above the levels cited as maximum safe thresholds by the sources mentioned above.
In the last three years the RSD for these revisions reached 71%.The variation in the annualized benchmark figures is worse. The RSD is 95% for the revisions from 2003-2024. This is surprising since the benchmark final revision represents the fifth attempt to pin down the true number, and yet the volatility in the signal seems to grow.Explanations, Or Excuses
Why does this high level of volatility occur? Two reasons are commonly given.
The BLS relies on surveys of businesses and households for much of its data – but survey response rates have collapsed.In other words, fewer than half of those surveyed are providing a response. Eventually this must undermine data integrity.
This growing nonresponse problem will also exacerbate the volatility problem.The BLS has committed to releasing the nonfarm payroll number as fast as possible. Each month's number is issued on the first Friday of the following month. Sometimes this means the release can occur on the very first day of the next month (as happened when the July 2025 numbers were issued on August 1). It is the fastest regular cycle for the production and release of any major economic indicator.
It sets up what The Wall Street Journal calls 'a difficult trade-off between speed and accuracy.' The time period for data collection ranges between 10 and 16 days, and some businesses may not be able to report within that period. For example, businesses that pay employees monthly instead of biweekly may not be able to report their data ahead of the deadline. In recent cycles only about 2/3rds of businesses surveyed have been able to respond in time.
The BLS seems to believe that timeliness is paramount, even though the result is an incomplete (and often half-baked?) product. Mainstream Critiques
The problems cited here are well-known in the profession. Just two days before McEntarfer's firing, a bipartisan group of 88 top-tier economists, representing most of the leading universities in the U.S., as well as many former top government officials and heads of several prominent think-tanks, submitted a letter to Congress which was politely but openly critical of the BLS.
An influential financial blogger agreed.
In May, Science magazine addressed the subject in its lead editorial.
Mohammed El-Erian, a prominent economic pundit, reviewed the furor, and acknowledged the institutional challenges.In Sum
Deplore all political motives, and ignore the histrionics on both sides.
Stipulate, for clarity, that the data is not 'rigged.' (That would be organizationally impossible.)
Allow that the firing of a career bureaucrat may have been unfair and unproductive.
Discount for all of that, and you still have a half-broken system that generates grossly inadequate and misleading information on the state of the labor market – creating risks for investors and policy-makers.
The theme of the non-partisan critics of the BLS is the need for modernization of its processes. It is long overdue.
The Bureau of Labor Statistics is the classic Old Dog that hasn't been able to learn the New Tricks. It began gathering statistics on employment and labor conditions in the United States when Grover Cleveland was President. It is still using data collection strategies and techniques that were developed decades ago in a very different technological era, and which no longer work very well. As the economists' letter suggests, the BLS has failed to keep pace with the digital transformation of the economy – citing especially the 'advent of artificial intelligence [which] promises to revolutionize how data are both produced and consumed.'
'Revolutionizing the BLS' is a hopeful goal – but Bureaucracy and Revolution are two words that are not generally found in the same sentence. Perhaps the impetus for change will have to come from outside rather than from within. In any case, the reckoning is timely and necessary.