Economists had hoped that the bigger pay increases in recent months would continue and boost Americans' ability to spend.
Some economists said they expect a slowdown this spring, though not as severe as in the past three years.
"We don't anticipate the slowdown becoming too severe, not when the housing recovery is firing on all cylinders, but it is a reminder that the U.S. is still unable to sustain what used to be just average rates of growth," said Paul Ashworth, an economist at Capital Economics.
The decline in the work force reflects several trends, economists say: Many of those out of work become discouraged and give up on their job hunts. And as the population ages, more people are retiring.
Most analysts think the economy strengthened from January through March, helped by the pickup in hiring, a sustained recovery in housing and steady consumer spending. Consumers stepped up purchases in February and January, even after Social Security taxes increased this year.
Still the higher taxes have reduced paychecks. And many economists say steep government spending cuts that began taking effect March 1 will slow growth in the spring and summer.
Mark Vitner, an economist at Wells Fargo Securities, thinks the economy expanded at a 3.2 percent annual rate in the first quarter. But he forecasts that growth will slow to a 2 percent annual pace in the current second quarter, and then rebound after the impact of the government spending cuts fades.
Economists expect the spending reductions will shave half a percentage point off economic growth this year. Many federal workers will experience pay cuts. And government contractors will likely cut jobs. That could also drag down overall monthly hiring.