NEW YORK (CNNMoney) -- Federal workers' jobs will be under fire if Mitt Romney and Paul Ryan win November's election.

Both Romney and Ryan have made it clear that they think federal workers are one reason the nation's deficit is too high, and have talked about shrinking payrolls and cutting benefits.

In stump speeches, Romney says he wants to cut the federal workforce to "stop the unfairness of government workers getting better pay and benefits than the taxpayers they serve." Ryan, a House Republican picked this weekend to be Romney's running mate, has a detailed budget plan that targets federal workers.

Unions, in particular, say they're concerned about Ryan, saying he has a record of "undermining" federal workers.

"Rep. Ryan's budget resolution showed a significant lack of understanding and appreciation for the important role federal workers have in protecting our borders, safeguarding our food and medicines, protecting consumers in financial markets, and providing so many other vital services to our nation," said Colleen M. Kelley, president of the National Treasury Employees Union.

Federal workers' pay has been frozen at 2010 levels, a move President Obama supported to save $60 billion from deficits over ten years. If Obama wins in November, the pay freeze is expected to ease.

New federal employees also face increased burdens to pensions. Under a law passed earlier this year, employees hired in 2013 with less than five years of service would pay 3.1% of their paychecks toward their pensions, up from the current rate of 0.8%.

In a Romney administration, three different cuts could be pushed targeting federal workers or their benefits, as suggested by the Ryan budget plan.

-- Federal pay freeze extended five years.

-- Cuts to 10% of the federal work force through attrition and a hiring freeze enacted, limiting agencies to hiring only one new employee for every three federal workers who retire.

-- Requiring federal workers to pay for as much as half of their defined retirement benefit they receive at retirement.

The Ryan budget suggests such changes would save as much as $375 billion from deficits over a decade.

Romney and Ryan, like other Republicans, generally believe that federal workers' pay and benefits are "out of line" with the private sector.

The question of whether federal worker pay is better than those in similar private sector jobs is a tricky one, much debated in Washington. Republican leaders like to say that federal workers are paid twice as much as those in the private sector. Yet, the federal Office of Personnel Management and unions say federal workers are paid 26% less than colleagues in the private sector. Both rely on government statistics to back up their claims.

On average, federal employees make $77,414 a year in wages, about 44% more than private-sector employees at $53,463, according to the Bureau of Economic Analysis. The difference in pay is due to higher education levels in the federal workforce, more white-collar jobs, and the concentration of jobs located in Washington, a pricier place to live than other parts of the nation.

Most nonpartisan policy wonks agree with a recent report from the nonpartisan Congressional Budget Office that federal employees get better benefits than found in the private sector -- as much as 46% better on average compared to the private sector and looking at similar jobs in similar fields, the CBO report said.

That same report found that the federal government paid roughly 2% more in wages than the private sector paid to workers with comparable positions.