An increasing number of seniors are choosing to get their Medicare benefits through Medicare Advantage plans. But do they understand what they’re signing up for?

A recent report suggests they may not, especially when it comes to which hospitals are included in the plans’ networks.

An alternative to traditional Medicare and administered by private insurers, Medicare Advantage plans are typically health maintenance organizations or preferred provider organizations that offer seniors hospital and medical coverage (Medicare Part A and Part B), and sometimes prescription-drug coverage (Medicare Part D), dental care or benefits such as gym memberships.

Using the directories to pick a plan is “like trying to do your taxes with an abacus; it can be done, but not easily,” says Gretchen Jacobson, associate director at the Kaiser Family Foundation and co-author of the report, adding that Medicare’s website doesn’t provide an easy way to compare plan networks side by side.

About 17 million beneficiaries, roughly 30% of the Medicare population, enrolled in a Medicare Advantage plan in 2015, according to the Congressional Budget Office, which expects private-plan enrollment to grow to about 29 million Americans, or about 40% of the Medicare population, in 2025.Many seniors like Advantage plans because in addition to added benefits such as dental coverage and gym memberships, they eliminate the need for additional insurance, such as Medigap, and some plans cover the gap in Medicare prescription-drug coverage known as the “Donut Hole.”

“These sorts of benefits can make a big difference to someone on a fixed income,” says Clare Krusing, a spokeswoman for America’s Health Insurance Plans, an industry group.

That said, some Advantage plan members can face significant expenses if they seek treatment out of the network, experts say, which is why having a clear understanding of the size and composition of each plan’s network is important.

Of the 409 plans studied by Kaiser, 23% offered what the report termed broad networks, meaning 70% or more of the hospitals in a county were included. About 61% had medium-size networks, meaning between 30% and 69% of all hospitals in a county were included; and about 16% had narrow networks, with less than 30%.

While some plans with narrow networks get good ratings from the Centers for Medicare and Medicaid based on metrics such as preventive care and customer service, the Kaiser study found that in general they are more likely to exclude institutions that specialize in treating rare or more complicated conditions.

According to the report, 75% of narrow networks excluded National Cancer Institute Cancer Centers, which have experience handling rare and complicated cancers and provide more access to clinical trials; 49% excluded academic medical centers; and 21% had no hospitals with an accredited cancer program.

In the worst case, by not including certain institutions, networks can be built to exclude the sickest, most expensive patients, says Karen Davis, a professor and director of Roger C. Lipitz Center for Integrated Health Care at Johns Hopkins Bloomberg School of Public Health.

The insurance industry’s Ms. Krusing says some plans may exclude brand-name hospitals from their networks if there is no evidence they provide better care than cheaper community or regional hospitals. Medicare Advantage plans are increasingly enrolling patients with chronic illnesses, focusing on preventive care and reducing avoidable hospitalizations, she says.

Whatever the merits of the various plans, experts say it’s easy for the details to get lost in the generally weak shopping experience they offer.

The Kaiser report found, among other things, that some plan directories mislabeled rehabilitation or post acute-care facilities as hospitals, while others included hospitals that had been closed for several years.

A separate analysis prepared for the Centers for Medicare and Medicaid Services and presented at a conference in Baltimore in September, found a 46% chance that an Advantage plan’s listing of a primary-care physician, oncologist, ophthalmologist or cardiologist contained some sort of inaccuracy, including where the provider was located or whether they were accepting new patients.

“You can’t shop around very well,” says Timothy Layton, an assistant professor of health-care policy at Harvard Medical School. Trying to anticipate what specialty care may be needed in the coming year, along with calculating deductibles, copays and other out-of-pocket costs makes the task even more difficult, he adds.

The Centers for Medicare and Medicaid Services (CMS) and insurers say they are beginning to address this problem.

“We are actively working with insurers and the provider community to further develop best practices and solutions,” says CMS spokesman Raymond Thorn.

In 2017, plans that apply to expand their networks will have to submit their entire network for review by CMS. Currently, CMS examines provider networks for accuracy of listings in new geographic areas to which a plan is expanding, or in response to triggers such as complaints, changes of ownership or termination of a large provider.

Meanwhile, “seniors may have to do some additional work to make sure they chose the best plan for them,” says James Cosgrove, director of health care at GAO. Ask questions about which providers are in which networks, Dr. Cosgrove says.

Robert A Berenson, a fellow at the Urban Institute, a Washington, D.C.-based think tank, recommends asking how frequently a plan refers patients out of its network. Medicare Advantage plans are often reimbursed at a rate close to that of traditional Medicare patients even if a provider is out of network, he says.

Dr. Jacobson recommends State Health Insurance Assistance Programs, or SHIPS, for help choosing a plan. These programs provide free insurance counseling to Medicare beneficiaries in all 50 states.