With the ascension of California, Maine, Massachusetts, and Nevada to the ranks of the legal marijuana states after last November’s election, nearly a sixth of the country now lives in places that have freed the weed.

Still, even though it may be legal under state law, marijuana remains prohibited under federal law. Pot smokers in California or Colorado don’t have to worry about a DEA agent breaking down their doors and taking them off to federal jail—there just aren’t enough DEA agents to actually enforce prohibition on the individual level. But the fact is, they are using a federally illegal substance, and there can be consequences to that.

1. Jobs

Most employment in the U.S. is “at will,” meaning employers can fire employees for any reason they like—or no reason at all—unless it violates anti-discrimination laws. That means employers can fire or refuse to hire marijuana users even where marijuana is legal, just as some companies have done with tobacco users.

For many employers, having a “no marijuana” provision is merely a choice, one that can be changed by changing norms and attitudes or, perhaps, by a paucity of applicants willing to work for a company that intrudes on their personal liberties. But for other employers, federal marijuana prohibition means they must bar marijuana use. That includes all federal agencies, many companies that contract to do federal work and sectors like the transportation sector, where federal law mandates drug testing and the firing of people who use federally illegal drugs.

2. Housing

Federal marijuana prohibition means no one living in Section 8 or other federally subsidized housing can use marijuana. Typically, housing authorities do not drug-test or otherwise attempt to screen residents or applicants for marijuana use, but they do see and act on reported violations or arrests reported to them, and the pot-using residents get evicted.

And residents don’t even have to be using marijuana themselves. There are many federal housing horror stories of long-time, elderly residents being evicted from their homes because their children or grandchildren got caught using or possessing pot on the premises. Young stoners: Do not get your grandma thrown out on the street by getting caught with weed at her place!

But it isn’t just residents of public housing. Renters, condo owners and mobile home park residents can all be subject to codes or codicils that no local, state, or federal law be violated. And smoking pot in a legal state is still a violation of federal law.

3. Gun Ownership

A federal appeals court has ruled that marijuana users do not have a Second Amendment right to gun ownership because federal law does not allow selling guns to “illegal” drug users. That ruling came in a case involving a medical marijuana patient, but it applies to all marijuana consumers because Congress thinks that marijuana use “raises the risk of irrational or unpredictable behavior with which gun use should not be associated.” (Alcohol, which certainly fits that criteria far more closely than marijuana does, is not included in the ban because it is a federally legal substance.)

The Bureau of Alcohol, Tobacco and Firearms wrote that decision into its rules last month, adding a new line to the gun ownership application form that states that marijuana is still illegal under federal law. For some reason, the National Rifle Association has not bestirred itself over this particular assault on gun owner rights.

Under current law and federal pot prohibition, would-be gun buyers face a dilemma: Lie about marijuana use and be able to get a gun, or be honest about marijuana use and be barred from buying one.

4. Military Service

Though marijuana has shown promise in treating conditions such as PTSD and chronic pain, active military members can’t use it without jeopardizing their careers. Facing social reality, some branches of the service are no longer barring recruits with a history of marijuana use, even if current, but it will still get service members in trouble and possibly booted from the service.