Whether you call it "tipflation" or "tip creep," Americans feel they're being asked to tip more, and more frequently, in recent years. And they're getting sick of it.

Some 6 in 10 adults view tipping negatively, and 35% say things have gotten "out of control," according to a recent survey from Bankrate.

It's a feeling that has many wondering where to draw the line when it comes to who receives a gratuity. And in some cases it really is black and white, says Ted Rossman, a senior industry analyst at Bankrate.

"I don't feel bad hitting 'no tip' at a self-checkout. I was asked to tip at a pick-your-own strawberry farm — that's another one that feels like overreach," he says. "But at places like sit-down restaurants, hair salons, barber shops or getting food delivered, tipping is customary."

Americans don't seem to always feel that way. Just 67% of adults in Bankrate's survey say they always tip at restaurants. Just over half of respondents always tip the people who cut their hair and deliver their food. And 4 in 10 always take care of taxi and rideshare drivers, while a mere 22% tip hotel housekeepers.

"I was very startled by that survey," says Thomas Farley, an etiquette expert and keynote speaker known as Mister Manners.

In the five scenarios mentioned above, tipping is all but mandatory. Crucially, though, "always" is a big word. While there are people and scenarios which generally call for a tip, there are cases where it may be appropriate not to leave one, etiquette pros say.

At a restaurant: 'downright rude' or 'deliberately sloppy' service

Things have to be pretty dire to justify not leaving a tip at a restaurant.

"The federal minimum wage for tipped workers is $2.13 an hour," says Rossman. "So if you're not tipping in those scenarios, you're really taking money out of the server's pocket."

That restaurants pass the burden of compensating their employees is not the server's fault. Nor, generally, are things you may perceive as bad service, such as food coming out late or a poor dining environment.

So what would it take to not tip a server?

"Your service would have to be objectively belligerent. Not just neglectful, but downright rude and deliberately sloppy," says Farley. "I don't think I've ever encountered that in my lifetime of dining out."

Which isn't to say that you or people in this survey haven't. If a server is rude, nasty or racist, "you should not be rewarding them for that behavior with a tip," says Farley.

But you shouldn't just walk out of the restaurant either. Rather, this is an opportunity for you to talk with management about the problem and explain why you docked or forwent the server's tip, says Elaine Swann, a lifestyle and etiquette expert and founder of the Swann School of Protocol.

"If you address management and then leave a lower tip, they'll know you weren't just a jerk or uneducated when it comes to tipping," she told CNBC Make It. "Whether they agree with your complaint or not, they'll have an understanding of why you left a lower tip."

A tip is 'a baseline for good service'

What about people who provide a service, such as cab drivers? When it comes to those who cut your hair and deliver your food, you should just about always tip, etiquette experts say.

But the service has to actually be there.

"Tipping is happening, in most cases, for at least a baseline of good service," says Farley. "If you went to the hair stylist and asked for a bob and got a crew cut, to generously tip the barber who did that would be sending a message that you were pleased with the service you got, which would not be the case."

In other words, if you asked for a service and out-and-out didn't receive it, you're OK reducing or withholding a tip, etiquette pros say. There's no need to tip a cabbie who was driving recklessly or got lost for an hour on the way to a destination.

There's also no need to tip the hotel staff on days you didn't receive housekeeping services.

But as a rule of thumb, barbers, delivery people, rideshare drivers and housekeepers all rely on customer tips to make ends meet. And if these people make your life easier, they deserve a tip — even if you have to go a little out of your way.

That may mean picking up some small bills to leave for hotel housekeepers, says Diane Gottsman, an etiquette expert and founder of the Protocol School of Texas.

"As a guest, you're directly benefitting from housekeeping," Gottsman told Make It. "They're scrubbing the toilet you're using, they're washing the sheets you're sleeping in …. It's a gritty, underpaid job."

Some tips are up to your discretion

Just 15% of Americans say they always tip furniture and appliance delivery people. Some 11% say they always tip when picking up takeout food and 10% always tip home services and repair people. Those numbers may actually be high, etiquette pros say, as you're under no obligation to tip in these situations.

It may make sense, however, to tip when someone goes above and beyond. If you paid for curbside delivery, for instance, and the workers bring a heavy piece of furniture into your home, that may be worth a few extra bucks, Farley says.

The same goes for a counter worker who knows your order as soon as you walk in the door or gives you special treatment. But you don't necessarily need to tip someone handing you a muffin at a bakery counter, Farley says.

"If they say, 'Oh, we just have some that may be coming out of the oven — let me go check for you,' that might be a different story," he says. "That's someone who is taking extra steps for you to give you the above and beyond experience."

As for home repairs or service people — the group in Bankrate's survey who the fewest Americans tip as a matter of course — you can probably keep the cash in your pocket.

"For service professionals like plumbers, my feeling is that unless there are extenuating circumstances and they came out in the wee hours of the morning, there's no expectation to tip," he says. "You're probably paying on an hourly basis, most likely a very high amount."

Want to be a successful, confident communicator? Take CNBC's new online course Become an Effective Communicator: Master Public Speaking. We'll teach you how to speak clearly and confidently, calm your nerves, what to say and not say and body language techniques to make a great first impression. Sign up today and use code EARLYBIRD for an introductory discount of 30% off through July 10, 2024.

Plus, sign up for CNBC Make It's newsletter to get tips and tricks for success at work, with money and in life.