Only sort of
First of all, delaying doesn't let you skip your turn. It lets you pick any later initiative in the round and has your turn happen then, or lets you wait till any time later that round and act then. Since even count $\infty$ in round 3 is still in round 3 not round 4, and since it only lets you wait till later that round, you still have to take your turn at some point, even if that point can be an arbitrarily large count. It also doesn't need to be part of your turn and doesn't usually happen on your turn; it's a not-an-action action that's also a special initiative action, and not-an-action actions can be taken whenever. That said, this doesn't completely eliminate the problem.
There are very few effects that tick down based on turns. There are a lot that tick based on rounds, and the interaction between rounds and turns means that things that require an action each round can have that action paid on any initiative count that round you're able to pay it.
For stuff like Bardic Performances this doesn't actually change how long the effect lasts because the effect ends immediately at the end of the round you didn't pay the cost, not just before your next turn. It does mean, however, that if your bard is surprised (and thus unable to take actions for a round) while maintaining a performance, it is worth taking the delay action (which you can take since it's a not-an-action action) in case a party member or something else renders you able to act somehow (so that you could then spend the free action needed to maintain the performance, which you can only do during your turn). Similarly for conditions, it might be worth delaying your turn so allies can help you if you're worried that e.g. bleed damage is going to kill you.
Buffs typically last a number of rounds or minutes, and I can't find any that only last a number of turns. 1 round in this context is the time from an initiative count in one round to the same count in the next, so this should not have any problems nor be affected by delaying.
Spellcasting, though, is affected, since spells with long casting times (1 round or more) come into effect just before your turn in the round when they go off. This means you can use delaying to have them go off any time you like after your 'normal' initiative count in the turn they happen, if you're willing to delay. You can't take them with you to the next round, though; delaying doesn't let you get out of the round without a turn.
The biggest effect is on conditions measured in turns, positive effects that happen on the start of people's turns, and other things like that. While you don't need to take delay actions on your turn, you can do it after your turn starts, and in so doing start any number of turns you like within a single round (so long as you take no actions, not even free actions, on the turns you delay). This rarely comes up, but it does trivialize the clear intention of some abilities. If you are looking to avoid 'shenanigans', as you call them, it's worth houseruling delay to only be usable outside of your turn, so you can use it just before your turn but not after it starts. When you do this you need to decide if the just before your turn can be voluntarily before or after other stuff that happens just before your turn (like summon monster spells), or if you wanna do something different with it.