I think everyone who has started GMing has had the same concerns.  

My personal recommendation regarding your idea of making players unlucky when not following the main plot is to do the opposite: 

- A wizard wants to brew the greatest potion? An alchemist in league with the BBG has years of research that could be vital to their task. 
- A warrior wishes to slay a dragon that destroyed his home? Perhaps the dragon is in possession of a relic that can help the party in the main plot.

What I'm trying to say here is that *the motivations of the player characters will always be more important to them than the main plot* (unless by some chance the main plot is their motivation).  Nobody likes it when they're being stopped from pursuing or achieving their goals.  

I've found the best way to keep everything running smoothly is to tie everything nicely together.  Not everything the PCs does has to relate to the main plot (the BBG might be laying low or a side quest might come up) but by rewarding the players for following it, and making it easier to achieve their personal goals, you not only gain more variety for your quests and adventures but you also make it more fun for everybody.

I'd recommend asking your party before the campaign what these personal goals are and how they plan to achieve them.  Not everyone will have one and some may not know how they will reach their goal but this means *the players can work with the GM and not against them*.