Survival and success do wonders.  As does easy access.  And mentoring helps a lot.

 - We run our own ruleset...have for decades...but all the rules are on
   the wiki (more on that afterwards.  And rules mastery and those who
   read the rules are always rewarded with success.  In a good game
   system this is easy.  When survival rate and character growth go hand
   in hand with Rules Mastery, then people start to notice.  One of our
   guys started to take advantage of some of the advanced rules for
   spell casting and spell recovery; his caster soon outpointed the
   others (he became the Mage mentor later, more on that).  Another one
   started to notice the advanced rules for shield use, and his
   character was one of 2 talk types who survived for over 2 years (that
   particular group is going strong on year 10 of the campaign).   **When
   Rules Mastery equates to character growth and character
   survival...players take note.**

 - We have all our rules on a wiki.  And the wiki has a damn search
   function.  And we have full laptop/ipad compliance in our groups.  So
   every rule is right there.  Mages are supposed to have their spell
   effects and notes up on the screen when casting; and will suffer an
   initiative penalty if they do not.   Make it this easy for the rules
   to be at everyone's fingertips, and they have no excuse.  **This way,
   everyone has the rulebook all the time.**

 - But since it a coimplicated game, as many RPGs are, we started having
   players with Rules Mastery in an area become mentors to other
   players.  The guy I mentioned above with the spell rules mastery? 
   All the otehr casters sit near him.  The tanks all sit near the guy
   with shield use (and other combat rules) mastery. Mentoring is a very
   powerful way to create rules mastery, as there is a social
   expectation between both the new and older player.  And the GM is not
   the only one involved in the use of the rules.  **Mentoring is one area
   that we have really sped up our game and one that alows us to use the
   more advanced rules that would otherwise slow down a game.**