Look at other spells in the book
I'm under the impression that, as Harry's classmates attempt to cast the spell, they are not quite getting it right some times. Normally you cannot cast a spell you don't know or one that you don't meet the requirements for. But, as nitsua60 points out, if the spell is a scroll, you can cast it at any level as long as it's on your class spell list. There is a save DC for casting it successfully (DMG pg. 200) and a list of mishaps for failure (DMG pg. 140). For teaching and learning a new spell as you've described, you can use this system to simulate failure.
However, there are many spells in dnd 5e that have variability of effect beyond the damage dice. I think if you take a look at these spells, you should be able to come up with a system that works for you. This wouldn't be unprecedented and as far as creating a spell it would be fairly "vanilla", without teaching your players additional concepts from other games.
Caveat:
I am shying away from directly suggesting how your spell should work since I've no experience with creating such a spell. Rather, to answer your direct question of "How can I use D&D 5.0 mechanics to simulate varying degrees of success with a harder spell like Expecto Patronum?", I suggest you can do so by looking at other spells within the book. What you want to get out of the spell is a little difficult for us to answer, so I am attempting to lead you to the answer you seek. I hope this answer can be a stepping stone for you.
Caster's Choice:
One of the more straight-forward ways is to determine the outcome based on what the caster is trying to accomplish. Look at Animate Objects, notice the chart gives different effects for how large the objects are. Larger objects are better, but you get less of them. This is pretty common among minion-controlling types of spells, where you can pick few large creatures or many small ones. This concept can be tweaked to work for your spell.
Target Specific:
Similarly, Creation allows you to create something, but based on its composition it lasts from as long as 1 day for vegetable matter to 1 minute for adamantine. This variable length puts limits on the caster based on the difficulty of what they're trying to simulate. In the same way, your spell could be castable by all wizards, but it's effect diminished based on what they're doing with it.
Stages:
You can affect it in stages, as is done with Control Weather and Prismatic Wall, where you must use the spell to change the outcome gradually over time. This makes the spells last/take a long time, but without being "on/off" in effect.
Variable DC:
You can also change the DC, as is done with Scrying. Scrying specifically changes the DC based on how well you know the target and whether or not you've got something of theirs. You can modify this system to change the difficulty of the spell. This behavior is quite common for the DM as they create DCs for skill checks and the like. You can find better explanation of such effects in the DMG.
Divine Intervention: Some spells' effects are determined entirely by the DM. Spells like Divination and Contact Other Plane are about asking the DM questions, which the DM answers in his/her own way. More famously, the Wish spell creates an effect by request of the caster that the DM carries out (or doesn't) in his/her own way. Your spell could also behave this way, but in my mind that might be less fun.
Random:
Finally, you can go random with it: roll a dice to determine the outcome. Spells like Prismatic Spray, Reincarnate, or Teleport use this system. Teleport is by far the most variable: There are 4 different outcomes based on 7 degrees of familiarity and a percentile roll. These are reconciled in a chart, so if you do go the route of extreme variability, a chart might also help you.