Your interpretation is correct.
For the duration, you can cast spells as though you were in the illusion’s space, but you must use your own senses.
The only thing Invoke Duplicity changes is the point of origin of the spell. There is no other text allowing for any other adjustments to how the spell is cast.
You can’t cast spells, and your ability to speak or take any action that requires hands is limited to the capabilities of your beast form.
The text on Wild Shape is equally clear: "You can’t cast spells." The next clause is joined by "and" but it parses out exactly to the same effect if you split it into two sentences:
"You can’t cast spells. Your ability to speak or take any action that requires hands is limited to the capabilities of your beast form."
Consider that Moon Druids get Elemental Wild Shape at level 10. The art for fire and earth elementals clearly depict hands with fingers, and all four forms have languages which means they're clearly capable of speaking. There's no text in the ability that supersedes "You can't cast spells" from the basic ability, so they still can't even though they have mostly the same bits a humanoid has.
As for balance...
Allowing the character to cast spells while Wild Shaped is the province of a much higher level ability, Beast Spells, that Druids do not get until Level 18. Allowing a character the ability earlier than that (significantly earlier, as a Druid 2/Cleric 2 has Wild Shape and Invoke Duplicity) is definitely unbalancing.
While the character in question may not be a Moon Druid, some folks consider the toughness of some the Moon Druid's Wild Shaped forms combined with Beast Spells (and a few levels later, Archdruid) to be overpowered by itself. Granting a similar capability to a multi-classed character at an even lower level is even more overpowered.