Movement through the wall should be voluntary
Below is a direct quote of the answer @gandalfmeansme provided to the related question "If a creature is moved through a prismatic wall multiple times is it affected multiple times?"
There are numerous spells that do damage or otherwise inconvenience a creature when that creature moves, or is moved, into their area of effect. Since you used Wall of Fire as an example, let's examine the text that spell uses:
Wall of Fire
A creature takes the same damage when it enters the wall for the first time on a turn or ends its turn there. (PHB, p. 285)
This movement does not need to be voluntary: if you are shoved into a Wall of Fire, or walk into it yourself, you still "enter" the wall. However, note the similar (but not identical) text of Prismatic Wall (PHB, p. 269, bold added):
When a creature attempts to reach into or pass through the wall, it does so one layer at a time through all the wall’s layers. As it passes or reaches through each layer, the creature must make a Dexterity saving throw or be affected by that layer’s properties as described below.
This "attempts" is crucial. If a creature is grappled and moved against its will, it did not "attempt" to move in the direction it is moving: it had no choice in the matter. For a creature to "attempt" to pass through the wall, it must be moving through it (or reaching into it) voluntarily [emphasis mine].
This is also complicated by the fact that the second sentence in the quote above doesn't require intent, but simply that a creature "passes through" or "reaches through" a layer of the wall. It is a DM's prerogative to decide which of these sentences takes precedence in this situation.
What happens if you're pushed/grappled into the wall?
It's very hard to say. Strangely, none of the layers of the wall prevents a creature from physically passing through it while they are active (layers prevent things like magical and nonmagical ranged attacks from penetrating the wall). Nor is there any text that directly states that the wall is solid. It is described as "opaque", but that simply means you cannot see through it.
It would be a DM's call what would happen if you shoved a creature towards the wall, or attempted to run through the wall with a creature grappled [emphasis mine]. A case could be made for the idea that the wall is solid when you are thrown into it, because of the "attempts" mentioned mentioned earlier: this time, "attempts" is significant not because it indicates willing movement, but because it suggests a possibility of failure. If you "attempt" to move through something, it does somewhat imply that moving through it is not guaranteed, which might imply some level of solidity. Again though, that's very much a DM's call.
One thing seems clear to me: it does not seem likely that the wall is meant to be both harmless and intangible to creatures who are pushed through it involuntarily [emphasis mine]. If that were the case, a group of enemies could simply shove each other through it and avoid its effects altogether (except the unlucky one who is last).
[Section on the blinded condition removed]