I've rephrased the issues the question raises to emphasize the game's mechanics. I hope that's okay. To be clear, though, the extraordinary ability grabbing appendages (as a racial ability search this page) of the trox (and, as a monster, here) is vastly underdetailed and rarely discussed, making any conclusion reached about it subject to substantial GM intervention.
Can a trox use its grabbing appendages to grapple a Large foe and, while maintaining the grapple, attack the grappled foe or other foes outside the grapple? Ask the GM. This reader hopes that your GM says yes. (See below.)
Is a trox that's maintaining a grapple with its grabbing appendages treated as a creature that's littler than Large because the appendages are littler than its normal limbs? This reader says no. For this reader, for that kind of exception to apply the description would have to say so explicitly.
Are the trox's grabbing appendages Medium or Large? Typically, this isn't a thing. That is, a creature's limbs don't usually have size categories—size categories are for entire creatures (and objects). A human arm, for example, isn't a different size category from the human to which its attached (even if the human's unarmed strike is treated as a Tiny weapon).
Can the trox use its grabbing appendages to initiate then maintain a grapple against a foe within its reach in addition to using its normal limbs to initiate then maintain a grapple? This GM would say no. If a typical giant octopus—that possesses the extraordinary ability grab—struggles to maintain a grapple against multiple foes simultaneously (see this question), then, in this GM's opinion, there's no way for the typical trox to maintain a grapple against multiple foes simultaneously. However, another GM's opinion may differ, and the house rules necessary to make the ability grabbing appendages worth the price might allow it. (See below.)
Also, just to be clear, a typical attacker—including a trox—that initiates a grapple must "move [a nonadjacent] creature to an adjacent open space [of the attacker] (if no space is available, your grapple fails)." A typical creature can't maintain a grapple against a foe at a distance.
When the description of the grabbing appendages ability says that the grabbing appendages are a "small group of appendages that are useful for little more than to aid in grappling," what does it mean? Nobody outside the creator of the trox knows, and, to my knowledge, that developer hasn't clarified. (See below.)
More on grabbing appendages
The extraordinary ability grabbing appendages of the trox does two things. First, the ability is an excuse for the writer to have given the trox the bonus feat Improved Grapple. (The feat Improved Grapple is, by the way, different from a monster's extraordinary ability grab, and that grab ability was called in Pathfinder's antecedent D&D 3.5 the extraordinary ability improved grab; if you're coming from that earlier game, it's especially easy to confuse the two!)
Second, the ability allows a trox to "maintain a grapple and still make attacks with their main [i.e. not grabbing] appendages." What this means is incredibly unclear.
See, normally, if a creature initiates a grapple, each round it must take a standard action to maintain the grapple, and, if it does, it only also either moves, damages, pins, or—rarely—ties up the victim and doesn't really per se make attacks. Yet, because maintaining a grapple consumes the grappler's standard action, the typical creature, for example, can't take a standard action to maintain a grapple and also take a a standard action to make a standard attack or take full-round action to make a full attack. (Bear in mind that the typical trox has only the feat Improved Grapple and the grabbing appendages ability and not the monster ability grab!) Technically, it seems, based on the ability's language alone, the grabbing appendages ability does not somehow add to the number of actions a trox can take in a round, and that makes the ability—still technically, by the way—not particularly useful.
That is, what a trox can do under this strict reading of the grabbing appendages ability—and what other creatures that lack grabbing appendages can't—is make attacks of opportunity while in a grapple. Normally, creatures in a grapple can't make attacks of opportunity at all, but a trox can "maintain a grapple and still make attacks with their main appendages," and that should include—because of the specific-beats-general fashion that underlies the Pathfinder rules—making attacks of opportunity. (Attacks of opportunity fall outside the action economy, being unique and neither free, swift, immediate, move, standard, full-round, nor 1-round actions, even though attacks of opportunity remain attacks.) As written, this is, so far as this reader can tell, pretty much the typical trox's only option for making any attacks while a trox maintains a grapple.
However, read this strictly, there's just no way the grabbing appendages ability is worth the 4 race points that the trox was charged for it. (According to the guidelines for Creating New Races the grabbing appendages ability may've cost 2 points for the Improved Grapple feat then—for just that other ability—another 4 more points!)
Thus a more generous reading sees the GM read the ability, look at the price of the ability, say, "That's a lot of race points for a really niche ability," and rule that, for example, a trox that starts a grapple can maintain that grapple without taking a standard action and can also both make attacks of opportunity while grappling and—while the grapple's maintained—either take a standard action to make a standard attack or take a full-round action to make a full attack, either taken normally as if the trox weren't grappled.
This alternative reading makes the trox a more attractive option to players looking for a race that's good at rasslin', justifies the high race point cost of the grabbing appendages ability, and to this GM seems a reasonable accommodation. However, because this writer is just a random Internet stranger with an opinion and because folks just don't really discuss the trox ability grabbing appendages on messageboards, expect table variation as each GM reads the grabbing appendages ability differently. Just hope it's not strictly, or else there's probably a better mechanical choice for a rasslin' race than the trox.
