Looking at the rules for maneuvers on the Athletic's skill page, Agile now applies to your Maneuvers. By the same logic present there (i.e. you are using your fists for the Trip attempt and thus get Agile), shouldn't you also inherit the Finesse trait from Fist and therefore get Finesse on the maneuver? Finesse:
You can use your Dexterity modifier instead of your Strength modifier on attack rolls using this melee weapon
While Agile states:
The multiple attack penalty you take with this weapon on the second attack on your turn is –4 instead of –5,
Shouldn't this allow me to use Finesse with Trip, Grab, etc. so long as I use a free hand to do it (as opposed to a non-Finesse unarmed attack or weapon)?