The claw damage is part of a melee attack; it happens "in an instant"
"In an instant" is not as yet a defined mechanic/action economy term. The Action in this case is a melee weapon attack.
This ability appears to be very similar to the Paladin's Divine Smite ability, or the Cleric spell Inflict Wounds. As this is playtest material, work with your DM to choose which it should be treated as. (See Below)
Textual analysis
Let's look at the rules text very closely. How are disciplines presented? On page 9 it says:
Each psionic discipline has several ways you can use it, all contained in its description. The discipline specifies the type of action and number of psi points it requires.
What is the type of action for this use of the Bestial Form discipline?
Bestial Claws (1–7 psi). You manifest long claws for an instant and make a melee weapon attack against one creature within 5 feet of you. On a hit, this attack deals 1d10 slashing damage per psi point spent. (page 11)
The text as written, in terms of the order of instruction presented, makes the claw damage contingent upon a successful melee attack "in an instant." It is not a bonus action; were it a bonus action the text would call that out. Compare to Bestial Transformation:
As a bonus action, you alter your physical form ...(page 11)
Is it more like a spell, or more like a class ability?
That's hard to say since this is play test material. Psi points are a limited resource just as a Paladin's, or a Cleric's, spell slots are.
I see the claws as similar to a Paladin's spell slots used for Divine Smite, due to the order of how the text was written.
Divine Smite
Starting at 2nd level, when you hit a creature with a melee weapon attack, you can expend one spell slot to deal radiant damage to the target, in addition to the weapon’s damage. The extra damage is 2d8 for a 1st-‐‑level spell slot, plus 1d8 for each spell level
When you hit you burn slots/psi points for damage from the ability.
My reading may be generous.
Another DM might treat it like a spell attack such as Inflict Wounds (Cleric, first level) such that you have to decide to use the spell and take the risk that you might miss, and burn the spell slot(s) (or in this case psi points) as part of the attack. A good reason to rule this way is that the text suggests that you make the decision, you manifest the claws "in an instant," then you attack, and then the damage happens, and psi points expended.
Inflict Wounds
Duration: Instantaneous
Make a melee spell attack against a creature you can reach. On a hit, the target takes 3d10 necrotic damage.
At Higher Levels. When you cast this spell using a spell slot of 2nd level or higher, the damage increases by 1d10 for each slot level above 1st. {similar to using higher spell slots}
Both use finite resources to boost damage, but one has a bit more risk of missing out on the bonus damage than the other.
Discuss this with your DM.