By default, yes, you can only recover 1 focus point after every use no matter how many you spent. However, you can take class feats that increase this amount. All classes with focus spells have access to an "X Focus" feat at level 10-14 that allows them to recover two points at once, and some have an additional "X Wellspring" feat at level 18 that increases it to three points (the others are always limited to two at most, unfortunately).
Bard: Inspirational Focus (12)
Champion: Devoted Focus (10)
Cleric: Domain Focus (12), Domain Wellspring (18)
Druid: Primal Focus (12), Primal Wellspring (18)
Monk: Meditative Focus (12), Meditative Wellspring (18)
Sorcerer: Bloodline Focus (12), Bloodline Wellspring (18)
Wizard: Bonded Focus (14)