(Edit: Assuming the other requirements are met as the other answer points out, such as any difficult terrain (high winds for example))
"Up to your speed or less"
So long as you have a flight speed, you can charge while flying - similarly for regular moving, climbing, swimming and burrowing. See feats like Death from above for corroboration.
You can't however charge while making those movement types without the relevant speed.
This is easier to see with climbing or swimming as flying without a flight speed generally gets referred to as 'falling'. But, for example, dropping* downwards from a tree in an ambush wouldn't count as a charge due to lacking the relevant movement speed, though you would probably get +1 for being on higher ground.
(While the climb skill specifically mentions not being able to run while climbing, even with a climb speed, RAW the ability to run or not doesn't actually hinder your ability to charge; RAW the two are different things. Similarly the swim skill limits running to a straight line, even with a swim speed.)
*You could however jump downwards (e.g. Branch Pounce), using the acrobatics skill, though this jumping charge would still need to be in a straight and unobstructed line. With a successful jump check, I would count this as movement made using your regular overland speed.