This is my second answer to your question covering specifically this parts: "Does holding someone hostage give partial cover and is there a chance of hitting the hostage if you miss hitting the intended target that is holding the hostage?" and "My opinion is that it should trigger a mechanic to see if the player hits or misses the hostage. [...] What do the rules say?".
I'm going to give names to people in this scenario; Grunt (male) takes Lidia (female) as hostage; Leonidas decides not to negotiate but attack Grunt.
For the first item in your question, cover:
I'm going to assume all are corporeal medium or small creatures. Tiny and smaller creatures, three or more step sizes of difference and incorporeality are going to change the situation a lot.
All of them occupy different spaces, then Lidia provides cover to Grunt for attacks through her space (+4 AC to Grunt) and interferes with other creatures actions (some other creatures cannot move through her space, etc.).
If Leonidas want to attack through Lidia's space he needs a reach or ranged weapon. Otherwise, he cannot hit Grunt. In a narrow corridor this strategy nicely works for Grunt. If Leonidas can move around Lidia, well, Leonidas best course is to move around Lidia then directly attack Grunt. Grunt can only counter it using a readied action to keep the grapple then move Lidia to keep her in between Leonidas and himself.
Also, if Lidia is considered a friendly character for Leonidas, Leonidas takes a -4 penalty to attack with ranged weapons, no matter the angle he attacks, except if having Precise Shot feat. If Leonidas doesn't care for her safety he can ignore that penalty. By RAW, missing because of that penalty doesn't mean he hits her.
For the second item in your question, chance of hitting the hostage, what do the rules say?:
By RAW, it only has more AC. Failing to hit him just means failing to hit him.
House rules, anyway, provide mechanics for the situation. I can only come with two house rules for this scenario. I'm going to assume in this situation Grunt's AC is 12, with Lidia's cover became 16. Lidia's AC is 14.
The simpler rule: Leonidas attack and miss, check if Leonidas's attack roll is 4 or less, if yes, Leonidas automatically hits Lidia. A variant of the rule is, Leonidas attack and miss, check if Leonidas's attack roll is between 12 and 15 (the AC range gained by Lidia's cover), if yes, Leonidas automatically hits Lidia. This first rule is nice for running the game faster.
The second rule: Leonidas attack and miss, check if Leonidas's attack roll is between 12 and 15 (the AC range gained by Lidia's cover), if yes, Leonidas probably hits Lidia, compare the attack with Lidia's AC, meaning if Leonidas attack was 12 and 13, he didn't hit Lidia, but 14 and 15 did. This second rule is nice for running a more realistic fun game.
Conditions: Determine if the hostage and holder have special conditions, like grappled, as secondary effect of the hostage situation, and if the hostage alone has shaken, pined, helpless conditions too. Adjust accordingly.
Sizes and cover: By RAW, you should consider partial cover and total cover, some of them at GM discretion. E.g., a GM could determine a gnome hostage only provides partial cover to a half-orc holder; conversely, a GM can grant total cover a gnome taking an ogre hostage (yes, possible, the ogre is hypnotized and the gnome i using a monstrous technological item to threaten its life). Larger creatures could have some spots you can attack that are not covered by smaller hostages.
Sizes and space: By RAW, it is not clear what happens in a grapple when two creatures that can occupy the same space are in a grapple. But i'd assume they can be in different spaces or the same space. A dragon holding a gnome as a hostage can place the hostage in an adjacent space between the dragon and the hostage friends. Also it can be in the same space as the dragon, as happens in the rules for the dragon crush attack. The dragon can also have the gnome inside his mouth threatening to swallow her if the party don't accomplish his demands. In those cases, the characters can attack the dragon directly, i'd provide no cover from the hostage to it, in fact the dragon could be providing cover to the gnome so the party can use an area spell.
Incorporealty: OMG. This is a mess. Rule it in a per case basis.