Attempting to break an unattended object may be an attack
You're right that's there's no formal action cost associated with attempts to break an object. However, Breaking Items says
When a character tries to break or burst something with sudden force rather than by dealing damage, use a Strength check (rather than an attack roll and damage roll, as with the sunder special attack) to determine whether he succeeds. Since hardness doesn't affect an object's Break DC, this value depends more on the construction of the item than on the material the item is made of. Consult Table: DCs to Break or Burst Items for a list of common Break DCs.
Emphasis mine. Reading between the lines, this sounds like a creature can make a Strength check to break an object instead of making an attack.
(By the way, it's rare for the game to allow anything potentially destructive to happen with less than a standard action's effort. Further, arguing that destroying is technically manipulating so a creature can take a move action to manipulate an item to destroy an object sadly limits the efforts of, for example, an angry, drunk giant octopus to wreck its aquarium furnishings, as on its turn the giant octopus could take the full attack action to make up to 9 attacks but can take at most 2 move actions.)
Thus, using the above ruling, in the case the question describes the ogre's foes have total cover (preventing the ogre from attacking its foes) and, likewise (and almost redundantly), total concealment. The ogre may, nonetheless, suspect its foes' location (whether that's because the location is so little or because of the ogre's successful Perception skill check (it has a +5, after all) or something else), so the ogre—because it has but one attack, even if it takes the full attack action—can take a standard action (instead of making its attack) to break an object by making a Strength check (DC 15 for the question's shack wall), with success destroying that 5-foot wall section and failure having no appreciable effect on the ogre-resistant shack wall. If successful, the next round that ogre can attack—or the other ogre can attack this round if it hasn't already taken its turn. (Also keep in mind rules for squeezing and threatening around corners when using such tactics. The game makes big creatures using this tactics do arithmetic and geometry.)
As an aside, most of the time in combat a creature isn't trying to break unattended objects, like furniture or walls, and, instead, a creature is trying to break its foe's objects, like their weapons or (if NPCs, anyway) their magic items. In this latter case, the creature always uses the combat maneuver sunder. Attacking a foe's stuff is what sunder is explicitly and only for, and a sunder can only leave a creature's objects damaged, broken, or destroyed. (That is, a creature can't, for example, employ the combat maneuver sunder yet opt to make a Strength check (DC = the object's break DC) instead of dealing damage.) (Also, by the way, a creature can employ the combat maneuver sunder when it could have made an attack instead of the attack. That's a thing.)
Also, against unattended objects, a creature can make normal attacks (even benefiting from some feats like Power Attack). (Such an object's AC is 10 −5 because of its Dex 0 −2 because it's usually inanimate = 3 +/− the objects's AC modifier due to size.) This will deal the object damage, and an ogre, for example, will with a perhaps just a lone mighty thwack render broken or destroyed a simple wooden door (hardness 5, hp 10) or, maybe, even a cheap shack's wall.