> I'm not really finding this quirk to be any real fun to roleplay, it's more of an annoyance than anything, because it's not really interacting with the story/character interactions.

And this is the real heart of the issue. What is happening here is a fundamental lack of proper integration of the quirk, not the quirk itself.

Quirks, disadvantages, etc. all generally intended to open up roleplaying opportunities for the character. Not just in the abstract but often in real mechanical terms as well.

As others have noted, this quirk needs a purpose:

 - You should be able to distract NPCs by fiddling with their things while you companions do something they couldn't do with the NPC watching. 
 - With respect to the comments by @TobiasKenzler and @SteveJessop, you should be more adept at finding/dismantling traps. ("XYZs aren't normally put together like that! I know. I have inspected them... *ahem*... closely.")
 - You should be able to take apart locks and figure out what bars are loose in the local jail window. 
 - Ancient puzzles with complex interlocking pieces that don't even look like puzzles should be child's play even if your actual INT score is nothing special (you rain man, you! =P ). Just don't unwittingly solve any puzzle boxes leading to hell. ;-) 
 - You should have the ability to extrapolate near instantly what an unfathomable war machines on the brink of destroying world actually does and have a good idea of how you might stop it.

In short, you are not a person who takes things apart -- you are a mechanically-inclined engineer. And breaking stuff is only half the person you are. Eventually you have to put things back together. Your other half is the guy or gal in every movie that, when the hero is completely stumped by the villain, you have "an idea" (at least if it involves anything McGuyver-like.)

You, my friend, are an indispensable asset to the group. And with your great luck who is to say that you might not "absentmindedly" break something which, on further inspection, reveals something interesting (a hidden object in a clockwork, for example, or a lever mechanism for a hidden passage).

# Talk To Your GM

> The GM keeps an eye on me and often when I (as the player) am not doing anything, will ask me what I'm dismantling.

I dislike criticizing anyone's technique but it feels like the GM is expecting a little too much imagination from you in this case and is failing to help make opportunities for your character to put your quirk to use in a way that will satisfy everyone.

I would have a talk with him or her and brainstorm some ideas about how the quirk could be useful (as mentioned) and politely ask that they make an effort to integrate some of those ideas into future sessions.

If they decline or the situation remains the same, well, that's that. You can probably drop the quirk. But hopefully, if things go well, you all may be able to have a little more fun.