Per the spell description for Prestidigitation (page 267 5e PHB),
... Components: V,S ... You create a nonmagical trinket ... that can fit in your hand and that lasts until the end of your next turn.
Per Spellcasting Focus for Bard (page 53 5e PHB),
You can use a musical instrument (see chapter 5, "Equipment") as a spellcasting focus for your bard spells.
Per the Musical Instruments section of the Tools section (page 154 5e PHB),
Bagpipes, Drum, Dulcimer, Flute, Lute, Lyre, Horn, Pan Flute, Shawm, Viol
The Material section of Components (page 203 5e PHB) does not make any reference to conjured items not being used as a spellcasting focus.
Relevant Trinkets (page 160-161 5e PHB)
A tiny gnome-crafted music box that plays a song you dimly remember from your childhood ... A tiny silver bell without a clapper ... A metal can that has no opening but sounds as if it is filled with liquid, sand, spiders, or broken glass (your choice) ... A whistle made from gold-colored wood ... A set of bone pipes
From these trinkets, we can see that they are able to make music (music box, whistle, possibly bone pipes). The metal can could be used as a ganza (a type of rattle/percussion instrument).
The last bit of evidence that I can see for this being a "legal" ability is the specification in the spell description of Creation (page 229 5e PHB)
... Using any material created by this spell as another spell's material component causes that spell to fail.
This condition is not present in Prestidigitation or any other Bard-relevant spellcasting focus description.
Also, I realize that "ask your DM" is the obvious thing to do in situations like this, but I'm more interested in a literal interpretation of the rules.