Some clear cases, some not
There are several possible ways to resolve this. There is a clear, direct answer: Any spell you cast that is on your class spell list counts as a spell of that class. This has the advantages of being simple, being found in multiple places all supporting each other, and being a response directed at the question itself. It has the disadvantage of being largely based on (now unofficial) tweets from Jeremy Crawford. I choose not to follow that path for reasons other than it being unofficial, per se. First, this is a complex question and I don't think all the nuances and caveats lend themselves to explanation within a tweet character limit. Second, I am uncomfortable with one logical conclusion that follows - an individual spell you cast that is on two different spell lists of yours is then simultaneously a spell of both of those classes. And third, it contradicts what is an official Sage Advice ruling (see below). I think unofficial answers (tweets) have a valuable role in clarifying things when official answers aren't forthcoming, but I hesitate to accept them when they contradict what is official.
Another way to resolve this is to look at what is actually within the the text of official sources such as the PHB and Sage Advice. Unfortunately, this approach relies on interpreting passages that are not explicitly directed at the question itself ('what makes a spell of a certain class?'), and interpretations can vary.
In particular, the passage that I find most helpful is in the rules on Multiclassing, which are themselves optional. I understand the arguments of those who would apply this passage narrowly ('this section about spells is only for PCs who multiclass, and only if you agree to that option'), but I choose to interpret it broadly ('this optional section illustrates a larger point about how spells work that is true across the core system'). My choice is undeniably biased toward utility - I find a speculative principle that I can actually use better than the interpretation that there is no clear answer to the larger question. The latter might be true, but it is not particularly helpful, and now, six years after publication of the PHB, after numerous errata and Sage Advice clarifications, and after two large presentations of new rules (XGtE and TCoE), at least one of which deliberately clarified questions about timing, sleeping, AoE on a grid, etc., it seems likely that WoTC either considers this issue settled or has no interest in addressing it.
Here is the passage in full from the Multiclassing Rules:
Your capacity for spellcasting depends partly on your combined levels in all your spellcasting classes and partly on your individual levels in those classes. Once you have the Spellcasting feature from more than one class, use the rules below. If you multiclass but have the Spellcasting feature from only one class, you follow the rules as described in that class.
Spells Known and Prepared. You determine what spells you know and can prepare for each class individually, as if you were a single-classed member of that class. If you are a ranger 4/wizard 3, for example, you know three 1st-level ranger spells based on your levels in the ranger class. As 3rd-level wizard, you know three wizard cantrips, and your spellbook contains ten [sic] wizard spells, two of which (the two you gained when you reached 3rd level as a wizard) can be 2nd-level spells. If your Intelligence is 16, you can prepare six wizard spells from your spellbook.
Each spell you know and prepare is associated with one of your classes, and you use the spellcasting ability of that class when you cast the spell. Similarly, a spellcasting focus, such as a holy symbol, can be used only for the
spells from the class associated with that focus.
Objection: This section is optional, and specifically says it is to be used only "once you have the Spellcasting feature from more than one class".
Response: Yes, but it also distinguishes between rules (like those about spells known and prepared) that are true "as if you were a single-classed member of that class" and rules (like those about total spell slots) that apply only when you have multiple Spellcasting classes. I don't think it is unreasonable to believe that the principles it clarifies 'as if you were single classed' also apply to when you actually are single-classed. As for it being optional, many of the times the greater question of 'how do you know when a spell is a class spell' actually comes up is when attempting to adjudicate other optional features, like Feats. I will take guidance where I can find it.
If we accept that the passage is establishing a general principle, how do we rule?
In what follows I am assuming that by "each spell you know and prepare is associated with one of your classes" means that:
If a feature that allows you to know a spell can be tied to a specific class, that spell counts as being of that class.
Indeed, what else could it mean that would not be redundant or a tautology? What other kind of association is worth mentioning?
Spells you know on your list because of your class's Spellcasting ability
Class spells are spells off the class spell list that you know because you are that class (you have the Spellcasting ability by virtue of you being in that class).
For example;
"Sorcerer: Spellcasting" (PHB 101; emphasis mine)
Spell Slots The Sorcerer table shows how many spell slots you have to cast your spells of 1st level and higher. To cast one of these sorcerer spells, you must expend a slot of the spell’s level or higher...
Spells Known of 1st Level and Higher The Spells Known column of the Sorcerer table shows when you learn more sorcerer spells of your choice...Additionally, when you gain a level in this class, you can choose one of the sorcerer spells you know and replace it with another spell from the sorcerer spell list, which also must be of a level for which you have spell slots.
Notice the subtle distinction between "sorcerer spells" and a "spell from the sorcerer spell list". A spell becomes a sorcerer spell for you when you know it. Just being on the sorcerer spell list is not enough; it does not become a sorcerer spell for you until you know it.
Spells you gain access to through a class feature but not through the Spellcasting feature of that class
If something besides your class Spellcasting feature grants you the ability to cast a spell, typically it will tell you what class of spell it is.
For example, the Cleric's Domain feature (PHB 58):
If you have a domain spell that doesn’t appear on the cleric spell list, the spell is nonetheless a cleric spell for you.
Here, although the spell does not appear on the cleric spell list, and in fact appears on another spell list, it is still a cleric spell because the feature granting it to you says it is (I argue that the underlying principle is that it is a cleric spell because knowledge of the spell is granted through a cleric class feature, but grant that it does not say that explicitly). Thus, if a specific class feature gives you a spell, it would be a spell of that class if the feature said it was (see also the Druid's Circle spells and the Bard's Magical Secrets).
Sometimes a class feature gives you a spell but explicitly says it is not of that class:
The Spells Known column of the Eldritch Knight Spellcasting table shows when you learn more wizard spells of 1st level or higher.
In this case, the spells you know as an Eldritch Knight are wizard spells, not fighter spells.
Spells you know because of general (non-racial) Feats
You might also know a spell through the optional rule of feats, which are not a class feature. However, these will still typically imply what kind of spell you are getting.
"Feats: Magic Initiate" (PHB 168)
Choose a class: bard, cleric, druid, sorcerer, warlock, or wizard. You learn two cantrips of your choice from that class’s spell list. In addition, choose one 1st-level spell from that same list. You learn that spell and can cast it at its lowest level. Once you cast it, you must finish a long rest before you can cast it again. Your spellcasting ability for these spells depends on the class you chose: Charisma for bard, sorcerer, or warlock; Wisdom for cleric or druid; or Intelligence for wizard.
This feat stops short of saying the spells you learn are spells of the class whose list you chose. Fortunately there is a Sage Advice Compendium that more explicitly addresses Magic Initiate:
If you have spell slots, can you use them to cast the 1st-level spell you learn with the Magic Initiate feat? Yes, but only if the class you pick for the feat is one of your classes. For example, if you pick sorcerer and you are a sorcerer, the Spellcasting feature for that class tells you that you can use your spell slots to cast the sorcerer spells you know, so you can use your spell slots to cast the 1st-level sorcerer spell you learn from Magic Initiate. Similarly, if you are a wizard and pick that class for the feat, you learn a 1st-level wizard spell, which you could add to your spellbook and subsequently prepare.
In short, you must follow your character’s normal spellcasting rules, which determine whether you can expend spell slots on the 1st-level spell you learn from Magic Initiate.
Here, we are told explicitly that a spell counts as a sorcerer spell not because you knew it through a sorcerer class feature, but because you deliberately learned it from the sorcerer spell class list. Had the same spell also been on another class list as well as that of the sorcerer, and you learned it through Magic Initiate as the other class, it would not count as a sorcerer spell. This is the most explicit contradiction of the Crawford tweets.
See also the "Ritual Caster" feat, and this question about sorcerers triggering wild magic surges when using the Magic Initiate feat.
Spells you know through a racial feat
Some racial feats tell you the class of the spells they are granting.
"Wood Elf Magic" in XGtE
You learn the magic of the primeval woods, which are revered and
protected by your people. You learn one druid cantrip of your choice.
You also learn the longstrider and pass without trace spells, each of
which you can cast once without expending a spell slot. You regain the
ability to cast these two spells in this way when you finish a long
rest. Wisdom is your spellcasting ability for all three spells.
Others do not.
"Drow High Magic" in XGtE
You learn more of the magic typical of dark elves. You learn the
detect magic spell and can cast it at will, without expending a spell
slot. You also learn levitate and dispel magic, each of which you can
cast once without expending a spell slot. You regain the ability to
cast those two spells in this way when you finish a long rest.
Charisma is your spellcasting ability for all three spells.
When the Feat tells you the class explicitly, those spells count as a spell of that class. When it does not, the spell you know is not associated with a class.
See also this question about the spells granted by racial feats.
Spells you have access to through magic items
Magic items that grant you the ability to cast spells do not typically tell you what kind of spells they are, although it is clear that it is you who are casting the spell, not the item, which is important if you are interacting with a feature that says 'when you cast a spell' or 'when you cast a {class} spell'.
"Activating an Item; Spells" (DMG 141)
Some magic items allow the user to cast a spell from the item...A magic item, such as certain staffs, may require you to use your own spellcasting ability when you cast a spell from the item. If you have more than one spellcasting ability, you choose which one to use with the item.
Magic items also typically say "you cast", "you can cast", etc in their descriptions. However, they do not usually say that you 'learn' or 'know' a spell, which is in clear contrast to the language used with feats.
Based on this, I would say that if the magic item allows you to cast a spell that you do not know, it is not a class spell. However, if the spell it is allowing you to cast is one that you already know, then you are casting a spell that you know from a specific class, and thus it is a spell of that class.
See also this question about sorcerers triggering wild magic surges when casting putatively sorcerer spells from magic items.
What if you know a spell through more than one means?
If you are multiclassed and have access to a spell from another class that is also on the sorcerer spell list, for example, you still can't cast it as a sorcerer unless you know it through being a sorcerer ("Each spell you know and prepare is associated with one of your classes").
But if you know the spell both as a sorcerer and as another class, is it a sorcerer spell? I would say that you make that decision at the moment you cast it, based on the multiclassing rules: "you use the spellcasting ability of that class when you cast the spell".
I realize that 'your spellcasting ability' (ie, Charisma) is not 'your ability to use Spellcasting' (ie, a class feature). But the precedent of, when a spell could be sourced from multiple classes, choosing the spellcasting ability at the time of casting seems a clear enough proxy for the class of the spell to me.
(See also this question)
In conclusion, I would say that
Any spell on a class spell list that you know from being that class is a class spell, if you choose to cast it as such.
If a feature allows you to know a spell and explicitly references a class, the spell can also be of that class, if you are of that class and you choose to cast it as such.