Question As Written - Multiple Problems
My bat familiar (from the find familiar spell) has a Strength score of 2, and can therefore carry 7 lbs. (Per the "Lifting and Carrying" rules (PHB, p. 176), its carrying capacity is its Str score times 15, divided by 4 for being Tiny.)
I cast enlarge/reduce on a rock that weighs 56 lbs. to Reduce it, making it weigh 7 lbs.
So far, so good.
I have my familiar fly over a target. Then, as an action, I recast enlarge/reduce on the rock to Enlarge it,
No. Per the description of Enlarge / Reduce, the spell targets an "object that is neither worn nor carried". If your bat is carrying it, you cannot cast Enlarge/Reduce on the rock.
cancelling out the first enlarge/reduce spell and enlarging the rock x8 to make it weigh 448 lbs.
Maybe not. When you begin to cast Enlarge, that immediately ends your concentration on the previous spell and the rock returns to 56 pounds:
As soon as you start casting a spell or using a special ability that requires concentration, your concentration on another effect ends instantly.
This optional rule from XGtE (p.5) was officialized in the Sage Advice Compendium 2.5 and more recently repeated in TCoE (p. 5)
"If you’re concentrating on a spell, your concentration on it ends immediately when you start casting another spell that requires concentration."
Although your previous Reduce ends immediately, your new cast of Enlarge does not take effect until the spell is complete. Thus, there will be a slight delay between ending the concentration on the first spell and completing the second spell so that the rock grows to 448lb. That delay is the casting time of the spell, which is one action.
One action does not map out to any amount of time more specific than "less than about 6 seconds", but we can estimate the actual amount of time using real world falling speeds. The range on your Enlarge is only 30 feet. With the maximum range being the 30 foot hypotenuse from you to the rock, you are standing 20 feet away from your opponent, and the rock is just 21.25 feet over them. To fall 21.25 feet will take the rock just 1.15 seconds. Can you begin and end your spell in just over a second? This is your DM's call, but possibly not.
The optional falling rules from XGtE (p.77) are explicitly for creatures, but in the absence of anything else to guide you could be applied to objects, meaning the rock actually falls the first 500 feet instantly. If this is the case you have no time to cast another spell on it as it falls.
This is too much for the familiar to hold and it drops the rock. Not as an attack, but because it has no choice.
It does have a choice. It can choose to hold on to the rock, and fall with it, rather than dropping it.
Question as Intended - What defines an attack?
Setting aside the Rube Goldberg (or, perhaps Heath Robinson, since you are Canadian) set up of your problem, what is it that you are really asking? Looking at your comments to others:
A familiar cannot attack. So it matters in the sense that if it is considered an attack, it will not work if the familiar is the one considering to have attacked. But if it is considered my attack then it would be legal
and
All that rule states is that IF there is an attack roll it is definitely an attack, it does not say that if there is no attack roll it is not an attack,
You clearly understand that a familiar by definition cannot attack, so dropping the rock cannot be the familiar's attack. I think you are trying to ask something like, "Can I make an attack if I haven't taken the Attack action?", or "Can the effects of my spell constitute an attack if it is not actually a spell attack?" or "If my decisions cause damage to happen, is that inherently an attack?"
If something like this is your question, then the answer is Did your DM call for an attack roll? The accepted standard of whether something is an attack is, 'Did it call for an attack roll, or is it described as an attack?' If the answer to either of these is yes, it was an attack.
Let's clear away the superfluous familiar and the double-casting of Enlarge / Reduce, and posit this: Your enemy is standing under a precariously balanced rock at the base of a small cliff. With a single casting of Enlarge, you could dislodge the rock, causing it to potentially fall on them. Is this an attack?
Again, the question devolves to, Did your DM call for an attack roll?
You did not choose the Attack action; your action this turn was to Cast a Spell. The spell you cast (Enlarge) by its nature does not call for an attack. You certainly used your spell to set up a potentially damaging environmental situation, and you may end up hurting your opponent, but whether or not it is an attack hinges on whether your DM had you make an attack roll.
At my table, in this particular case, I would allow the target a saving throw, decide the ability score targeted, assign the DC, and assign advantage or disadvantage - you would not be an agent in the process once your spell was cast. Or, perhaps for narrative reasons, I might simply decide that your opponent steps out of the way or gets flattened by the rock. In any event, this would not be an attack. I would be loathe to give you the attack, because your action was to Cast a Spell, and nothing in the spell description specified that it made an attack. I would want you to have all the stated benefits of the spell that you cast, but I would prefer to run any environmental consequences as my purview of 'Rule 3: The DM narrates the results'.
However, another DM with a different playstyle might allow you to make some sort of attack roll against your opponent's AC - in that case, it would most definitely be an attack.
From this framework, let's consider your follow-up questions:
An alternative but similar thought for comparison: If I use an action to dismiss the familiar, does it drop the rock? (ending concentration on the enlarge/reduce spell and still dropping a 56 lbs. rock)
You can end Concentration simply by choosing to, without an action. You can dismiss the familiar as an action. I would be fine with, in either case, allowing you to time the end/dismissal to correspond with dropping the rock on the opponent, and would allow you to make an attack roll - I might allow you to include your proficiency bonus on that roll, and I might allow you to use your spellcasting ability as the to-hit modifier. Since you were making an attack roll, this would be an attack.
Another DM might might treat this as I did the Enlarged rock, and simply run a saving throw for the target, without your involvement. This would not be an attack.
A tricky case would be if the DM had you make an Arcana or Animal Handling check contested by the Athletics or Acrobatics of the opponent to decide whether or not the rock hit. In this case, it would be an attack if the DM said it was. This is the part of the rule that says something is an attack if it says it is, like a Grapple. In this case, since the DM is defining the action itself, they are free to say that it is an attack.
In either case, does this count as an attack from the familiar, or because I used an action to cause it, is it my attack?
It would never be an attack from the familiar, by definition. A familiar can certainly cause damaging things to happen, but these are not attacks. Using your action, by itself, would not make it an attack, but it would be an attack if you made an attack roll, or if your DM declared that it was an attack.
