Question As Written - Multiple Problems
My bat familiar (from the find familiar spell) has a Strength score of 2, and can therefore carry 7 lbs. (Per the "Lifting and Carrying" rules (PHB, p. 176), its carrying capacity is its Str score times 15, divided by 4 for being Tiny.)
I cast enlarge/reduce on a rock that weighs 56 lbs. to Reduce it, making it weigh 7 lbs.
So far, so good.
I have my familiar fly over a target. Then, as an action, I recast enlarge/reduce on the rock to Enlarge it,
No. Per the description of Enlarge / Reduce, it may target an "object that is neither worn nor carried". If your bat is carrying the rock, you may not cast Enlarge/Reduce on it.
cancelling out the first enlarge/reduce spell and enlarging the rock x8 to make it weigh 448 lbs.
Maybe not. When you begin to cast Enlarge / Reduce, that immediately ends your concentration on the previous spell and the rock returns to 56 pounds, per Xanathar's Guide to Everything, if that is in use.
As soon as you start casting a spell or using a special ability that requires concentration, your concentration on another effect ends instantly.
This optional rule from XGtE (p.5) was recently officialized in the Sage Advice Compendium 2.5 and more recently repeated in TCoE (p. 5)
"If you’re concentrating on a spell, your concentration on it ends immediately when you start casting another spell that requires concentration."
However, your new Enlarge/Reduce does not take effect until your spell is complete. Thus, there will be a slight delay between ending the concentration on the first spell and completing the second spell so that the rock grows to 448lb. That delay is the casting time of the spell, which is one action. Now, one action does not map out to any amount of time more specific than "less than about 6 seconds". (https://www.sageadvice.eu/2018/01/30/a-round-is-about-6-seconds/)
However, consider the very small window of time for you to do this. The range on your Enlarge/Reduce is only 30 feet. If you are standing just 20 feet away from your opponent, the rock is just 21.25 feet over them (with the maximun range being the 30 foot hypotenuse from you to the rock). To fall 21.25 feet it will take the rock just 1.15 seconds. Can you begin and end your spell in just over a second? DM's call, but possibly not.
This is too much for the familiar to hold and it drops the rock. Not as an attack, but because it has no choice.
It does have a choice. It can choose to hold on to the rock, and fall with it, rather than dropping it.
Question as Intended - Not an attack for either of you
Setting aside the Rube Goldberg (or, perhaps Heath Robinson, since you are Canadian) set up of your problem, what is it that you are really asking? Looking at your comments to others:
A familiar cannot attack. So it matters in the sense that if it is considered an attack, it will not work if the familiar is the one considering to have attacked. But if it is considered my attack then it would be legal
All that rule states is that IF there is an attack roll it is definitely an attack, it does not say that if there is no attack roll it is not an attack,
You clearly understand that a familiar by definition cannot attack, so dropping the rock cannot be the familiar's attack. I think you are trying to ask something like, "Can I make an attack if I haven't taken the attack action?", or "Can the effects of my spell constitute an attack if it is not actually a spell attack?" or "If my decisions cause damage to happen, is that an attack?"
If something like this is your question, then I would answer no, this is not an attack, for design principle reasons.
If you will allow me to clear away the apparently superfluous familiar and double-casting of Enlarge / Reduce, and posit this: Your enemy is standing under a precariously balanced rock at the base of a small cliff. With a single casting of Enlarge/Reduce, you could dislodge the rock, causing it to fall on them. Is this an attack?
No. Not because you don't make an attack roll specifically, but rather because you don't have an available attack action to make. Your action this turn was to cast a spell. You did not choose the attack action, therefore you cannot make an attack. You can certainly use your spell to set up a potentially damaging environmental situation, and you may end up hurting your opponent, but it was still not an attack. The result of this is that you are not allowed to make an attack roll, but the cause of it is that you did not spend an attack action to make something like an attack happen.
My perspective here is informed by my (honestly limited) understanding of action economy, agency, and game balance. As a PC, you receive a limited number of actions. However, when you take those actions, you have a great amount of control over them. If you choose to cast a spell on a legitimate target, most times the spell will work and do what you wanted it to do - you will enlarge that rock and cause it to tumble off the cliff. If instead you choose to attack (perhaps run up that cliff and push the rock down on your opponent), you have some control over that - you get to make the attack roll, perhaps drawing on ability bonuses, inspiration, etc., or even your familiar giving you Help. That attack is mostly about you and your rolls, and the opponent is just a passive AC score for you to hit.
In either of these cases, you have paid for your agency in the currency of actions. Allowing you to both cast a spell that significantly changes the world, and then have an ancillary result of that change be an attack, your attack, would be going beyond your allotted actions. Rather, your spell action has now created an environmental situation that could damage your opponent. But the resolution of that is now not under your control or influence - it is the DM's concern. Perhaps they will allow a saving throw, but they get to decide the ability score, assign the DC, and assign advantage or disadvantage - you are not an agent in the process. Or, perhaps for narrative reasons, they will simply decide that your opponent steps out of the way or gets flattened by the rock. The point is that this is the DM's action, not yours.
From this framework, let's consider your follow-up questions:
An alternative but similar thought for comparison: If I use an action to dismiss the familiar, does it drop the rock? (ending concentration on the enlarge/reduce spell and still dropping a 56 lbs. rock)
You can end concentration by choosing to without an action. You can dismiss the familiar as an action. For me, you have paid for that to be an attack by spending that dismissal action on it. I would be fine with, in that case, allowing you to time the dismissal to correspond with dropping the rock on the opponent, and allow you to make an attack roll - I might allow proficiency and I might allow you to use your spellcasting ability as the to-hit modifier. Even if I didn't allow an attack roll, I might have you make an Arcana or Animal Handling check contested by the Athletics or Acrobatics of the opponent. In any event, the important thing is that you would get to make the roll. You have earned that by spending the action; I would consider it an attack.
In either case, does this count as an attack from the familiar, or because I used an action to cause it, is it my attack?
It would never be an attack from the familiar, by definition. A familiar can certainly cause damaging things to happen, but these are not attacks, because their resolution is squarely under DM, not player, control.
If you used an action whose primary function was cause damage, that to me is an attack (even if it also results in the dismissal of your familiar). If you used an action whose principle result was the casting of a spell and subsequent change in the nature of reality (growing or shrinking a rock), that is not an attack, that is casting a spell.