Spell scrolls and spell books are written differently
Each wizard's spellbook is unique, written in their own, personal notation.
Copying that spell into your spellbook involves reproducing the basic form of the spell, then deciphering the unique system of notation used by the wizard who wrote it. You must practice the spell until you understand the sounds or gestures required, then transcribe it into your spellbook using your own notation. (PHB)
Conversely, the contents of a spell scroll can be read by anyone with the ability to cast that spell, including people who don't normally need to read or study magic. The same scroll could potentially be used by casters of extremely different backgrounds.
A spell scroll bears the words of a single spell, written in a mystical cipher. If the spell is on your class’s spell list, you can read the scroll and cast its spell. (BR p.200)
Another piece of evidence that spell scrolls are more than just a written copy of a spell or a page taken out of an existing spell book is that, while a wizard can copy from them into their spellbook, there's a risk that the scroll will be destroyed without being copied:
When a spell is copied from a spell scroll, the copier must succeed on an Intelligence (Arcana) check with a DC equal to 10 + the spell's level. If the check succeeds, the spell is successfully copied. Whether the check succeeds or fails, the spell scroll is destroyed. (BR p.200)
Additionally, we can see from the downtime rules in Xanathar's Guide to Everything that creating a spell scroll takes longer and is far more costly than copying a spell from one spell book to another. If you're creating a backup copy of your spellbook, copying spells is rather inexpensive:
You can copy a spell from your own spellbook into another book—for example, if you want to make a backup copy of your spellbook. This is just like copying a new spell into your spellbook, but faster and easier, since you understand your own notation and already know how to cast the spell. You need spend only 1 hour and 10 gp for each level of the copied spell. (PHB)
The costs for scribing a new spell scroll are far higher, but can be done by anyone proficient in Arcana, not just wizards. In addition to requiring the material components for the spell, a character must spend anywhere from 1 workday and 25 gp for a first level spell to 48 workweeks and 250,000 gp for a ninth level spell. (XGtE). Clearly, there's something fundamentally different between spell scrolls and spell books.
Spell scrolls do more than just convey knowledge of a spell
Spell Scrolls are more than just information - they are magic items that contain not only the knowledge of how to cast a spell, but essentially contain the spell itself. Casting from a spell scroll does not require any (potentially costly) material components and doesn't require the user to expend any spell slots:
If the spell is on your class’s spell list, you can read the scroll and cast its spell without providing any material components. Otherwise, the scroll is unintelligible. Casting the spell by reading the scroll requires the spell’s normal casting time. (BR p.200)
Some magic items allow the user to cast a spell from the item. The spell is cast at the lowest possible spell level, doesn't expend any of the user's spell slots, and requires no components, unless the item's description says otherwise. (DMG p.141)
So, even if the notation used between spell scrolls and spell books is the same, it wouldn't mean that someone could cast a spell they didn't know from a spell book.