The two games are very different, despite sharing the same underpinnings. I know plenty of people who played previous editions who don't like 4e, and I know plenty of people who played previous editions who loved 4e. Hopefully we can navigate these rocky, contentious waters without flames.
First off, 4e is fairly light on non-combat rules. This doesn't mean that 4e games are all about combat; it means that the rules assume that a lot of the roleplaying activities that were codified in 3e will be done via freeform roleplay. For example, there aren't any crafting rules for anything other than magic items. There also aren't any general professional skills, and there aren't any NPC classes. If you prefer to have rules for that sort of thing, 3e will be a better choice for you.
Second, 4e uses a power-based design methodology. Classes can be thought of as collections of powers; the differences between classes are defined by the different power choices they have. This makes for a very modular and flexible system. Some people find that it makes the classes overly homogeneous; some people like it.
Third, every 4e class uses powers. That's implied by my second point but it's worth mentioning specifically. A character begins with two at-will powers, that he can use whenever he wants; one encounter power, that can be used once per fight; and one daily power, that can be used once per day. Even martial characters, such as warriors, use this paradigm -- although their "powers" might be better thought of as something akin to a martial arts kata. This was intended to make combat more interesting for (say) fighters, in comparison to the earlier model where fighters just tended to hit things over and over again. If you didn't mind that model, this change may be unnecessary for your play style.
Fourth, 4e leans more heavily on the battlemap. My impression is that the large number of movement-oriented powers both make the battlemap more important and make combat more fluid, but that's definitely a subjective opinion on my part: consider it something to think about if you try 4e rather than a definite fact.
Fifth, 4e introduces the concepts of roles. Roles are a way of classifying classes by what they tend to do in combat. You've got leaders, who heal. There are more of them than just the cleric; for example, the bard is also a leader. You've got defenders, who control the battlefield by encouraging enemies to focus their attacks on them. The fighter is a defender; so is the paladin. You've got controllers, who are somewhat difficult to define, but you can think of them as the classes that affect the flow of a fight: they can hamper enemies, reshape terrain, and so on. The wizard is a classic controller. And, finally, you've got strikers, who purely focus on doing damage. The ranger and the sorcerer are strikers. Every class is primarily one role, but every class has the ability to take on aspects of another role, depending on what the player wants to do.
Sixth, multiclassing is more limited than in 3e. You can multiclass in a couple of ways, but you don't get the same ability to take six or seven classes/prestige classes during the course of your career. 4e classes are fairly flexible, but you don't get the same complete freedom you would with 3e multiclassing.
Seventh, the scope and feel of 4e can be somewhat more epic; or, to put it differently, more broad. The highest level is level 30, and that's very epic play, with abilities that allow characters to come back from the dead. Even at level 1, your characters are significantly more durable than third edition characters, and they'll be able to pull off some really wild things.
I think that hits most of the major differences. It's good to remember that it's still a heroic fantasy game in which characters fight monsters. It still uses a 20 sided die. Also, if you want to try it out, WotC has a free Quick Start kit available.
Alternatively, the new Essentials Red Box will be out in a few weeks; at $20 US, it might be a good way to take a peek at the game and decide if you like it. The Essentials core books will present a bunch of new class variants that change some of the things above: e.g., fighters won't have the same power structure I mentioned. So that might be a better entrance point.