I got some of my friends to play a very small game of D&D, with my being DM to see if they will like to play more often. Our quest consisted of PvP, hiding, going to the tavern to get drunk, and sexual harassment to the female NPCs. How do I get them to stop without sounding like a strict DM?
Agree on the rules of the table before the game starts.
Before you start a campaign with them, take them through the same page tool and talk to them about what they want to do at the table. If they want a game of PvP, hiding, and harassment, then enjoy that world!
In order to have rules, everyone must accept the rules. One thing I've noticed is that people tend to descend to silliness to mask nervousness, to upstage their friends, and to reduce necessary knowledge of the rules. Make sure your group accepts ignorance (not mocks it) and has a way of pausing and answering questions about the game.
Beyond that, players are not unlike puppies. Immediately reward actions you like, and do not reward actions you dislike. Note! Attention is a reward. If they spend lots of time in a tavern, but the group has decided that that's not what you all want, gloss over tavern scenes.
I like giving out bonuses (Ars Magica calls them confidence points) where you can use it for a +2 to an upcoming roll if you do something that I, as gm, approve of. These things include being ready for game, good roleplaying, or something that delights the table.
Beyond that, try to ritualize your gaming context. Agree with your group that if a certain large die or statue or something is on the table, you'll all try to act in character. Agree to take breaks, so you can joke. See if this works for you all.
From a communications point of view, the acting out you described is part of the Storming phase of small groups. Especially considering the age of your group, the best way to resolve storming is to talk about the actions and try to form consensus about what is and is not appropriate for that specific group. Part of this is to try to create a shared understanding of why certain things are inappropriate.
Give them some time.
We all have played those kind of sessions. It's typical to beginners players, or sometimes to players who are beginners in a specific game, or sometimes to players beginning a new campaign.
Your friends are just experimenting the freedom of the roleplaying format. In contrast with a videogame, they can do literally whatever they want (and characters are able to). They engage in this behaviour because it what is fun for them, and because they feel free.
Hopefully, in the next session continuing this way would be boring, and they will expect some plot. You'll have some mixed sessions with plot and silliness, but they will get increasingly interested in the story.
If that doesn't happen it's when I'd try more formal tools as the said same page tool and such.
Get the silly out of their system
Sounds like a bunch of new players wanting to go wild and let off steam; although aspects of what they're doing are more worrying - anyway...
What I'd suggest is taking a few games of more abstract and silly comedy RPGs so they can get the crazy out of their system and get interested in more long term games and aspects rather than just goofing around, which should - hopefully - get boring for them. As previously mentioned, you may need to find new players or determine if it's a single player leading them all astray.
Gamewise I'd suggest running a game or two of Paranoia - it's fun, brutal and actively encourages PvP and if they fall out of line, you can kill them in droves. Hopefully this will help them get to grips with games and then you can then move onto a more serious long term game.
New players are usually excited about the freedom in roleplaying games.
Players who are used to playing video games (which, for obvious reasons, are usually more streamlined and allow less freedom than tabletop roleplaying games) get quite excited about the fact that they can just tell you what they want to do, "and it happens". There are no real boundaries in tabletop roleplaying games, other than the system used.
This over-excitement and urge to "test the waters" and push the boundaries of the system should fade away naturally over time.
Try to excite them with interesting things to do, actual quests, without forcing them to do them. Introduce an NPC who asks them to do something for him. If they keep ignoring those opportunities and just keep fooling around, perhaps try casually asking them out-of-character what kind of quests they would like to experience. Try to keep these quests light-hearted and perhaps even comical, so that you can slowly move on towards more serious adventures - it sounds like it might be difficult for them to jump in straight into a serious, morbid quest.
Another way of limiting such behaviour is trying to introduce dice-rolls for certain actions (I'm not sure what sexual harassment should be treated as... diplomacy, or intimidation?) and let the system itself eliminate that behaviour. I've actually tried this before with one person who was "misbehaving" and it worked. As a DM you are allowed to push the boundaries of the system in such a way and treat certain actions as things requiring a dice roll. Obviously if they're really lucky they will continue being a menace, but perhaps the feeling that their action might not necessarily succeed might make them a bit more focused on the actual game. They don't sound like people who will be quoting pages from the core rulebook and not letting you do that, but don't overdo it.
If this juvenile behaviour continues, it might mean that they're sadly not the right people to play D&D with. From a psychological point of view either they try to impress others by behaving like "lads" instead of doing "boring stuff" like killing goblins and unlocking treasure chests, or perhaps that's just their way of saying that the whole concept of RPGs does not really interest them. It is hard sometimes to find the right people to play with, and as a DM you will get disappointed at least once, but that's a completely normal occurrence, especially when you invite new people to play with.
I feel the root of this problem is that there is a mismatch of expectations between you and the players. Sit down and talk to them out of character about what they want/expect from an RPG at this point. I feel like #2 listed below is the best options for the problem you are facing as you and your friends can find a game together that fits what you are trying to achieve.
1) Knuckle down on the rules. 4e implictly does not support PVP in any sense. Beyond that, try to use a pre-made module campaign. It will be railroady, but at the same time it will help them learn the system.
3) Change players. As most other have focused on, these people just might not be the right people to play rpgs with right now.
It seems to me that your core problem may be your players' lack of immersion in your game world. Basically, it sounds like your players don't really care what happens to their characters, or to the world around them. They don't see why they should go fight imaginary bandits or hunt for imaginary treasure, nor why they shouldn't grope the barmaids or moon the evil wizard, given that they won't gain or suffer any consequences for it.
In a word, they haven't really figured out what's supposed to make this "role-playing" stuff fun — and so they do the natural thing one does when given a boring toy to play with: try to break it and see if that makes it any more fun.
Obviously, you could try to engage your players by making their actions have consequences, but that's not very likely to go over well on its own. On one hand, those consequences will still only touch the characters, not the players, and so your players may well decide to just ignore any in-game consequences that don't actually prevent them from wreaking more havoc. And if you do force them to stop — say, by having the town guard arrive and drag the characters to jail — then you may find that you've just prevented your players from doing the only thing they currently find interesting in your game, and thus they may just decide that the game is boring and quit.
My recommendation, instead, would be to try the carrot instead of the stick: essentially, to try to lead the characters into the parts of the game you hope both you and they will find more interesting, rather than trying to beat them away from their current behavioral patterns.
One way to do that might be to have the action come to them: for example, instead of having the town guard come and arrest the characters, have their carousing and misbehavior be interrupted by a big monster attacking the tavern they're in. Don't worry too much about how or why the monster came to be there — trust me, your players probably won't, either. Just make sure that it's big, impressive, and something the players can defeat — but only if they band together and actually use their characters' skills.
I can't really stress the word "impressive" there too much. At this stage, you can't really effectively motivate your players by the threat the monster presents to their characters' survival, so you'll have to play on their sense of "Wow, it'll be really cool if we can beat that thing!"
Once the players have defeated the monster and gained the appropriate rewards, then it's — hopefully — time to start figuring out answers to questions like "Why did it attack us?", "Where did it come from?" and, above all, "Where can we find more?"
Another, even more direct option for motivating your players to engage in the game could simply be to bribe them with an out-of-game reward. So the king has offered a gold reward to anyone who defeats the evil sorcerer? You can mention to your players that you'll also buy them pizza (or whatever works best for your players) to celebrate their victory if they can do it. Just make sure to tell your players that it'll take at least a few sessions before they'll even stand a chance; by the time they finally do it, they'll hopefully be ready for the next challenge without even needing any extra rewards.
Of course, it's always possible that none of this may work, and that your players simply aren't interested in the kind of game you want to run. If that's the case, the only real choices are to change the game (Paranoia has been recommended several times, or you might want to consider just getting a deck of Munchkin cards), change the players or change your expectations.
Actually, that last option may well be worth considering, anyway: just because your players would rather engage in wacky Chaotic Stupid[warning: TV Tropes] hijinks (regardless of their nominal alignment) than set out on your carefully planned quest to rid the world of an ancient evil, it doesn't necessarily mean you can't find a workable and, above all, fun middle ground. Besides, even if the characters your players want to play are the masters of Doing Shenanigans And Getting Away With It, just think of all the juicy plot hooks their antics can create once the players start identifying with their characters even a little. That barmaid they groped in the tavern two sessions ago? Well, it turns out that her boyfriend is a high-level adventurer, and he's not amused...
Set an appropriate play guideline, then step back.
I've mentioned my first Shadowrun campaign here before; and I had this exact problem, with people constantly wanting to go and get drunk/do 2060's drugs/raid a brothel, and the way that I got through it all was just saying "Okay, you do that. What's next?".
Basically, if players are doing something that makes you uncomfortable, either tell them outright to stop, or just gloss over the more explicit encounters. For bonus points, you can punish inebriated characters by having sudden combat/whatnot as soon as they get their dose/binge drinking and watch them flail around with reduced reaction times and whatnot. Alternatively, they are caught by an importnat ally when they're totally out of sorts and have to save face once they wind up vomiting/being caught in the act/thinking they're actually a space zebra intending to process them into dandelion feed. Just as in real life, the sort of antisocial behavior you don't want players' characters engaging in can have very real in-universe consequences.
My one caution would be to remain subtle about this-don't instantly kill them for getting drunk (even if real life sets a precedent for that both through alcohol poisoning and the wonders of canals), or getting a little too wild (in game, of course; I have a rule that if a player brings alcohol to a session it's their last game, if they're even allowed to stay for the session, which they usually aren't).
For less puerile behavior, such as PvP or hiding, sometimes it's okay to permit, especially if it fits the characters-I've had a character knock an ally unconscious because he disagreed with a decision to execute someone, and nobody complained, because it made sense for my character to do it (since it involved our boss wanting us to kill a hostage, and my character was against it, he had moral reasons to do so). The catch there is that it's in-character, and I gave plenty of warning beforehand that it would lead to a fight. Hiding is often an expression of just wanting to do their own thing-if they want to go and do things they can, but life goes on around them-if they hide from the king's summons to go on a quest they may face exile for it, or if they're not around when the wagon train leaves, they'll get left behind.
Sexual harassment, on the other hand, is an iffy area. Very few people would be comfortable with it in real life, but a lot of people don't find anything wrong with their characters doing it. If it makes you really uncomfortable, put your foot down and explicitly forbid this. If your players won't listen, they're not appreciating you enough as a GM. If you're not necessarily uncomfortable with it, but would prefer that they focus on the plot, you need to entirely devalue the behavior so they won't do it again, especially by adding consequences. One way I avoided this while my group was puerile was to simply use primarily male NPC's, so that they didn't really have an opportunity to misbehave, though if you make it clear ahead of time that the NPC is powerful, or has powerful allies, feel free to bring in the firepower on the unruly group. In addition, if players violate the law, they may become persona non grata and be forced to look elsewhere; it may take a couple towns, but if adventures start to dry up because the band's reputation is so low even the criminals won't work with them they'll either shape up or break up.
Eventually, I've found that players will mellow out; puerile impulses aside I've found that my group, while still mostly the same individuals that caused a lot of angst me for the first few months, has become a lot more mellow and really fallen more into the traditional gameplay style that I try to encourage rather than running around on drug-addled brothel crawls.
The game can mature with them. Introduce real consequences for their behavior. Send a local Sherrif to them first. Have him explain that their behavior cannot be tolerated but since they are new to the area it will be overlooked this time.
If they continue, send someone with more authority accompanied by guards. Throw them in jail for a few nights.
If it continues, send a Paladin with his retinue.
You could create an entire campaign out of this. The players behaving badly, becoming outlaws, and constantly on the run from the law. Or, if they get their acts together they could one day be hired to enforce the very laws they have broken as a form of restitution to the community.