There is a very old and very polarizing debate in the world of photography: primes versus zooms.
Prime lenses are lenses which cannot zoom, whereas zoom lenses can. Inherently, it seems that zoom lenses are better, right? Well, not for everyone.
Reasons to Buy a Prime
A prime lens will, by definition, be smaller, lighter, and less expensive than a counterpart zoom (with equal aperture size and optical performance). And if the prime lens weighs as much as the zoom, it will, all else equal, have either a wider maximum aperture or better optics (or some additional feature, like tilt-shift lenses and macro lenses).
Take the Sigma 50mm f/1.4 A for example. It weighs about as much as a Nikon 24-70mm f/2.8 lens, but it's a prime lens. What does this mean? Well, the Sigma is two stops brighter than the zoom, and it is also sharper. It has less vignetting, it has almost no chromatic aberration whatsoever, and it costs about $1000 less than the 24-70mm. In almost every way, the 50mm f/1.4 beats the zoom, except that it doesn't zoom. Below is a photo from the 50mm.
Granted, the Nikon 24-70mm f/2.8 is no slouch. It's very sharp, and it has extremely fast focus. But, in terms of optical quality at 50mm, the Sigma wins, hands-down. I'm not a test-chart guy, but you can see that this is true by looking at comparisons around the web.
Before I go any further, it's important to know that I am biased on this subject, but only because of my own experience. I started out with the 18-55mm kit zoom, then I bought Nikon's 17-55mm f/2.8 zoom. Along the way, I also bought the Nikon 105mm f/2.8 VR macro lens, which showed me the image quality possible with a prime lens, especially along the edges of a photo. Not to mention that the macro feature of the lens let me take pictures that no current zoom allows. I soon realized that the 17-55 was not a particularly sharp lens, and I sold it. I bought the Nikon 24mm f/1.4 lens, and later the Sigma 50mm f/1.4 A.
Keeping track of that all? I now have three lenses, and all are primes: the Nikon 24mm f/1.4, the Sigma 50mm f/1.4 A, and the Nikon 105mm f/2.8 VR macro. At the time I write this, I'm using them all on the Nikon D7000. Below is a sharpness comparison of the upper-left sixteenth of an image shot at f/4 and 24mm with both the 17-55mm f/2.8 and the 24mm f/1.4. If I printed the final photo at a 16x24 size, these crops would be 4x6 inches of the frame. (Click on one of the images and scroll between the two for the most obvious results.)
For my photography, a zoom lens would be nice. However, it also would not be necessary, and, as the above crops show, it would not be good for my prints.
My style of composition is to find a great scene, set my camera on a tripod, and then find the composition that looks the best when I move around the camera. I default to my 24mm, and I only put on another lens when the "great scene" is in the distance. Also, I hike a lot. If I'm on the move, I can use just my 24mm lens and save a considerable amount of weight over a 14-24mm or 24-70mm zoom, and my photos will be a bit sharper, too.
If you have to work quickly, and if you cannot move from where you are, I recommend a zoom. For most other people, a prime lens is ideal.
If you're a parent photographing your child, a prime lens is probably best. Prime lenses have wider apertures, so you can make your child stand out from the background. Also, it takes just as long to walk forward with a prime lens as it does to change a zoom lens to the setting you want.
If you're a landscape photographer, primes are probably best for you too. First, a prime is typically lighter than a zoom (helpful on treks to distant landscapes). Also, if you're working on a tripod, you'll always be able to get the focal length you want by making a panorama. And if you're taking star shots, you need the widest aperture you can get.
Sports and wildlife shooters will want at least one prime lens, just because primes are the best way to get a super-long focal length at a decent aperture.
Street shooters probably don't have time to zoom how they'd want, plus a prime lens is smaller and more discreet.
Macro photographers have essentially no zoom lenses that even work as macro lenses.
Travel photographers may want a zoom for its convenience. Or, they may want a prime for its convenience. Prime lenses are quicker to use, and they work better in low-light areas, such at cathedrals and museums. The photo below was taken at f/1.4 and ISO 800, meaning that Nikon's most expensive zoom lenses would have required me to be at ISO 3200.
And, for some people, a prime lens helps with composition. For me, it does, but only when I work handheld. I typically take five or six photos of the best scenes I see, and I would rather bracket composition with a prime lens than bracket focal lengths with a zoom lens. For people who like to shoot quickly, and whose compositions often suffer as a result, a prime lens is a good remedy. It makes you less likely to blindly fill the frame with an interesting object, and instead to consider how to make the scene look its best.
However, for tripod-based work (or for slow and methodical handheld work), a prime lens does not help my composition. If I spend two minutes envisioning the scene ahead before I ultimately click the shutter, I should probably know the best way to photograph that scene. Sometimes, that lines up with my focal length, and sometimes it doesn't.
This, believe it or not, is another advantage of prime lenses. With a zoom, I would line up the scene perfectly to the focal length that looks best, and then I would take the picture framed exactly how I want.
With a prime lens, this composition still works, just with a different process. Instead of framing the shot with the correct focal length, I would change lenses to my prime lens that is longer than the correct focal length. Then, since I'd be on a tripod anyway, I would shoot a panorama photo. Every time I do this, I make sure that the entire perfectly-composed scene is well within the boundaries of the panorama. Just like in the photo below:
The next step is to assemble the panorama in my software, and then crop to the ideal composition. Now, I have a perfectly-composed photo with higher detail than a single shot would offer. The only issue: moving subjects. If you need to capture a photo in a single frame, this technique doesn't work too well.
In all, primes aren't perfect. But, in general, they are sharper, better in low light, and lighter than zooms. This makes them worthwhile for a wider audience of people than may be expected.
Reasons to Buy a Zoom
Zooms still have a key place in photography. They work best if you need to change perspective quickly, especially if you can't move much. For example, wedding photographers and close-range sports photographers will probably rely on a fast zoom for much of their work.
Also, a lot of zooms are faster to focus than prime lenses. Documentary photographers will probably carry a zoom or two, primarily for this quality.
Some real estate photographers will probably want a wide-angle zoom instead of a prime, because they can afford the weight and aperture loss of a zoom in exchange for the ability to quickly change perspective. Some landscape photographers are like this as well, although I personally find it worthwhile to spend the time creating a panorama if necessary. If you're a tripod-bound photographer but you are also on a tight schedule, a zoom lens could be your best option.
Another possibility is that a lens manufacturer doesn't have the focal length you want in anything other than a zoom. For example, there is no autofocus prime lens under 20mm for a Nikon camera, other than a fisheye and an outdated and overpriced 14mm f/2.8. Prior to 2014, there was no autofocus lens under 24mm aside from the zooms. I've heard that Canon is just as bad, too.
Lastly, if you can only have one lens (for monetary or space purposes) a zoom is probably the way to go. Primes work best when you have a tag-team; your widest prime will probably work best most of the time (at least for landscape photographers), but there are also situations which demand telephotos.
Myths of a Zoom Lens
I only have one myth, actually, so this section probably has a bad title. Regardless, a key myth of zoom lenses is that they let you avoid changing lenses for every shot. From my experience owning zooms, this simply isn't true. Unless you have just a single lens, you will always want to change lenses. It doesn't matter what your kit looks like, because you will always seem to have the wrong lens on your camera at any given time.
Recommendations for a budget kit
For Nikon users, I highly suggest the following kit: 20mm f/1.8, 35mm f/1.8 FX, and 60mm f/2.8 macro (or the 50mm f/1.8, if you don't like macro). This kit is light, inexpensive, high-quality, and versatile. Depending upon your style, you'll probably use the 20mm or 35mm the most, and you'll switch to the others as needed. There are no zooms which can capture images this sharp at this price, and especially none which have such wide apertures and also cover full-frame cameras. Also, for cropped sensors, this kit works very well. (Another, crop-only option is the Sigma 18-35mm f/1.8 zoom, paired with the 60mm f/2.8 macro. This should be just as good, although it won't work if you transition to full frame.)
This kit doesn't include any telephoto lenses, because your telephoto lineup will depend more on your budget than anything else. The 300mm f/4 lens does seem promising at a good price, but I haven't tried it.
Also, I'm not a Canon guy, so I can't offer a great recommendation if that's what you shoot. The new 16-35mm f/4 seems like a good lens, though, especially considering that Canon doesn't have a lot of wide-angle autofocus primes (although their manual focus tilt-shift lenses are supposed to be phenomenal).
Although I'm a prime person, I understand that most people are not. Zooms are more versatile, they generally focus faster (not the telephotos, though), and they often come with the camera you've already bought. But for me, a prime kit is my preference. The wide aperture sometimes comes in handy, and I love the better price/performance/weight tradeoff of primes.
Shoot whatever makes you happy, though, or whatever you already have. Some of my best photos, like the one below, were taken using Nikon's cheapest 18-55mm zoom, and I'm perfectly happy with that.