Geek Girls has a pretty good explanation of the what ("Sometimes when you install a program or create a data file, the file ends up chopped up into chunks and stored in multiple locations on the disk. This is called fragmentation"), and a partial expanation of the why:
But there's one site that goes more in-depth, and explains the processes behind it all. It explains why Windows disks tend to be fragmented much more often, and more quickly, and why Linux doesn't have quite so many problems. It's a great write-up, concise, clean, and even has a few illustrations thrown in to make it easier for those who are more visually attuned to learning.
Why should you bother with the housework? A couple of reasons. First, disks are hard working, mechanical devices and, like all mechanical devices, prone to failure. A little preventative maintenance can warn you of potential problems and fix minor glitches before they can do damage to your data.
Second, the way files are organised on your drive has a perceptible impact on the performance of your computer. If your files are stored neatly, end-to-end, without fragmentation, reading and writing to the disk is speedier.
The site explains the main difference between how Linux and Windows palce files on their disks, and why Linux suffers much less than Windows from fragmentation:
OneAndOneIs2 - Why doesn't Linux need defragmenting?
Windows tries to put all files as close to the start of the hard drive as it can, thus it constantly fragments files when they grow larger and there's no free space available.
Linux scatters files all over the disk so there's plenty of free space if the file's size changes. It also re-arranges files on-the-fly, since it has plenty of empty space to shuffle around. Defragging a Windows filesystem is a more intensive process and not really practical to run during normal use.