Cascading Style Sheets (CSS) offer a lot of control over your website and if you use an external CSS (recommended for several reasons) you can quickly make uniform changes to your entire site by changing one file. Here are some great resources for learning CSS and building skills.
This article gives a nice broad overview of how to think about CSS and avoid making some common mistakes early on. It is not so much a how-to article, but it is important for planning before you dive into converting / building your site using CSS.
This collection of tutorials is one of the best around. It breaks topics down into Basic, and advanced and the nice part is you get to edit the CSS right there and see the results immediately. The one drawback is that it uses "inline css" which means the code is on the page with the content, but that is just to allow you to play. The code looks the same when it is put into an external css.
A collection of website examples that use the same content, but dramatically change the appearance just by using different stylesheets. Pretty incredible just for inspiration. Don't be intimidated though, many of these sites were developed by true masters with backgrounds in graphic design.
Very impressive examples with explanations about how it was done, complete with a display of the source code and css used to create the example. Some of it is a bit bleeding edge and doesn't always work in all browsers, but most of the time it is pretty well tested.