While this has been explained to me and written about in many different variations, I will attempt to convey it in the most universal method possible. Just to clarify-I said universal, which in some cases may mean overgeneralized, and for this I apologize but I feel the benefits of generally understanding the topic outweigh the issues of oversimplification it creates.
The building envelope is akin to a balloon. Under certain conditions it is almost pressurized (hopefully not under pressure-but close) and this is normal. A good building envelope can almost reach a pressurized state if it is not leaky. Now, it doesn’t necessarily pop like a balloon, but air goes in and air goes out. For efficiency purposes we want to control this air in and air out. First, you must seal any leaking areas that are not related to doors and windows, and in an old house you’d be surprised to find out jut how many holes a building can have in it! Good tuck-pointing can provide quite a bit of air sealing. Also, pipe and wire protrusions are notorious wind tunnels.
Start in the basement and work your way up. Check the foundation for cracks and open mortar joints (for those of us who live in quarried-rubble or concrete block) for leaks and penetrations. Check around locations where pipes, wires, and conduit enter/exit the building. Then look at the location where the sill board rests on the foundation. For those of you in homes with all-masonry exterior walls, check the locations where the joists pocket into the masonry and where the masonry rests on lintels/headers that support weight over basement windows. Note: in all masonry constructed houses, you may want to avoid the insulating books recommendations on rim joist insulating. In an all-masonry home, it is rather unlikely that you will even have rim joists at the load bearing walls. However, you should check as anything is possible and we have installed a few in the past. A rim-joist, by the way, is a joist that goes around the perimeter of each story and “bands” the joists together. In all-masonry homes, the brick/stone/etc acts as the rim-joist and joist-hanger with the joists “pocketed” (inserted into a recessed pocket 4″ to 8″ deep) into it.
-A note that is worth mentioning, if you should decide to put your hand over the basement floor drain or any other portion of the sewer system and feel a draft, this is technically air infiltration as you find at bad window insulation. However, DO NOT INSULATE THIS LEAK! This leak is the result of improper p-traps (or proper s-traps that are not code legal anymore as they are better siphons than traps) or a sewage yard trap that is not functioning correctly (possibly it is just not there as we have found once or twice). That draft or foul wind is sewer gas and you should skip calling the insulators, call a plumber or a drain layer. (Note: these are similar professions but plumbers work inside more, are often cleaner, and do less actual labor 
.
After dealing with the basement, move to the first floor. Remember, we are not even addressing windows and doors yet so relax. As this is the case, we will count the first floor as all floors above the basement and below the top floor.
First, feel around the surfaces of the cover plates of electrical/cable/phone outlets on exterior walls. Feel a breeze? Yep, air can sneak in anywhere. At this point,I should mention that discovering air leaks by this method is much easier in cold weather than warm weather as the cold draft is a sure giveaway. You can address these and other such leaks by sealing with foam and other methods which are discussed in the “Thermal Barriers-Sealing Leaks” blog post.
Next, check out the kitchen cabinets. Open cabinet doors and feel down around the kick plates. If you feel a breeze, it is likely coming up through a hole to the basement. But sometimes-an oblivious cabinet installer will not properly seal the walls/floorplate locations behind the cabinets before installing and this leaves behind the pesky leaks that are rather irritating to address. Generally, you will need to remove the cabinet to access the leak location, sealing the cabinet fronts will stop the symptom, but not address the problem and is therefore only a cosmetic fix.
Third, check around the entry of plumbing pipes into the bathrooms and around any recessed medicine cabinets. Pipe protrusions are notorious leak locations and can usually be easily fixed. You will need to lift any escutcheons or decorative cover rings out of the way to determine this one.
Finally, you should check around all the trim work. Casings (the decorative stuff nailed around the actual door jamb but not the door or door jamb) at doors and windows are usual offenders. But you might also look around the place at baseboard on exterior walls. Don’t be surprised to find a leak here as it is usually an element of the drywall hangers not taking the drywall to the floor before trim installation or the plaster has a break or fissure that is allowing air infiltration from the outside. Remember, in the case of a masonry house, brick is porous, and will allow a certain amount of air through. Imagine it as a rather hard sponge with tiny holes that air can pass through. A decent way to stop it is to make sure the plaster goes all the way to the floor behind the trim and is painted with a paint that has low vapor/air permeability (check the can or the manufacturer’s info sheets).