The fuel tank ventilates into a small plastic expansion tank in the trunk. From this tank, there is a blue plastic tube that runs through the length of the car and into the engine bay and then into the charcoal canister. From there, a second tube runs to your air cleaner (or intake manifold on fuel-injected models), where the fuel vapor from the expansion tank is eventually burned by the engine. The charcoal in the canister serves two purposes: first, it stores the hydrocarbons from the fuel tank ventilation while the engine is not running (that is, while engine vacuum isn't pulling the vapor into the air cleaner or manifold), secondly, it is a safety mechanism: a backfire in your intake won't make it back into the fuel tank.
Now sure, after many years, the canister looks pretty nasty (mostly due to battery acid eating away at it). Replacements are readily available new (example: "activated carbon filter") and they're not that expensive. I've seen the damn thing disconnected by the previous owner of every car I've owned or helped a friend buy and there's never much of an explanation offered as to why:
- Sometimes the owner is confused, the last one handed me the car's original Zenith carburetors in a box and in there was the canister. He asked me, "what is this thing anyway? I took it out".
- Sometimes it's "I was taking out the smog equipment and this seemed part of it".
On the 2002 we have, as a bonus, the fuel tank vent tube was plugged with a bolt by some previous owner -- that's really bad, as pressure in the tank will then keep building up with nowhere to go.
If you're really set on the idea of taking all this stuff out, or you have nowhere to connect the line because you've converted to Weber side drafts and have no air cleaner to speak of, look at how it was done on European-market cars: they simply ran the vent line from the expansion tank down under the rear bumper, but there was still a vent line. On even earlier cars, there was no expansion tank but the fuel filler cap or neck were vented to the outside.
Please also consider this when you complain of fuel vapor inside the car. Perhaps this small set of tubing that is supposed to vent out the fuel tank into the air cleaner is no longer hooked up, venting fuel vapor into the engine bay? Or did someone disconnect the line at the expansion tank, causing vapor to leak into the trunk?
In addition, on later cars like the E12 528i and E28 five series, there is a thermal/vacuum-operated switch in-line with the tubing from the canister on its way to the intake manifold. This additional step keeps the fuel vapor from being burned before the engine is warmed up (and then is able to burn it better), which decreases emissions. This is probably overkill for early cars that never had it, but that's what it's for in case someone is wondering.
(thanks to Tony and Keith on the SSR list and to john_a on the 2002 FAQ message board for additional details).