Deal Extreme - Twenty CR2016 lithium batteries for less than $0.17 each, shipped.
If you see something that is branded with a known name, probably best to assume that it's either a fake or it's a second in some way and then decide if you want to buy it. I'd also avoid things that might hurt if they fail eg I wouldn't buy a memory card from them as I'd hate to lose photos if the card turned out to be faulty.
Anyhow, as an example I recently bought twenty CR2016 batteries for USD$2.98 (AUD$3.31). So under AUD$0.17 per battery. These are used in things like watches, torches, calculators, remote controls (car, credit card size devices) and cost $4.19 for one battery at Dick Smith. So 20 batteries delivered to my home for less than the price of one that I'd have to find time to go into a bricks & mortar store (or pay even more to have it shipped).
Other recent purchase have been:
http://www.dealextreme.com/details.dx/sku.751 20*CR2032 batteries for USD$3.62. These are similar to the 2016 but about twice as thick and used in calculators, remotes (garage, tv, etc), toys, lights - all sorts of things and cost $4.19 for one battery @ dse. So $USD0.18 per battery or just under AUD$0.20 per battery at current exchange rates.
http://www.dealextreme.com/details.dx/sku.60 $4.65 50*AG13 LR44. These are used in toys, night lights, torches, remotes, talking books - if you have young kids, you probably go through a lot of these and they cost $4.19 for a twin pack @ dse. Again, around twenty batteries for the price of one.
http://www.dealextreme.com/details.dx/sku.24512 $2.99 USB to Serial Port adapter. Handy for the kit if you are a computer geek and/or have any older electronic kit (eg a weather station) that requires a serial interface and your computer only has USB ports. They cost $79.94 @ dse.
http://www.dealextreme.com/details.dx/sku.4256 $4.92 480p Component AV cable for Wii. If you have a Wii and have used the included cable to hook it up to the tv, if your tv supports component in then get one of these as you can improve the visual quality of games. It's not a huge difference and can vary depending on the game (the Wii is not an xbox 360 or PS3] but it is noticeable. A similar cable costs $49.76 at dse.
Yes you can get some of these things cheaper at places other than dse - I've just used that as an example. And yes the DX prices I've quoted are $USD and the dse prices are $AUD but I would note that in general you would be hard pressed to get cheaper (or even as cheap) for most of these, plus also the convenience of having the stuff shipped to your door is hard to beat. Shipping can take a while, in particular if they hold up shipping waiting for stock to arrive, but other than that I've been very happy with virtually everything I've got from them. Some people on the DX forums have complained about some failed batteries, but all the ones I've got from them have been well wrapped and worked just fine.
The savings on the batteries alone have easily paid for any other stuff I may have bought from dx, but I have a personal limit of generally not buying anything that costs more than $10 as I figure it's the cost of a magazine or two that I'd read and throw away.... so what the hey.
To that end though, there are lots of weird and wonderful things on the site - frankly I could spend hours going through it. Some stuff makes for wonderful cheap presents, or simply impulse buys for yourself. Try to pass up the solar cockroach (http://www.dealextreme.com/details.dx/sku.32431) or locust (http://www.dealextreme.com/details.dx/sku.30658) or how about a "drinky bird" (http://www.dealextreme.com/details.dx/sku.29233)?