I'm going to be a bit different in my answer. The best place to start is by taking a class. I know, spend $150 for 3 hours? Yes. Spend it. The hands-on time and instruction from a competent smith is worth it at twice the price. It will save you months of mistakes, teach you proper form, and give you some very important ideas about how to set up your own shop.
If you look for local blacksmiths, they are all over the place. You can usually find a class or workshop to attend, too.
After I took my first class, I made about the simplest propane forge there is. 2 soft firebricks (soft is important - the hard ones don't insulate) and a bead-maker's torch (Hot Head - but MagTorch sells the exact same head as a Large Pencil Tip torch for half the price of the HH) on a hose to a bbq tank. This was only meant to hold me over until I built the big forge. I built the big one. Spent more money than I like to admit on it, too. And I still do at least 90% of my forging with the little 2 brick forge.
As far as anvils go, you don't actually need 100 pounds or more. It is nice, but you don't need it. You do need stable and hard. A large flat rock will work. You can use a stump anvil. For inspiration, check out YouTube videos of Indonesian and Thai blacksmiths. There is at least one that shows a head from a sledge hammer used as an anvil. It is set into the stump so it doesn't go bouncing away, but he makes machetes on that little anvil. Railroad track is a time-honored starter anvil, of course. If you have any sort of scrap yard near you, go look for large hunks of steel with a flat face.
But, as I said at first, the best place to start is with a class. You might find that the instructor can hook you up with some tooling, too. Most smiths are tool and iron collectors. Much to the chagrin of our wives...
-Patrick
