Astronomy is one of those hobbies in which it is easy to spend a lot of money up front and then find your interest is waning.
There is no set way to start out, but to help prevent being burned, here is one procedure where you can learn astronomy from the ground up (so to speak :-)), and spend little, if any, money.

Step 1: JOIN A CLUB!
Joining your local astronomy club is the first step; not only will you get advice and assistance from its members, but they may also loan you equipment  and materials. Don’t be afraid to ask “stupid” questions; the members will be glad to help.
Many clubs meet at an observatory, which may have access to a library, where you can find books and magazines for learning the sky, amongst other things. 
Speaking personally, my local Astro Society meets at an observatory. It has a library where members can loan out books and periodicals for free. Members also receive free training on how to use the Observatory’s main telescope.
Clubs frequently hold beginner’s classes or evenings, which give plenty of good information on starting out, with demonstrations and discussions on various topics. 
Most clubs also host xporn star parties at dark sky sites, where you can try out other people’s equipment to get a good idea about what you can really see through binoculars and telescopes.
Various magazines such as Sky & Telescope periodically publish listings and contact details of clubs, if you are not sure about any clubs local to you.

After joining a club, you should begin learning the sky. You can start doing this before joining a club, but I recommend the club first. 
Constellations and some deep sky objects are visible with the naked eye. Binoculars will show many fainter objects such as galaxies, and other objects will appear brighter and with more detail. Of course, a telescope shows the most. 
When I first started, I found the constellation Orion with little difficulty. From there, I just worked my way around. 
The first thing you need for learning the sky is a set of star charts / books, magazines, a planesphere, or planetarium software (all are discussed later).
Before you start to go out observing, it is a good idea to set targets for the night: make a list of what you want to see, and when you find it, make a record of its position and the date / time, and if you like, a sketch. Once you have found a constellation or an object, it is easy to locate it again, and you will probably find that you learn other ones quite quickly. You will not need to know the whole sky of course, but the more you know, the easier it is to find objects.

There are many books, magazines, and practical aids that will help you learn the sky. These are discussed on the Essential Materials page.
Once you have put the effort into learning the sky, you will be ready to begin using optical aids: Binoculars and telescopes. 
These are discussed on the Equipment Page.