applications - What is the right way to close apps in Android?

I'm new to the Android platform, and this is probably a dumb question, but...

How do you close an app?

When I'm in an app, and I want to get out, I just click on "Home" and move on, but I just installed a Task Manager app, and I noticed everything is left running.

Is there a way to exit apps? Or do I need to use the Task Manager every time I exit something?