Here is the IRS definition of a business expense found on their site at http://www.irs.gov/publications/p535/index.html:
To be deductible, a business expense must be both ordinary and necessary. An ordinary expense is one that is common and accepted in your industry. A necessary expense is one that is helpful and appropriate for your trade or business. An expense does not have to be indispensable to be considered necessary.
Now I am not going to get into the deep dark complicated details. What I am going to get into is a few ways to keep track of your expenses.
The simple thing many do is keep a shoe box or envelope that holds all receipts for expenses. This can get messy but at least it is a start. If you are going to use this method alone try to keep it semi organized. Have more than one envelope and have a tracking page attached to each. Keep one for office supplies (paper printer ink, envelopes, tape, paper clips, pens, etc) gas receipts, (this is for keeping up with gas prices at the time of business travel. You also need to keep track of miles traveled for business only) meal receipts (these are meals strictly for closing the deal, negotiations, contracts, etc. These are not for personal meals turned into business chit chat), charity donations, misc, and so forth.
Keeping a ledger is a step up from the above. You still have to keep receipts but you can keep a running total better. You can purchase a ledger at an office supply store. Still takes work but a cheap way to keep those records.
Yet another step up create or have someone create an excel sheet. All the numbers can be tallied for you. Again you still have to keep those receipts but this gives you an instant view of all your expenses.
If you are familiar or willing to learn check out Quickbooks for all your accounting needs. These types of software are the best way to stay organized and on top of all your business expenses.
Software and spread sheets are the preferred methods or staying organized but the old school methods work as well. No matter the method you have to keep these up. They don't keep themselves up.
Now I am not a tax expert and can only help you in the direction to find the information you need. For all your questions regarding deductions and how to sort these for tax purposes please visit the IRS site at http://www.irs.gov/publications/p535/index.html.