Ideally you will need a broadband connection with a speed of at least 128 kbps (like ADSL) to ensure good call quality and successfully use VoIP. There are various reasons for this.
1. You need at least 64 kbps download plus 64 kbps upload bandwidth to make an uncompressed VoIP call. Otherwise you will need to compress your VoIP traffic using compression codecs such as G.729 which can decrease the bandwidh to approximately 30 kbps. However such methods of compression are usually used in situation when multiple calls are going over the same connection, not when bandwidth is restricted to under 128 kbps. If you have dial-up you will be very fortunate to achieve decent call quality even when using compression.
2. A braodband connection is 'always on', thereby ensuring that you can receive and make calls with minimum interruption.
3. Depending on how you use your Internet connection, you want to make sure that you have enough bandwidth left over for typical Internet traffic (e.g. emails, web browsing, file downloads).
If you are unsure of how your Internet connection will be able to cope, try testing your connection for VoIP here