Well for the most part I'm sure that nobody cares about an accent. I know several people that find it kind of sexy. As long as you're getting your point across and it's not for something strictly academic there's never anything to worry about. It's not wrong; it's just different. Well actually, used to could, supposebly and irregardless can be pretty bad, but the way you say the right things should never matter in a casual setting. Sometimes you can not mean to say it but they tend to leak out anyway because our parents say it and all of our friends say it. I'm mainly speaking to everyone who isn't from the South:

Southern accents are by no means an indicator of poor education; being American is.