Mint.com uses something called OFX (Open Financial Exchange) to get the information in your bank account. If someone accessed your mint account they would not be able to perform any transactions with your bank. All they would be able to do is view the same information you do, which some of it could be personal <- that's up to you.
Generally the weakest point in security is with the user. An "attacker" is far more likely to get your account information from you then he is from the site your registered with.
Why you're the weakest point:
When you enter your account information, your password is never saved exactly how you enter it. It's passed through what is called a "one way function", these functions are easy to compute one way but given the end-result is EXTREMELY difficult to compute in reverse. So in a database if someone looked up your password they would see it something like this "31435008693ce6976f45dedc5532e2c1". When you log in to an account your password is sent through this function and then the result is checked against what is saved in the database, if they match you are granted access. The way an attacker would go about getting your password is by entering values into the function and checking the values against yours, this is known as a brute force attack. For our example (31435008693ce6976f45dedc5532e2c1) it would take someone 5 million years to decry-pt using a basic brute force attack. I used "thisismypassword" as my example password, it's 12 characters long. This is why most sites urge you to create long passwords with a mix of numbers, uppercase, lowercase and symbols.
This is a very basic explanation of security and both sides have better tools then the one explained but this gives you an idea of how security works for sites like these.
You're far more likely to get a virus or a key logger steal your information.
I do use Mint.
Edit:
From the Mint FAQ:
Do you store my bank login information on your servers?
Your bank login credentials are stored securely in a separate database
using multi-layered hardware and software encryption. We only store the
information needed to save you the trouble of updating, syncing or
uploading financial information manually.
Edit 2:
From OFX
About Security
Open Financial Exchange (OFX) is a unified specification for the
electronic exchange of financial data between financial institutions,
businesses and consumers via the Internet.
This is how mint is able to communicate with even your small local bank.
FINAL EDIT: ( This answers everything )
For passwords to Mint itself, we compute a secure hash of the user's
chosen password and store only the hash (the hash is also salted - see
http://en.wikipedia.org/wiki/Sal... ). Hashing is a one-way function
and cannot be reversed. It is not possible to ever see or recover
the password itself. When the user tries to login, we compute the
hash of the password they are attempting to use and compare it to the
hashed value on record. (This is a standard technique which every site
should use).
For banking credentials, we generally must use reversible encryption
for which we have special procedures and secure hardware kept in our
secure and guarded datacenter. The decryption keys never leave the
hardware device (which is built to destroy the key material if the
tamper protection is attacked). This device will only decrypt after
it is activated by a quorum of other keys, each of which is stored on
a smartcard and also encrypted by a password known to only one person.
Furthermore the device requires a time-limited
cryptographically-signed permission token for each decryption. The
system (which I designed and patented) also has facilities for secure
remote auditing of each decryption.
Source: David K Michaels, VP Engineering, Mint.com - http://www.quora.com/How-do-mint-com-and-similar-websites-avoid-storing-passwords-in-plain-text