How secure are password-protected WinRAR archives?

What I'm seeing (http://en.wikipedia.org/wiki/RAR) says that RAR3-format files use AES for the encryption algorithm. It's unclear to me on first glance if the RAR3 file format is published or if there are open source implementations of the decryption / uncompression algorithm. If the format isn't published / or there aren't free implementations of the decryption / uncompression algorithm I think I'd be pretty wary of the "security" since there's always the possibility that tricks like placing a known-plaintext into the header of every encrypted file, leaking bits of the key, etc could be at play.

The older RAR formats used a "proprietary encryption algorithm". You should always be VERY wary of programs that use "proprietary encryption algorithms". The phrase "proprietary encryption algorithm" is often code for "something knocked togther in a basement by a coder who doesn't really know much about cryptography", or more loosely as "has not been peer-reviewed".

Edit: I'm seeing what look to be free implementations of at least the uncompression portion of RAR3 (http://sourceforge.net/projects/java-unrar, for example). As long as the file format is out in the open it should be difficult for a large amount of your key's bits to be leaked by an untrustworthy implementation. Still, I'd feel better with something that's been peer reviewed or certified (FIPS, etc).

As you have stated there are password crackers/removers out there. I would not trust my files to a password protected archive files. I would suggest some type of file level encryption like GnuPG or AES Crypt

If I remember correctly then using password on an archive doesnt necessary include encryption (it's a separate option).

Evan's answer is much more informative when it comes to the algoritm :)