Interesting question, it depends on the use in my opinion. You're still protected in terms of the session is encrypted but you have no way of telling if it's the correct SSL cert presented to you unless you distribute your CA root cert to users/clients.
For internal testing/dev projets this would work great, you generate a root CA certificate that you use distribute to your users (can be done via Group Policy in Windows and via openssl command line in Linux/BSD) and then use that root cert for signing your CSR's. Users will not see a warning or anything and you know the certificate is signed by your internal CA.
For external sites where you cannot assure this, I'd still say a self signed cert is better than no SSL at all if you are sending passwords or other sensitive information over the connection.
However, on the plus side there are plenty of very cheap "commercial" certificate issuers, GoDaddy being one of them. You can get a cert for about 40 euros per year. GoDaddy even offer free certs to OpenSource project web-sites.
I'm going to disagree, once on narrow technical grounds, and once on general grounds.
The narrow technical ground is that the OP asked about self-signed certificates, and several other answers refer to certificates signed by private CAs, which is a slightly different issue. But not very different, so that is really only a note in passing.
The major objection is that I think that as long as commercially-signed certificates are of more than trivial expense - and $40 a year is not a trivial expense for a lot of people on this planet - self-signed certificates have a major role to play in internet security, provided their limitations are recognised.
A self-signed certificate is like an ssh key from my known_hosts file. Without independent verification, it can't assure me that I'm talking to the system that I believe I am; but it can assure me that the system I am talking to now is the same system I talked to last time I thought I was having a conversation with it. We cache ssh keys all the time, and I've never yet met a system administrator that independently verified more than a fraction of the public keys in his/her known_hosts file.
Self-signed certificates (and, for that matter, certificates signed by not-generally-valid CAs) are much better than no SSL at all, as long as people realise that unless they verify them they only secure the communication to themselves, the server at the other end of the DNS record, and any men-in-the-middle currently on the line. If they independently verify the certificate, then the authentication and encryption are at least as strong as that provided by a certificate signed by a recognised CA.
For external sites, where the users don't have your CA certificate installed (which is the most common case), yes, a self-signed certificate gives a false sense of security, and thus it's worse than useless:
First, without a pre-installed CA certificate, how does the user verify that the certificate indeed comes from you, and not from an attacker? By matching the certificate fields (CN, fingerprints, etc), of course - but against what? So now you need a side channel to verify the certificate - and in the few instances I've seen that, the support line operators (who should have served as the side channel for verification) have no clue what that is supposed to mean; also, operating such side channel is orders of magnitude more expensive than getting a trusted-CA-signed certificate, so the user must blindly trust you.
Second, the scary alert the user gets is scary for a good reason: since the user can't/won't verify the presented certificate, they might be securely transmitting data to the Elbonian Haxx0r D00dz.
Third, and worst of all, you are desensitizing the users: "well, they told me that I should ignore that warning on https://mysite.example.com/ , and an anvil didn't drop on my head, and my goldfish didn't die either; sooooo, that means that it's just another alert box without any actual meaning, so I can ignore this box whenever I encounter it - I get that nice lock icon thingy anyway, and that's important."
In other words, the level of protection is comparable to plain HTTP (except for on-the-wire sniffing: although the data are encrypted in transit, that is a rather anemic feature without endpoint verification), but the feeling of protection is unreasonably high. Bad car analogy: "I have ABS, so I can now drive safer in bad conditions" - except the ABS only exists in the car's sales brochure, without actually being present in the car.
Depends. If you think it makes you more secure then it increases your risk as you choose to do riskier things with your false sense of security. If you treat it functionally equivalent to HTTP, then I'd say you are slightly more secure.
Without SSL/HTTPS anyone with wireshark on your network (or the local network of anyone logging in) can trivially listen in and capture username / passwords sent as plain text.
With self-signed SSL they cannot simply listen in, but now must fake your site, potentially alter DNS to do a man it the middle (MITM) attack. This is still a threat, but significantly more difficult for them to accomplish.
The other problem with using self-signed SSL is that many browsers treat self-signed certificates as a major security threat and warn you before entering (e.g., chrome) with a giant red page. http://www.sslshopper.com/article-ssl-certificates-in-google-chrome.html This could be a major inconvenience.
So bottom line is: if you run something that doesn't need to be particularly secure (e.g., no credit card data, no social security #s) and can't afford a proper certificate, a self-signed certificate could make some sense (to say prevent other network users from easily sniffing their login information).
It depends what you mean with "security", and which is the extent.
By example, your browser comes with a set of CA accepted by default. This means, any certificate issued by this CA is accepted by your browser (until it matches with the dns name) . Now, imagine some evil government owns a CA, or can force the CA to issue a certificate for the site you are visiting.
Then, to do a MITM is very easy: they can proxy your connection, sending to your browser the certificate they have, and your browser will accept it, since it comes from a "trusted" CA. Until they are DNS-transparent (which is the base for MITM) this is ok.
So, the "list of accepted CA" is basically a big security hole, until one of this CA cooperate with some evil government. Having your own CA is much better.
Of course, you can do it in your company or in your home server, because you can install your own CA certificate into the client, which you control too. In a public server, you can't deal with any user, asking him to add an exception, or to add your certificate.
So, it depends what you mean for "security", and the scope. IF you own the clients, FOR SURE, the self signed is safer, until you install on the client itself the certificate of your own CA.
OF course, you can't do it on a public website since you don't own all clients. So you will get a jeopardy of different behaviours, which is not safe.
Interesting question, it depends on the use in my opinion. You're still protected in terms of the session is encrypted but you have no way of telling if it's the correct SSL cert presented to you unless you distribute your CA root cert to users/clients. For internal testing/dev projets this would work great, you generate a root CA certificate that you use distribute to your users (can be done via Group Policy in Windows and via openssl command line in Linux/BSD) and then use that root cert for signing your CSR's. Users will not see a warning or anything and you know the certificate is signed by your internal CA.
For external sites where you cannot assure this, I'd still say a self signed cert is better than no SSL at all if you are sending passwords or other sensitive information over the connection.
However, on the plus side there are plenty of very cheap "commercial" certificate issuers, GoDaddy being one of them. You can get a cert for about 40 euros per year. GoDaddy even offer free certs to OpenSource project web-sites.
I'm going to disagree, once on narrow technical grounds, and once on general grounds.
The narrow technical ground is that the OP asked about self-signed certificates, and several other answers refer to certificates signed by private CAs, which is a slightly different issue. But not very different, so that is really only a note in passing.
The major objection is that I think that as long as commercially-signed certificates are of more than trivial expense - and $40 a year is not a trivial expense for a lot of people on this planet - self-signed certificates have a major role to play in internet security, provided their limitations are recognised.
A self-signed certificate is like an ssh key from my
known_hostsfile. Without independent verification, it can't assure me that I'm talking to the system that I believe I am; but it can assure me that the system I am talking to now is the same system I talked to last time I thought I was having a conversation with it. We cache ssh keys all the time, and I've never yet met a system administrator that independently verified more than a fraction of the public keys in his/herknown_hostsfile.Self-signed certificates (and, for that matter, certificates signed by not-generally-valid CAs) are much better than no SSL at all, as long as people realise that unless they verify them they only secure the communication to themselves, the server at the other end of the DNS record, and any men-in-the-middle currently on the line. If they independently verify the certificate, then the authentication and encryption are at least as strong as that provided by a certificate signed by a recognised CA.
Moreover, those wishing to present the use of certificates signed by a recognised CA as the one-and-only internet security panacea may need to think hard about issues such as the inclusion of the Chinese Government's signing CA in the standard Mozilla bundle, and the fraudulent SSL certificates signed by Comodo.
For external sites, where the users don't have your CA certificate installed (which is the most common case), yes, a self-signed certificate gives a false sense of security, and thus it's worse than useless:
First, without a pre-installed CA certificate, how does the user verify that the certificate indeed comes from you, and not from an attacker? By matching the certificate fields (CN, fingerprints, etc), of course - but against what? So now you need a side channel to verify the certificate - and in the few instances I've seen that, the support line operators (who should have served as the side channel for verification) have no clue what that is supposed to mean; also, operating such side channel is orders of magnitude more expensive than getting a trusted-CA-signed certificate, so the user must blindly trust you.
Second, the scary alert the user gets is scary for a good reason: since the user can't/won't verify the presented certificate, they might be securely transmitting data to the Elbonian Haxx0r D00dz.
Third, and worst of all, you are desensitizing the users: "well, they told me that I should ignore that warning on https://mysite.example.com/ , and an anvil didn't drop on my head, and my goldfish didn't die either; sooooo, that means that it's just another alert box without any actual meaning, so I can ignore this box whenever I encounter it - I get that nice lock icon thingy anyway, and that's important."
In other words, the level of protection is comparable to plain HTTP (except for on-the-wire sniffing: although the data are encrypted in transit, that is a rather anemic feature without endpoint verification), but the feeling of protection is unreasonably high. Bad car analogy: "I have ABS, so I can now drive safer in bad conditions" - except the ABS only exists in the car's sales brochure, without actually being present in the car.
Suggested reading: OWASP SSL Best practices
TL;DR: By using self-signed certificates on public-facing sites, you are making the Net a worse place, one clueless user at a time.
Depends. If you think it makes you more secure then it increases your risk as you choose to do riskier things with your false sense of security. If you treat it functionally equivalent to HTTP, then I'd say you are slightly more secure.
Without SSL/HTTPS anyone with wireshark on your network (or the local network of anyone logging in) can trivially listen in and capture username / passwords sent as plain text.
With self-signed SSL they cannot simply listen in, but now must fake your site, potentially alter DNS to do a man it the middle (MITM) attack. This is still a threat, but significantly more difficult for them to accomplish.
The other problem with using self-signed SSL is that many browsers treat self-signed certificates as a major security threat and warn you before entering (e.g., chrome) with a giant red page. http://www.sslshopper.com/article-ssl-certificates-in-google-chrome.html This could be a major inconvenience.
So bottom line is: if you run something that doesn't need to be particularly secure (e.g., no credit card data, no social security #s) and can't afford a proper certificate, a self-signed certificate could make some sense (to say prevent other network users from easily sniffing their login information).
It depends what you mean with "security", and which is the extent.
By example, your browser comes with a set of CA accepted by default. This means, any certificate issued by this CA is accepted by your browser (until it matches with the dns name) . Now, imagine some evil government owns a CA, or can force the CA to issue a certificate for the site you are visiting. Then, to do a MITM is very easy: they can proxy your connection, sending to your browser the certificate they have, and your browser will accept it, since it comes from a "trusted" CA. Until they are DNS-transparent (which is the base for MITM) this is ok.
So, the "list of accepted CA" is basically a big security hole, until one of this CA cooperate with some evil government. Having your own CA is much better.
Of course, you can do it in your company or in your home server, because you can install your own CA certificate into the client, which you control too. In a public server, you can't deal with any user, asking him to add an exception, or to add your certificate.
So, it depends what you mean for "security", and the scope. IF you own the clients, FOR SURE, the self signed is safer, until you install on the client itself the certificate of your own CA.
OF course, you can't do it on a public website since you don't own all clients. So you will get a jeopardy of different behaviours, which is not safe.