Do I really need client certificates to be signed by a trusted CA?

Sure, you can do this, but you lose the benefits of a CA. The benefit of a CA is that you only need to know the CA, you don't need to know every client individually beforehand. If you use a scheme where you recognize only the key, you'll need to individually configure each key in the server. If you just want to know that it's the same user, then you can just confirm it's the same public key.

Sure, and you can do that, by having clients self-sign every certificate and manually trust each one of those as its own root CA, instead of having them signed by a trusted CA at all.

But there really isn't much sense in doing that - it sounds like you have clients who have certificates that are signed by a trusted CA. Are you just trying to avoid the trust relationship validation step?

Can you clarify what you're trying to achieve?

Suppose I have a site that I want users to be able to log into with client certificates.

To be clear, we're talking about logging in with certificates and not speaking of any encryption topics.

As I understand it, the client is presenting the site with the public half of a keypair and proving that they have the corresponding private half. Is that correct?

That is correct for keypair authentication.

If so, then isn't checking that a client is presenting a known (previously-authorized) public key sufficient to know that it's a known user, without the certificate having been signed by a trusted CA?

It's sufficient to know that the client trying to authenticate has the private key (and perhaps the private key password if it's been protected that way). As long as this is merely for authentication and not encryption, you don't need to worry about certificate authorities. You just need to collect public keys and delegate authority based on those keys appropriately.

When you use client-certificate authentication, towards the end of the handshake, the client sends a Certificate Verify TLS message where it signs with its private key the concatenation of all the TLS messages that have been exchanged between the client and the server: something commonly known by both.

This is independent of whether the client-certificate is trusted or not. The server still has verify the signature against the public key presented in the client certificate. If it failed, the handshake would fail.

At the end of the handshake, whether or not the server trusts what the certificate asserts, that is, the binding between the public key, the identifier and various other attributes, it will know at least that the client has the private key for the public key in this certificate (the rest may or may not be true).

If you have a pre-defined list of know public keys (akin to public keys you would set up for an SSH connection, for example), you can perform authentication this way. What you miss out on is the PKI: the whole infrastructure to help you manage the keys and who they belong to. Since most configuration settings are intended for use within a PKI, this may also need more work (including additional programming perhaps).

All the other properties of the TLS connection are intact: the encryption is still guaranteed in the same way as it would be within the context of a PKI. I'm not sure what @WesleyDavid is talking about in his answer on this subject. Anyway, it's about client certificates, so encryption between the client and the server would take place anyway, whether or not a client-certificate is presented (provided a cipher suite with non-null encryption is used, of course).