Will a standard NAT-router firewall stop outgoing traffic?
I can see that the firewall will stop incoming traffic, but what will stop outgoing traffic in the case of malware being on the machine?
Is it solely the responsibility of the operating system's firewall to stop outgoing traffic?
Most firewalls (that I've worked with) have an implicit allow rule that allows all traffic from a more secure network (usually the internal LAN) to a less secure network (usually the internet) so that all outbound traffic is allowed from your internal LAN to the internet (or to a DMZ) You do not usually need to create explicit rules to allow outbound traffic.
Most firewalls (that I've worked with) have an implicit deny rule that denies all traffic from a less secure network (usually the internet) to a more secure network (usually the internal LAN) so that all inbound traffic is denied from the internet (or from a DMZ) to your internal LAN. You do usually need to create explicit rules to allow inbound traffic.
The hardware firewall cannot readily distinguish between "good" or "bad" packets, since they all come from the same source. The software firewall can ascertain which application generated the packet, so is better prepared to block them as desired.
It is up to the firewall to determine which traffic can flow in which direction, if you haven't configured any outbound rules, then, by default it will probably allow all outbound connections.
"In the case of malware" is separate, as that can only be determined by the PC itself. (Unless you invest in a N/IDS).
There are ways of preventing outbound traffic unless a human is involved, see http://en.wikipedia.org/wiki/Captive_portal
For standard over-the-retail-counter router/firewall combos, outbound gets less attention than inbound traffic. The last few I've seen had the ability to limit outbound as well. The focus is reversed though. On my home NetGear it allows inbound and blocks outbound. I can allow SSH into a specific host and deny telnet outbound. In this sense it is not a true full-featured firewall. Some such routers do have the ability to block specific internal IP addresses from accessing the Internet side of such devices, though not all do.
As with so many things, "it depends."
Generally speaking, the typical home Broadband Router does not by default place any restrictions on outgoing traffic. Some can be configured to manage outgoing traffic, some have limited outgoing options, come have pretty comprehensive ability to manage outbound.
Any "enterprise" class or business firewall will absolutely have the capability to manage outgoing traffic, including free ones (search the site, there are a number of posts on firewalls).
Most sysadmins agree that it is necessary to manage outgoing traffic in order to be a good internet citizen and also to manage bandwidth and performance. For instance, generally outgoing SMTP and POP traffic is only allowed from the mail server, to prevent any malware infected PCs from producing spam. Or, if there is a proxy server to manage browsing, outgoing web browsing traffic is only allowed from the proxy server.
In case of malware on a host, what is the point of blocking outgoing traffic with the same machine's firewall? The astute malware author will use whatever means (standard APIs for exception rules or something rootkit-like) to allow it to communicate with the internet.
Consider trying to detect that kind of traffic — from the router rather than the host.