I have a Chef server running on our local network and tested it with other machines on the same network and everything worked fine. I started up a ec2 instance, and try to bootstrap and it works until it tries to connect to the Chef server. My question is, is there any way around having the Chef server public? Or is there a way that I can bypass whatever the node requires from the server?
When you apply the most common model to use Chef Client together with Chef Server, then the client needs to reach the server. You can apply any networking technique from VPN to whitelisting client IPs, but the client (normally) checks in regularly.
Another way would be to use Chef Zero (in the good old days people used Chef Solo for that and you might find some blog posts about this). This way, you can run your cookbooks from the local disk (the one of the EC2 instance) or IIRC directly from a HTTP URL, i.e. S3.
Chef does not follow a push model, which would allow you to set up the target instance just through an SSH connection. If you really need that, then Ansible might be the better fit for you.
Nevertheless, I would recommend you to let your Chef Server also run on AWS - or directly use their Chef offering AWS OpsWorks.
I currently have the same setup as you:
When I bootstrap from the Chef Server to the AWS EC2, it will fail.
Solution that worked for me:
On the AWS EC2 Instance, modify the hosts file to include the following
knife bootstrap aws-ec2-public-ip -x root