spuder's questions

How would you remotely wipe a linux system so that on the next boot it will not attempt to boot from the hard drive, instead it will prefer booting from the network?

Reasoning

I am setting up an environment where servers will pxe boot. I have one working example, however it installs the OS to the drive. When I want to run another test, I need to walk over to the machine, restart it, and boot off of a live disk and format the drive. On the next boot it will attempt to network boot again.

What I've tried

I am able to ssh to the machine, and have tried the following:

• fdisk
• rm -rf / --no-preserve-root

Both fail because the device is in use. I've tried to unmount the device with no luck.

• umount /dev/sda

Is there a way to remotely make a machine unbootable so I can more quickly test pxe configs?

I've setup ubuntu 16.04 to authenticate to active directory using realmd according to the azure instructions

I find that I can login as bob@example but not [email protected]

How can I configure ubuntu to allow logging in as both the short name and the long name.

[sssd]
domains = example.com
config_file_version = 2
services = nss, pam

[domain/example.com]
ad_domain = example.com
krb5_realm = EXAMPLE.COM
realmd_tags = manages-system joined-with-adcli
cache_credentials = True
id_provider = ad
krb5_store_password_if_offline = True
default_shell = /bin/bash
ldap_id_mapping = True
use_fully_qualified_names = True
fallback_homedir = /home/%u@%d
access_provider = ad

The audit logs for both an unsuccesfull and a successful login

[email protected]

Oct 24 20:25:59 server1 sshd[55960]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=192.0.2.0  [email protected]
Oct 24 20:26:00 server1 sshd[55960]: pam_sss(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=192.0.2.0 [email protected]
Oct 24 20:26:00 server1 sshd[55960]: pam_sss(sshd:auth): received for user [email protected]: 17 (Failure setting user credentials)
Oct 24 20:26:02 server1 sshd[55960]: Failed password for [email protected] from 192.0.2.0 port 9682 ssh2
Oct 24 20:26:04 server1 sshd[55960]: error: Received disconnect from 192.0.2.0 port 9682:13: Unable to authenticate [preauth]
Oct 24 20:26:04 server1 sshd[55960]: Disconnected from 192.0.2.0 port 9682 [preauth]

bob@example

Oct 24 20:26:24 server1 sshd[55975]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=192.0.2.0  user=bob@example
Oct 24 20:26:26 server1 sshd[55975]: pam_sss(sshd:auth): authentication success; logname= uid=0 euid=0 tty=ssh ruser= rhost=192.0.2.0 user=bob@example
Oct 24 20:26:27 server1 sshd[55975]: Accepted password for bob@example from 192.0.2.0 port 9693 ssh2
Oct 24 20:26:27 server1 sshd[55975]: pam_unix(sshd:session): session opened for user bob@example by (uid=0)
Oct 24 20:26:27 server1 systemd-logind[961]: New session 325 of user [email protected].

I'm trying to setup rundeck so that it authenticates against Active Directory

I keep getting this error

The wiki contains information about the 403. Reason: !role error https://github.com/rundeck/rundeck/wiki/FAQ#i-get-an-error-logging-in-http-error-403--reason-role

Rundeck 2.6.2-1 (installed from .deb)
Ubuntu 14.04

jaas-ldap.conf

ldap {
    com.dtolabs.rundeck.jetty.jaas.JettyCachingLdapLoginModule required
    debug="true"
    contextFactory="com.sun.jndi.ldap.LdapCtxFactory"
    providerUrl="ldap://DC01.example.com:389"
    bindDn="cn=rundeck,OU=MyOU,DC=example,DC=com"
    bindPassword="correct-horse-battery-staple"
    authenticationMethod="simple"
    forceBindingLogin="true"
    userBaseDn="DC=example,DC=com"
    userRdnAttribute="sAMAccountName"
    userIdAttribute="sAMAccountName"
    userPasswordAttribute="unicodePwd"
    userObjectClass="user"
    roleBaseDn="DC=example,DC=com"
    roleNameAttribute="sAMAccountName"
    roleUsernameMemberAttribute="cn"
    roleMemberAttribute="member"
    roleObjectClass="group"
    cacheDurationMillis="300000"
    supplementalRoles="user"
    reportStatistics="true"
    timeoutRead="10000"
    timeoutConnect="20000"
    nestedGroups="true";
};

/var/lib/rundeck/exp/webapp/WEB-INF/web.xml

    ...
    <security-role>
            <role-name>Enterprise Admins</role-name>
    </security-role>
    ...

profile

...
export RDECK_JVM="-Djava.security.auth.login.config=/etc/rundeck/jaas-ldap.conf \
    -Dloginmodule.name=ldap \
...

I'm aware this is goes against best practices for the following reasons:

• Uses "simple" authentication and port 389. All passwords sent in plain text!!!
• baseDN are way too broad, should be narrowed down to speed up ldap search
• Don't need a bindDN if using forceBindingLogin

Additional resources:

https://github.com/rundeck/rundeck/issues/590
https://github.com/rundeck/rundeck/issues/620
http://www.bitester.com/2015/12/ldap-authentication-with-rundeck.html

I have two windows domain controllers.

10.10.10.10 Primary ( win 2008 r2 )
10.10.10.20 Replica ( win 2012 r2 )

The second one is configured as a replica of the first.

About once per week, the primary DC will negatively cache most .io domains. This makes it so noone in the company can access sites like:

chef.io
packer.io
yahoo.io
github.io

Strangely I can still access some .io pages, like the ones at github.io

spuder.github.io/

The solution is to RDP into the DNS server and run dnscmd /clearcache. That fixes the problem for 7 to 10 days.

Further symptoms

• Only affects the primary domain controller (the secondary, and other domain controllers can resolve these sites just fine)
• google dns servers also work
• Usually happens at about 11 am on wednesdays.

I'm not very familiar with windows, but here are the things I've tried

• Look at logs, I only see the following lines that look interesting

8:15AM
The DNS server wrote version 4638 of zone 254.10.in-addr.arpa to file 254.10.in-addr.arpa.dns..in-addr.arpa to file 254.10.in-addr.arpa.dns.

8:16AM
A more recent version, version 4639 of zone 254.10.in-addr.arpa was found at the DNS server at 10.254.40.51. Zone transfer is in progress.ic replication between domain controllers in a common domain or forest. By installing multiple domain controllers in a domain running DNS Server, you can ensure that DNS will continue to work when a domain co

• Verify there are no forward or reverse lookup zones for the .io domain
• Ensure there is nothing in the hosts file blocking the .io domain
• Compare the output of ipconfig /displaydns on all domain controllers

Is there anything else I can investigate to find out why the dns cache keeps getting corrupt so predictably? Is there a windows dns setting that can forcibly flush the cache when doing zone transer

Update
I've narrowed this down to the fact that I often switch from wired to wireless right before the Wednesday meeting. The wireless has 1 windows 2008 dns server and 1 windows 2012 dns server. When the 2008 server is selected as primary, the problem returns. The workaround is to run this dnscmd /clearcache. Since the 2008 server is going away, I'm sure this problem will fix itself.

I have a complicated software stack that I manage with 2 recipes.

recipe[stack::foo]
recipe[stack::bar]

foo is installed on every server, whereas bar is installed on only a subset of servers. bar depends on the file and service foo.

recipe[stack::foo]

  file 'fooFile' do
    source 'somewhere'
    notifies :restart, service[barService] #bar needs to be restart first, if it is present
    notifies :restart, service[fooService]
  end

  service 'fooService' do
    action :start
  end

recipe[stack::bar]

  file 'barFile' do
    source 'somewhere'
  end

  service 'barService' do
    action :start
  end

Is it possible to make a conditional dependency, so that if the bar recipe is present on the node, it will restart it. If it isn't, it will skip it.

I'm trying something like this

  file 'fooFile' do
    source 'somewhere'
    notifies :restart, service[barService] if exists? "service[barService]" 
    notifies :restart, service[fooService]
  end

I have a DNS server that is misbehaving.

Some of my windows machines only have 2 dns servers

server01  192.0.2.1
server02  192.0.2.2

Some of my windows machines have 3 dns servers

server01 192.0.2.1
server02 192.0.2.2
server03 192.0.2.3

How can I remove server03 (192.0.2.3) from my dns settings using powershell?

PS C:\Windows\system32> Get-DnsClientServerAddress

InterfaceAlias               Interface Address ServerAddresses
                             Index     Family
--------------               --------- ------- ---------------
Ethernet 2                          12 IPv4    {192.0.2.1, 192.0.2.2, 192.0.2.3}
Ethernet 2                          12 IPv6    {}
Loopback Pseudo-Interface 1          1 IPv4    {}

I have very little windows experience. On linux, I would edit the resolv.conf

Update These are windows 2012 R2 servers

I'm trying to interact with an F5 load balancer using the REST api. I have verified that iControl is enabled, however when I try and run a command, I am told that I am unauthorized.

curl -k -u someone -H "Content-Type: application/json" -X GET https://f5.example.com/mgmt/
Enter host password for user 'someone':
{"code":404,"message":"http://localhost:8100/mgmt/","restOperationId":202459,"errorStack":["com.f5.rest.common.RestWorkerUriNotFoundException: http://localhost:8100/mgmt/","at com.f5.rest.common.RestServer.trySendInProcess(RestServer.java:231)","at com.f5.rest.common.RestRequestReceiver.dispatchToService(RestRequestReceiver.java:93)","at com.f5.rest.common.RestRequestReceiver.processNext(RestRequestReceiver.java:57)","at com.f5.rest.common.RestHelper$2.run(RestHelper.java:1910)","at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)","at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)","at java.lang.Thread.run(Thread.java:722)\n"]}

My user is authenticated against Active Directory.

Do I need to explicitly grant user access to use the REST api?
Do I need to have a local account?

Update

Here is another example

The user 'someone' does in fact exist since that is how I login to the web portal.

curl -k -u someone -H "Content-Type: application/json" -X GET https://f5.example.com/mgmt/tm/sys
Enter host password for user 'someone':
{"code":401,"message":"Authorization failed: no user named someone found. Uri:http://localhost:8100/mgmt/tm/sys Referer:null","restOperationId":869853,"errorStack":["java.lang.SecurityException: Authorization failed: no user named someone found. Uri:http://localhost:8100/mgmt/tm/sys Referer:null","at com.f5.rest.workers.ForwarderWorker.evaluatePermission(ForwarderWorker.java:411)","at com.f5.rest.workers.ForwarderPassThroughWorker.onForward(ForwarderPassThroughWorker.java:191)","at com.f5.rest.workers.ForwarderPassThroughWorker.onGet(ForwarderPassThroughWorker.java:321)","at com.f5.rest.common.RestWorker.callDerivedRestMethod(RestWorker.java:735)","at com.f5.rest.common.RestWorker.callRestMethodHandler(RestWorker.java:702)","at com.f5.rest.common.RestServer.processQueuedRequests(RestServer.java:1092)","at com.f5.rest.common.RestServer.access$000(RestServer.java:45)","at com.f5.rest.common.RestServer$1.run(RestServer.java:136)","at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)","at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)","at java.lang.Thread.run(Thread.java:722)\n"]}

I have two domains

foo.local
test.local

I can run powershell hyper-v cmdlets against the test.local hyper-v server if I am using a VM that is in the test.local domain.

Get-VM -ComputerName hyperv01.test.local

I want to be able to run this same command from my workstation which is in the foo.local domain.

I believe that I need to enable delegation on the hyper-v server so that my workstation can run the hyperv commands, however it appears that I can't change domains in the Administrator Console.

Is there a way to allow my workstation in the foo.local domain to run Get-VM on a hyperv server in the test.local domain?

I'm tasked with designing and implementing configuration management (Chef) for a infrastructure where the production servers are completely segmented and isolated behind a jump box.

 ---------------------------------
| Production | Staging, Test, Dev |
 ---------------------------------

The only way to get files into production is with a sftp connection. There are only two people who have the ability to jump into production directly. An exception could be made to allow a tightly controlled automation system (like jenkins) to copy files into production.

Normally, I would recommend setting up a single chef server in the cloud, however the production servers can not have external access. (Without a lot of convincing of some very reluctant people).

One solution I see is to use two Chef servers

 ---------------------------------------
| chef-production | chef-local         |
| Production      | Staging, Test, Dev |
 ---------------------------------------

Using two chef servers has some problems that I'm not sure how to overcome.

• How can I keep the production and the local servers cookbooks in sync?
• How could we add new production nodes to the production chef server, without requiring one of the admins to run the knife command?
• How could we make changes to the databags in production?

I'm open to other suggestions. I'm considering not using a chef server and instead use chef-zero or chef-solo. My understanding is that it isn't the best solution when using multiple environments.

I'm a linux admin by trade, and my new job has me managing windows servers.

I'm trying to create a windows server 2012 base image using packer. As part of the provisioning, the VM needs to be connected to active directory via a script. Obviously I don't want to put my personal password into the script.

Is it possible to create a user in Active Directory who has rights to bind a machine to AD, but can't perform any other actions (for compliance)?

How would you configure a VM so that it will automatically register with a chef server?

I know that I could use knife bootstrap, however that is very tedious if you are starting dozens of servers at once.

The Chef documentation says to use chef-client -j /etc/chef/file.json and bake the that into the image along with your client.rb and validation.pem files. However that still requires ssh-ing into the machine to run the chef-client command.

Is there not a more automated way to do this?
How do other people provision thousands of machines at once?

Update

I contacted Chef support, they said you can start chef on boot in the following ways.

1. Add /usr/bin/chef-client to the /etc/rc.local file and ensure you have a client.rb and validatation.pem files in /etc/chef

2. Use the chef-client cookbook to manage chef and control the init script.

They also said that the chef-client doesn't run as a daemon. Instead it only runs when it needs to do a convergence. (Which seems contradictory to the documentation.)

These solutions introduce new problems though. 1. You will have duplicate chef-client processes running if you add it to the rc.local file 2. You need to run chef in order to manage chef with the chef cookbook. How do you then clean up and unregister the node so that you can take a snapshot?

I'm attempting to get my infrastucture defined as code. To do this I've spun up 2 jenkins servers with puppet. I then manually configured one, and copied the working config.xml to the other.

Since jenkins has the ability to read in the config.xml file, I would expect the second jenkins server to be an identical twin of the first.

Unfortunatly the second jenkins server crashes at startup with the following error:

Manager password must not be empty or null.

Things I've tried:

• do a recursive grep of the /var/lib/jenkins directory looking for any string that matches 'managerPassword' (found 0)
• copy the secrets files and directories from the old to the new one.

About:

Jenkins 1.572
Ldap plugin 1.10.2

I'm sure there is a way to copy the config from one jenkins server to another.
Has anyone done it? What files am I missing?

Full Error:

hudson.util.HudsonFailedToLoad: org.jvnet.hudson.reactor.ReactorException: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'initialDirContextFactory': Error setting property values; nested exception is org.springframework.beans.PropertyBatchUpdateException; nested PropertyAccessExceptions (1) are: PropertyAccessException 1: org.springframework.beans.MethodInvocationException: Property 'managerPassword' threw exception; nested exception is java.lang.IllegalArgumentException: Manager password must not be empty or null. at hudson.WebAppMain$3.run(WebAppMain.java:234) Caused by: org.jvnet.hudson.reactor.ReactorException: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'initialDirContextFactory': Error setting property values; nested exception is org.springframework.beans.PropertyBatchUpdateException; nested PropertyAccessExceptions (1) are: PropertyAccessException 1: org.springframework.beans.MethodInvocationException: Property 'managerPassword' threw exception; nested exception is java.lang.IllegalArgumentException: Manager password must not be empty or null. at org.jvnet.hudson.reactor.Reactor.execute(Reactor.java:269) at jenkins.InitReactorRunner.run(InitReactorRunner.java:44) at jenkins.model.Jenkins.executeReactor(Jenkins.java:896) at jenkins.model.Jenkins.<init>(Jenkins.java:795) at hudson.model.Hudson.<init>(Hudson.java:82) at hudson.model.Hudson.<init>(Hudson.java:78) at hudson.WebAppMain$3.run(WebAppMain.java:222) Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'initialDirContextFactory': Error setting property values; nested exception is org.springframework.beans.PropertyBatchUpdateException; nested PropertyAccessExceptions (1) are: PropertyAccessException 1: org.springframework.beans.MethodInvocationException: Property 'managerPassword' threw exception; nested exception is java.lang.IllegalArgumentException: Manager password must not be empty or null. at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyPropertyValues(AbstractAutowireCapableBeanFactory.java:1279) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1010) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:472) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory$1.run(AbstractAutowireCapableBeanFactory.java:409) at java.security.AccessController.doPrivileged(Native Method) at

Update

I tried recursivly (and while preserving permissions) copying the entire /var/lib/jenkins directory from the primary server to the secondary and restarted. It gives the same error about 'managerPassword'

I have an ubuntu 12.04 LTS server that is tied to Active Directory. This is a git server running gitlab. Rather than using gitolite it uses the gitlab shell

In the logs I notice that every time the 'git' user authenticates, I get the same 11 errors in the /var/log/auth.log

May 14 15:06:10 gitlab sshd[14775]: pam_unix(sshd:session): session opened for user git by (uid=0)
May 14 15:06:10 gitlab sshd[14775]: pam_mkhomedir(sshd:session): unknown option: skel
May 14 15:06:10 gitlab sshd[14775]: pam_mkhomedir(sshd:session): unknown option: =
May 14 15:06:10 gitlab sshd[14775]: pam_mkhomedir(sshd:session): unknown option: /etc/skel/
May 14 15:06:10 gitlab sshd[14775]: pam_mkhomedir(sshd:session): unknown option: mask=0077
May 14 15:06:10 gitlab sshd[14775]: pam_mkhomedir(sshd:session): unknown option: skel
May 14 15:06:10 gitlab sshd[14775]: pam_mkhomedir(sshd:session): unknown option: =
May 14 15:06:10 gitlab sshd[14775]: pam_mkhomedir(sshd:session): unknown option: /etc/skel/
May 14 15:06:10 gitlab sshd[14775]: pam_mkhomedir(sshd:session): unknown option: mask=0077
May 14 15:06:10 gitlab sshd[14775]: pam_mkhomedir(sshd:session): unknown option: skel
May 14 15:06:10 gitlab sshd[14775]: pam_mkhomedir(sshd:session): unknown option: =
May 14 15:06:10 gitlab sshd[14775]: pam_mkhomedir(sshd:session): unknown option: /etc/skel/

The 'git' user already has a home directory.

I'm at a bit of a loss on this one. None of the top hits on google shed any light on the problem. Does anyone have any ideas?

Update

I'm pretty new to pam, here are what I think are relevant config files

ls /etc/pam.d
atd   chpasswd  common-account  common-password  common-session-noninteractive  login     other   polkit-1  samba  su
chfn  chsh      common-auth     common-session   cron                           newusers  passwd  ppp       sshd   sudo

cat /etc/pam.d/common-session | grep -v '#'
session [default=1]         pam_permit.so
session requisite           pam_deny.so
session required            pam_permit.so
session optional            pam_umask.so
session required    pam_unix.so
session optional                    pam_sss.so
session optional pam_mkhomedir.so skel = /etc/skel/ mask=0077
session optional pam_mkhomedir.so skel = /etc/skel/ mask=0077
session optional pam_mkhomedir.so skel = /etc/skel/ mask=0077
session optional            pam_ck_connector.so nox11


cat /etc/pam.d/sshd | grep -v '#'
@include common-auth
account    required     pam_nologin.so
@include common-account
@include common-session
session    required     pam_limits.so
session    required     pam_env.so user_readenv=1 envfile=/etc/default/locale
@include common-password

I have a rpm (redis) that I need to install on a dozen servers.

The version of redis that is available in EPEL is too old (2.4.x). The software providers do not offer a repository online that I could download.

I need to get the latest versions of either 2.6.x or 2.8.x installed (preferably 2.6.x)

I've tried the following solutions. Each has their own problems.

• Compile 2.8.x from source (Doesn't scale, have to repeat 12 times)

• Use redis puppet module from puppet forge

All of the puppet modules are either undocumented, or install using the old redis url's and thus are locked to older versions. The only manifest that I've got to work, installs a maximum version of 2.6.14 which has broken init scripts on Centos

• Install using ruby gems, (there are no gems for versions older than 3.x.beta )

The only other way I see to do this is make my own repo and find a place to host it. That would require that I create a web server or ftp server internally which I really don't want to do.

Surely there is an easier way to get redis 2.6.x or 2.8.x installed on multiple machines isn't there? How else could this be done?

There appear to be many questions and guides out there that instruct how to setup nginx to redirect http requests to https. Many are outdated, or just flat out wrong.

# MANAGED BY PUPPET

upstream gitlab {
  server unix:/home/git/gitlab/tmp/sockets/gitlab.socket;
}

# setup server with or without https depending on gitlab::gitlab_ssl variable
server {
  listen *:80;
  server_name gitlab.localdomain;
  server_tokens off;
  root /nowhere;
  rewrite ^ https://$server_name$request_uri permanent;
}
server {
  listen *:443 ssl default_server;
  server_name gitlab.localdomain;
  server_tokens off;
  root /home/git/gitlab/public;

  ssl                           on;
  ssl_certificate               /etc/ssl/certs/ssl-cert-snakeoil.pem;
  ssl_certificate_key           /etc/ssl/private/ssl-cert-snakeoil.key;
  ssl_protocols                 SSLv3 TLSv1 TLSv1.1 TLSv1.2;
  ssl_ciphers                   AES:HIGH:!ADH:!MDF;
  ssl_prefer_server_ciphers     on;

  # individual nginx logs for this gitlab vhost
  access_log  /var/log/nginx/gitlab_access.log;
  error_log   /var/log/nginx/gitlab_error.log;

  location / {
    # serve static files from defined root folder;.
    # @gitlab is a named location for the upstream fallback, see below
    try_files $uri $uri/index.html $uri.html @gitlab;
  }

  # if a file, which is not found in the root folder is requested,
  # then the proxy pass the request to the upsteam (gitlab puma)
  location @gitlab {
    proxy_read_timeout 300; # https://github.com/gitlabhq/gitlabhq/issues/694
    proxy_connect_timeout 300; # https://github.com/gitlabhq/gitlabhq/issues/694
    proxy_redirect     off;

    proxy_set_header   X-Forwarded-Proto $scheme;
    proxy_set_header   X-Forwarded-Ssl   on;

    proxy_set_header   Host              $http_host;
    proxy_set_header   X-Real-IP         $remote_addr;

    proxy_pass http://gitlab;
  }
}

I've restarted after every configuration change, and yet I still only get the 'Welcome to nginx' page when visiting http://192.168.33.10. whereas https://192.168.33.10 works perfectly.

Why will nginx still not redirect http requests to https?

I've also tried the following configurations

  listen *:80;
  server_name <%= @fqdn %>;
  #root /nowhere; 
  #rewrite ^ https://$server_name$request_uri? permanent; 
  #rewrite ^ https://$server_name$request_uri permanent;
  #return 301 https://$server_name$request_uri;
  #return 301 http://$server_name$request_uri;
  #return 301 http://192.168.33.10$request_uri;
  return 301 http://$host$request_uri;

The logs

tailf /var/log/nginx/access.log
192.168.33.1 - - [22/Oct/2013:03:41:39 +0000] "GET / HTTP/1.1" 304 0 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:24.0) Gecko/20100101 Firefox/24.0"
192.168.33.1 - - [22/Oct/2013:03:44:43 +0000] "GET / HTTP/1.1" 200 133 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:24.0) Gecko/20100101 Firefox/24.0"

tailf /var/log/nginx/gitlab_error.lob
2013/10/22 02:29:14 [crit] 27226#0: *1 connect() to unix:/home/git/gitlab/tmp/sockets/gitlab.socket failed (2: No such file or directory) while connecting to upstream, client: 192.168.33.1, server: gitlab.localdomain, request: "GET / HTTP/1.1", upstream: "http://unix:/home/git/gitlab/tmp/sockets/gitlab.socket:/", host: "192.168.33.10"

Resources
https://www.nginx.com/resources/wiki/start/topics/tutorials/config_pitfalls/
How to make nginx redirect
How to force or redirect to SSL in nginx?
nginx ssl redirect
Nginx & Https Redirection
https://www.tinywp.in/301-redirect-wordpress/
How to force or redirect to SSL in nginx?

I need to install puppet 3.2.0 or greater along with ruby 2.0.0-p247 on an ubuntu 12.04 server.

Is there no way to install puppet without ruby 1.8?

Most my experience comes from CentOS so I may be missing something simple with apt.

My setup:

Make sure no ruby is present

root@gitlab:~# which ruby
root@gitlab:~# ruby --version
The program 'ruby' can be found in the following packages:
 * ruby1.8
 * ruby1.9.1
Try: apt-get install <selected package>

Install puppet from the official ppa

root@gitlab:~# cat /etc/apt/sources.list.d/puppetlabs.list

deb http://apt.puppetlabs.com precise main  
deb-src http://apt.puppetlabs.com precise main  

deb http://apt.puppetlabs.com precise dependencies  
deb-src http://apt.puppetlabs.com precise dependencies  

The latest version of puppet is listed in apt

root@gitlab:~# apt-cache policy puppet|head
puppet:
  Installed: (none)
  Candidate: 3.3.0-1puppetlabs1
  Version table:
     3.3.0-1puppetlabs1 0
        500 http://apt.puppetlabs.com/ precise/main amd64 Packages
     3.2.4-1puppetlabs1 0
        500 http://apt.puppetlabs.com/ precise/main amd64 Packages
     3.2.3-1puppetlabs1 0
        500 http://apt.puppetlabs.com/ precise/main amd64 Packages

Installing puppet 3.3.0 brings ruby 1.8 along automatically.

 root@gitlab:~# apt-get install puppet-common
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following extra packages will be installed:
  augeas-lenses debconf-utils facter hiera libaugeas-ruby libaugeas-ruby1.8 libaugeas0 libjson-ruby libruby libruby1.8
  libshadow-ruby1.8 ruby ruby-json ruby-rgen ruby1.8 virt-what
Suggested packages:
  augeas-doc augeas-tools ruby-selinux libselinux-ruby1.8 librrd-ruby1.9.1 librrd-ruby1.8 ri ruby-dev ruby1.8-examples ri1.8
The following NEW packages will be installed:
  augeas-lenses debconf-utils facter hiera libaugeas-ruby libaugeas-ruby1.8 libaugeas0 libjson-ruby libruby libruby1.8
  libshadow-ruby1.8 puppet-common ruby ruby-json ruby-rgen ruby1.8 virt-what
0 upgraded, 17 newly installed, 0 to remove and 92 not upgraded.
Need to get 3,664 kB of archives.
After this operation, 14.1 MB of additional disk space will be used.
Do you want to continue [Y/n]?

puppet version

root@gitlab:/tmp/ruby-2.0.0-p247# puppet --version
3.3.0

ruby version

root@gitlab:/tmp/ruby-2.0.0-p247# ruby --version
ruby 1.8.7 (2011-06-30 patchlevel 352) [x86_64-linux]

(note that ruby is automatically installed via update alternatives)

root@gitlab:/tmp/ruby-2.0.0-p247# ll `which ruby`
lrwxrwxrwx 1 root root 22 Sep 18 02:57 /usr/bin/ruby -> /etc/alternatives/ruby*


root@gitlab:/tmp/ruby-2.0.0-p247# update-alternatives --list ruby
/usr/bin/ruby1.8

I then install ruby 2.0 into it's own directory, from source

./configure --prefix=/usr/bin/ruby2.0

Then register ruby 2.0 with update-alternatives

update-alternatives --install /usr/bin/ruby ruby /usr/bin/ruby2.0/bin/ruby 10 

Switch to ruby 2.0 via update-alternatives

    root@gitlab:/tmp/ruby-2.0.0-p247# update-alternatives --set ruby /usr/bin/ruby2.0/bin/ruby
update-alternatives: using /usr/bin/ruby2.0/bin/ruby to provide /usr/bin/ruby (ruby) in manual mode.
root@gitlab:/tmp/ruby-2.0.0-p247# ruby --version
ruby 2.0.0p247 (2013-06-27 revision 41674) [x86_64-linux]

I've done this several times, as long as which ruby points to any version of ruby other than 1.8, puppet returns the following error:

root@gitlab:/tmp/ruby-2.0.0-p247# puppet --version
/usr/bin/ruby2.0/lib/ruby/2.0.0/rubygems/core_ext/kernel_require.rb:45:in `require': cannot load such file -- puppet/util/command_line (LoadError)
    from /usr/bin/ruby2.0/lib/ruby/2.0.0/rubygems/core_ext/kernel_require.rb:45:in `require'
    from /usr/bin/puppet:3:in `<main>'

How can I get puppet installed with ruby 2.0?

Puppet 3.3 is compatible with ruby 2.0 because it is listed on puppet's website

Additional resources

http://projects.puppetlabs.com/issues/18494

RVM appears to be far more complex than what I need. Also, the faq's say not to use it with sudo, or run it as root which I must do. https://rvm.io/
http://rvm.io/support/faq

https://stackoverflow.com/questions/1892877/how-do-i-make-ruby-1-9-the-default-ruby-on-ubuntu

How would you automate the installation of a specific version of git using puppet?

apt-get update && apt-get install git-core on my 12.04 ubuntu server results in git version 1.7.9.
I must have 1.7.10 or newer.

There are two options that I can see. 1. Add the ppa
2. Install git from source

I think adding a ppa would be easier that compiling from source, so that is what I am attempting.

I've tried using the puppetlabs/apt module to install the git-core ppa, yet my git version is still 1.7.9 after the puppet run.

root@gitlab:~# puppet module list
/etc/puppet/modules
├── puppetlabs-apt (v1.2.0)
├── puppetlabs-git (v0.0.3)
├── puppetlabs-stdlib (v4.1.0)
└── ruby (???)


root@gitlab:~# cat /etc/puppet/manifests/git.pp  

class { 'apt': }
apt::ppa { 'ppa:git-core/ppa': 
  before => Exec['apt-get update'],

}

exec{'apt-get update':
  path => ['/usr/bin', '/usr/sbin'],
}

package {'git-core':
   ensure => latest,
   require => Exec['apt-get update'],
}

root@gitlab:~# puppet apply /etc/puppet/manifests/git.pp --verbose
Info: Loading facts in /etc/puppet/modules/stdlib/lib/facter/facter_dot_d.rb
Info: Loading facts in /etc/puppet/modules/stdlib/lib/facter/pe_version.rb
Info: Loading facts in /etc/puppet/modules/stdlib/lib/facter/puppet_vardir.rb
Info: Loading facts in /etc/puppet/modules/stdlib/lib/facter/root_home.rb
Warning: Config file /etc/puppet/hiera.yaml not found, using Hiera defaults
Info: Loading facts in /etc/puppet/modules/stdlib/lib/facter/facter_dot_d.rb
Info: Loading facts in /etc/puppet/modules/stdlib/lib/facter/pe_version.rb
Info: Loading facts in /etc/puppet/modules/stdlib/lib/facter/puppet_vardir.rb
Info: Loading facts in /etc/puppet/modules/stdlib/lib/facter/root_home.rb
Info: Applying configuration version '1379214336'
Notice: /Stage[main]//Exec[apt-get update]/returns: executed successfully
Notice: Finished catalog run in 5.80 seconds
root@gitlab:~# git --version
git version 1.7.9.5

How would you use facter and puppet to determine if the OS is running Cent 6.x or Cent 5.x ?

facter operatingsystemrelease 
6.4

I only care about the major release (6)

I've thought about using awk, but there must be a better way that is more 'puppet manifest' friendly.

   #This works, but is ugly trying to use this in a puppet manifest

facter operatingsystemrelease |awk -F. '{print $1}'
6

Update:

It looks like the newer versions of facter have some additional information about major releases that isn't in my version. My initial provisioning needs to assume that facter is out of date.

facter --version
1.6.4
puppet --version
2.7.20

I've tried searching for any additional facts that might show the major release, with the following command

facter |grep 6

An apartment complex has fiber internet and is experiencing latency problems over the last month.

Tenants frequently experience timeouts and corrupted webpages. The current work around it to refresh the webpage half a dozen times until it loads correctly.

The symptoms are:

• Inconstant, happening a couple times per day per tenant
• Requesting a new dhcp lease on laptop does not solve the issue
• Affects both mac and windows machines Update: ONLY AFFECTS MAC USERS!*
• Affects both wireless and wired
• Is not a DNS issue because we have tried ISP's dns and google's dns servers with no improvement
• iTunes is heavily affected by this. iTunes store frequently timesout (iPad, iPhone, Mac)

What other diagnostic tools could be used to identify the problem? The ISP says everything looks fine.

A traceroute shows huge latency (several seconds) on hop 9.

traceroute google.com

traceroute: Warning: google.com has multiple addresses; using 74.125.224.168
traceroute to google.com (74.125.224.168), 64 hops max, 52 byte packets
 1  10.90.4.1 (10.90.4.1)  3.086 ms  0.738 ms  0.683 ms
 2  69.169.148.1.provo.static.broadweavenetworks.net (69.169.148.1)  0.907 ms  1.135 ms  0.893 ms
 3  10.8.201.41 (10.8.201.41)  1.040 ms  1.552 ms  11.494 ms
 4  97.75.190.142 (97.75.190.142)  1.343 ms  1.347 ms  0.946 ms
 5  97.75.190.137 (97.75.190.137)  1.290 ms  1.609 ms  1.202 ms
 6  97.75.191.66 (97.75.191.66)  2.463 ms  2.146 ms  2.161 ms
 7  97.75.191.54 (97.75.191.54)  2.406 ms  2.281 ms  2.616 ms
 8  te-9-3.car1.saltlakecity1.level3.net (4.53.40.105)  3.014 ms  2.330 ms  2.241 ms
 9  * * *
10  ae-61-61.csw1.losangeles1.level3.net (4.69.137.2)  15.805 ms
    ae-91-91.csw4.losangeles1.level3.net (4.69.137.14)  15.441 ms  15.160 ms
11  * ae-1-60.edge1.losangeles9.level3.net (4.69.144.10)  17.204 ms  15.983 ms
12  google-inc.edge1.losangeles9.level3.net (4.53.228.6)  92.445 ms  82.679 ms  107.813 ms
13  64.233.174.238 (64.233.174.238)  21.234 ms  21.016 ms  21.321 ms
14  72.14.236.11 (72.14.236.11)  21.577 ms  21.630 ms  21.568 ms
15  lax02s01-in-f8.1e100.net (74.125.224.168)  20.798 ms  20.687 ms  20.666 ms

Affects most webpages (google, apple.com, facebook.com ect..)

(lines 9, 17 and 18 all take a long time).

traceroute beachbody.com
traceroute to beachbody.com (66.208.81.68), 64 hops max, 52 byte packets
 1  10.90.4.1 (10.90.4.1)  1.038 ms  0.830 ms  0.767 ms
 2  69.169.148.1.provo.static.broadweavenetworks.net (69.169.148.1)  0.988 ms  0.934 ms  0.928 ms
 3  10.8.201.41 (10.8.201.41)  1.357 ms  1.375 ms  1.500 ms
 4  10.8.101.5 (10.8.101.5)  1.405 ms  1.579 ms  1.115 ms
 5  eth_3-3_prv02-rt02.veracitynetworks.com (97.75.190.166)  10.601 ms  1.563 ms  1.754 ms
 6  97.75.191.66 (97.75.191.66)  2.857 ms  13.554 ms  2.833 ms
 7  97.75.191.54 (97.75.191.54)  2.760 ms  2.394 ms  4.350 ms
 8  te-9-3.car1.saltlakecity1.level3.net (4.53.40.105)  2.352 ms  2.311 ms  2.340 ms
 9  * * *
10  ae-61-61.csw1.losangeles1.level3.net (4.69.137.2)  29.086 ms
    ae-71-71.csw2.losangeles1.level3.net (4.69.137.6)  28.958 ms
    ae-91-91.csw4.losangeles1.level3.net (4.69.137.14)  28.863 ms
11  ae-82-82.ebr2.losangeles1.level3.net (4.69.137.25)  28.075 ms
    ae-72-72.ebr2.losangeles1.level3.net (4.69.137.21)  28.508 ms
    ae-62-62.ebr2.losangeles1.level3.net (4.69.137.17)  29.029 ms
12  ae-6-6.ebr2.sanjose5.level3.net (4.69.148.202)  28.672 ms  28.586 ms  28.223 ms
13  ae-2-2.ebr2.sanjose1.level3.net (4.69.148.142)  28.426 ms  28.341 ms  29.611 ms
14  ae-4-4.car2.sacramento1.level3.net (4.69.132.157)  28.834 ms  29.236 ms  29.231 ms
15  ragingwire.car2.sacramento1.level3.net (4.53.202.22)  29.339 ms  29.406 ms  29.584 ms
16  resisp-74-221-224-49.smf.ragingwire.net (74.221.224.49)  26.096 ms  25.930 ms  26.575 ms
17  * 204.212.188.26 (204.212.188.26)  28.459 ms !X *
18  204.212.188.26 (204.212.188.26)  25.650 ms !X *  26.197 ms !X  

Update 1
Here is a traceroute with the same laptop, but different network location (sanitized).

beachbody.com fails 95% of the time at location 1. beachbody.com succeeds 100% of the time at location 2.

traceroute beachbody.com
traceroute to beachbody.com (66.208.81.68), 64 hops max, 52 byte packets
 1  foo.acme (y.y.y.y)  1.716 ms  13.343 ms  6.139 ms
 2  x.x.x.x (x.x.x.x)  74.524 ms  158.532 ms  6.721 ms
 3  tg9-2.cr01.slkcutxd.integra.net (209.63.98.37)  33.225 ms  24.794 ms  24.587 ms
 4  * be4.sc01.sntdcabl.integra.net (209.63.82.166)  32.474 ms  36.895 ms
 5  be1.br02.plalca01.integra.net (209.63.100.118)  24.120 ms  22.298 ms  22.176 ms
 6  peer-02.palo.twtelecom.net (198.32.175.111)  21.401 ms  22.576 ms  21.492 ms
 7  oak1-ar1-xe-0-1-0-0.us.twtelecom.net (206.222.120.214)  23.042 ms  22.441 ms  48.562 ms
 8  74.202.6.2 (74.202.6.2)  29.358 ms  32.253 ms  30.283 ms
 9  204.212.188.26 (204.212.188.26)  25.949 ms !X  30.199 ms !X *  

Update 2
Further investigation reveals that this only affects Mac Users!
2nd phone call with Veracity confirms that unusually high percentage of mac users have been reporting problems with iTunes. Level 3 techs have no idea what is causing this.

Update 3
Captured event in wireshark on 2 computers at the same time

Mac (has issue)
http://cl.ly/0o1D2r0K1s2s
Filter = "ip.dst==e3570.b.akamaiedge.net"

Windows (problem doesn't affect windows pc's)
http://cl.ly/3v3e1s2M1W27
Filer = "ip.dst==e3570.b.akamaiedge.net"
Ctrl + F "beachbody"

I don't know why the source/destination is ip.dst==e3570.b.akamaiedge.net and not "beachbody.com" or 66.208.81.68 (the beach body website ip)

Each user creates and destroys 15+ VM's per day. The VM's are created in the company's internal OpenStack cloud.

Every time a new vm is assigned an ip address which has previously been handed out, the user gets the dreaded host key verification failed error. This is because the ssh key does not match the IP address in the users known_hosts file.

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@    WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
It is also possible that the RSA host key has just been changed.
The fingerprint for the RSA key sent by the remote host is
xxxxxxxxxxx
Please contact your system administrator.
Add correct host key in /home/user/.ssh/known_hosts to get rid of this message.
Offending key in /home/user/.ssh/known_hosts:4
RSA host key for domain.com has changed and you have requested strict checking.
Host key verification failed.

The two solutions I can see are:

• Turn off strict checking - (Security risk)
• Have the users run ssh-keygen -RipAddress - (users are getting tired of this solution, since then run into it multiple times per day)

Is there any way to prevent this error message, yet stay secure? perhaps turn off security checking for just a specific subnet?