I'd like to be able to grant permission to a particular Security Group to be able to create/alter the SMTP addresses for a specific mailbox. Happy for this to be done via GUI or via PowerShell. I can find nothing about this level of granularity through Googling.
Because of errors=remount-ro an underlying NFS blip caused my root mount to be remounted as read-only. I know there is nothing wrong I need to worry about. How can I remount this as rw without rebooting the server?
Some tests:
simon@steve:~$ sudo hdparm -r /dev/sda1
/dev/sda1:
readonly = 0 (off)
simon@steve:~$ sudo blockdev --setrw /dev/sda1
simon@steve:~$ sudo blockdev --setrw /dev/sda
simon@steve:~$ sudo hdparm -r /dev/sda1
/dev/sda1:
readonly = 0 (off)
simon@steve:~$ sudo hdparm -r /dev/sda
/dev/sda:
readonly = 0 (off)
simon@steve:~$ sudo mount -o remount,rw /
mount: cannot remount block device /dev/sda1 read-write, is write-protected
How can I get a view of current and average IOPS for an entire datastore in vcentre? Ideally in fact I'd like to see IOPS across an entire sDRS pool.
I'm interested to know what best practices are for setting limits and policies on an ASA to protect the device to maintain service when used as a multi tenant firewall. For example following several recent incidents of compromised servers we now have bandwidth policing and connection limits in place on all interfaces.
In this scenario it is preferable for a single tenant to hit a wall rather than all tenants go offline due to overloading or license limitations.
Edit
ASA in question is a pair of 5525-Xs in failover HA, running ASA9.0(1) but it would be good to know general answers for any ASA platform.
I'd like to have a UNIX domain socket, say /var/program/program.cmd for example, exposed via TCP, lets say on port 12345. I'd also like this to be running full time in the background.
What's the best way to do this? If its relevant the system is running Ubuntu 12.04.2.
Also with the proposed solution will it survive the domain socket being removed and recreated?
EDIT
Here is the outcome of the accepted answer in the form of an init script: https://github.com/Wirehive/haproxy-remote
This relates to multi-tenant environments such as a small hosting company.
Is Puppet (or similar) a suitable technology for taking care of basic but critical mass changes? For example:
- Updating DNS resolvers (resolv.conf)
- Setting SSH keys
- Updating NTP configuration
- Configuring snmpd
- Deploying monitoring scripts such as SNMP Perl extensions or Nagios scripts
My concerns are around security and invasiveness:
- I don't want any server to be able to see any config it shouldn't
- I'm worried that a Puppet master may be vulnerable to attack by a compromised server
- I don't want Puppet to make any changes which it shouldn't, or revert any manual changes done on the server.
I should caveat this by saying I've never used Puppet in production, only had a quick play around on a test lab, so it's possible I'm thinking about this the wrong way!
We frequently see errors like this in our backup jobs:
10/05/2013 12:21:18 :: Cannot use CBT: Soap fault. Error caused by file /vmfs/volumes/781f6afc-ea228458/VDED-CIT-010(Exchange EX01)/VDED-CIT-010(Exchange EX01).vmdkDetail: '', endpoint: ''
In the last job that ran 11 out of 235 VMs backed up had this as a warning.
As a full example here's the output from statistics for one VM:
09/05/2013 22:02:14 :: Queued for processing at 09/05/2013 22:02:14
12/05/2013 02:30:00 :: Required backup infrastructure resources have been assigned
12/05/2013 02:30:02 :: VM processing started at 12/05/2013 02:30:02
12/05/2013 02:30:02 :: VM size: 20.0 GB (3.9 GB used)
12/05/2013 02:30:02 :: Using source proxy 10.24.0.11 [nbd]
12/05/2013 02:30:36 :: Creating VM snapshot
12/05/2013 02:30:46 :: Saving '[SAN3-NFS2] VDED-IL-001(LB)/VDED-IL-001(LB).vmx'
12/05/2013 02:30:51 :: Saving '[SAN3-NFS2] VDED-IL-001(LB)/VDED-IL-001(LB).vmxf'
12/05/2013 02:30:56 :: Saving '[SAN3-NFS2] VDED-IL-001(LB)/VDED-IL-001(LB).nvram'
12/05/2013 02:31:01 :: Hard Disk 1 (20.0 GB)
12/05/2013 02:31:14 :: Cannot use CBT: Soap fault. Error caused by file /vmfs/volumes/ca67d9a6-d95e1a1f/VDED-IL-001(LB)/VDED-IL-001(LB)-000001.vmdkDetail: '', endpoint: ''
12/05/2013 02:42:17 :: Removing VM snapshot
12/05/2013 02:42:21 :: Finalizing
12/05/2013 02:42:30 :: Network traffic verification detected no corrupted blocks
12/05/2013 02:42:30 :: Busy: Source 99% > Proxy 4% > Network 0% > Target 0%
12/05/2013 02:42:30 :: Primary bottleneck: Source
12/05/2013 02:42:30 :: Processing finished with warnings at 12/05/2013 02:42:30
My question is:
- What does this error mean exactly?
- If it's a problem (I think it makes VMs transfer the whole disk rather than changes, so it's not ideal) how can I fix it?
I'm trying to figure out what component of vSphere or other VMWare product is used to manage automatic scaling of resources for individual VMs in the way that these two providers describe:
http://vps.net/cloud-servers/flexible-cloud
I've seen you can run triggered workflows with orchestrator, and you can trigger new vApp deployments with vFabric, but they are both outside of an individual VM as far as I can tell.
I'd like to set up an IPv6-in-IPv4 SIT tunnel server. I've combed through the Ubuntu IPv6 wiki page but everything is geared towards connecting to a tunnel as a client, rather than hosting one. I'd like to use 6in4 as it's quite simple/versatile and seems to negotiate most home routers/firewalls without issue.
This will be primarily used by our sysadmin staff to access IPv6 from home on DSL/Cable connections that don't natively support it. The reason I'd like to run this myself is twofold:
- I find the likes of Hurricane Electric tunnels to be quite slow and occasionally unreliable.
- I don't like the idea of our traffic going off through HE's network. We have our own core transit network and IPv6 allocation from RIPE so we may as well make use of it!
This would be used by 5-10 users simultaneously.
Equipment I have at my disposal that could be used as a host:
- Cisco ISRs
- Cisco ASAs
- Juniper MXs
- Linux/Windows servers (Ideally would be Ubuntu)
Thanks in advance.
We used to keep our Nagios config up to date through SVN, with users commiting their changes and a script running every 15 minutes. The script checked for updates, then checked those updates parsed the config check, then reloaded the config. Nice and simple, did the job perfectly.
We recently moved to GIT as part of a wider migration towards GIT for other projects and I'm having difficulty reworking this script to match.
Here is the original SVN version:
cd /usr/local/nagios
RESULT=`svn update | grep Updated | wc -l`
echo $RESULT
if [ "$RESULT" != "0" ]; then
/etc/init.d/nagios reload
fi
cd -
And here is my best efforts so far with GIT:
cd /usr/local/nagios
RESULT=`git pull 2>&1 | grep Unpacking | wc -l`
echo $RESULT
if [ "$RESULT" != "0" ]; then
/etc/init.d/nagios reload
fi
cd -
The problem I'm having is I can't seem to get the output parseable so I can match against it. I thought about going a different route by examining the most recent commit in the local working copy, and then checking if the remote most recent commit was different. I can't figure out though how to get this information.
I'm pretty new to GIT and this is driving me nuts, so my huge thanks in advance for any assistance!
We have a multi-tenant email relay set up that has a transport map file that looks like this:
domain1.com smtp:mail.domain1.com
domain2.com smtp:mail.domain2.com
domain3.com smtp:mail.domain3.com
[etc]
In the event mail.domain1.com is down, email for domain1.com will be held by the postfix server until mail.domain1.com starts responding again. However we have a customer who has a backup DSL line on their site, an their email server is also available over this. How can I tell the transport to failover to a different host if the first is unavailable?
Clarification I think there is some confusion over the purpose of this setup. This postfix server is an inbound mail relay for clients who do not have AV and Spam protection on site. It is one of a pair, which are configured as the 2 MX records for these customers. They receive and clean email before forwarding it on to their local mail servers, as well as acting as a buffer in case of an outage on their end. These customers don't generally have multiple on site mail servers, they are too small hence this service. What they do often have though is a secondary connection, eg fibre and DSL, so I'd like to be able to direct the onward SMTP to their second connection should the first be unreachable.
The title is pretty self explanatory really! How can I trigger an email if physical disk health degrades? Also as a side point, how can I do similar for software raid or zfs array health?
I'm not sure if this is just a quirk of how ZFS ZPL is presenting the available space, but it seems that SNMP is reporting the free space no more than 2TB higher than the used space.
This is on a mount that has no quota or reservation set, just a simple zpool create and then filling with data over time.
The total space in that pool is actually 6TB.
I wondered if maybe it could be happening because of an integer type used in my Cacti graph template.
Any ideas?
Going to cut right to the point on this question, as I'm after as diverse range of solutions as possible so don't want to effect any opinions with the question too much.
- Client is a UK based company.
- Organisation is 95% Windows with AD
- They have an IT policy of keeping as little infrastructure "on premises" as possible, as such they have a 1Gbps line to a data centre which houses all server infrastructure.
- UK branches who can't justify a high speed link run a local server and Windows DFS for fast file access with synchronisation - works fine.
- This company have decided to open an office in Sydney, Australia.
- Currently they have 20 people in this office, as well as 1 man "presences" around the country.
- They are having issues with both latency and bandwidth accessing the UK. Typical tests from their office yield usually no greater than 4Mbps and 320ms on a good day.
- The high latency is preventing use of terminal services.
- They need access to a lot of the same data as the UK staff.
We've had quite a few ideas already, but I'd like thoughts on how the users of ServerFault would solve this problem. Feel free to ask questions :)
I've just booted to find my software RAID 5 in Ubuntu not mounting. When trying to mount it gave me an NFS error (which was confusing). I ran fsck on /dev/md0 and my screen scrolled with fixes for about an hour. It claimed to be complete, however I've mounted it and the folder structure is empty. It just has a lost+found folder containing hundreds of files like the screenshot below:
This is an entirely hypothetical question, I'm not in this situation now.
What happens to your VLAN topology and routing if you run a multi tenant network and you have more than 4096 customers?
Do you have to start "zoning" your network at the highest level in to new trees of routers/switches and start again with reusing VLAN numbers?
The Problem
We have an issue with performance on an existing platform, so I'm turning to the hive mind for a second opinion on this. The performance issue so far relates to IOPS rather than throughput.
The Scenario
A blade centre of 16 hosts, each with 64GB of RAM. (It's a Dell M1000e w/ M610s, but that's probably not relevant) 500 VMs, all web servers (or associated web technologies such as MySQL, load balancers, etc), around 90% are Linux and the rest Windows. Hypervisor is VMWare vSphere. We need to provide host HA, so local storage is out. As such the hosts just have an SD card to boot.
A bit of background thinking
At the moment we are up to 6 hosts (the blade centre will be at full capacity in a years time at current growth) and we are running iSCSI to a Dell MD3220i w/ MD1220 for expansion.
Possible options we have considered, and immediate thoughts along with them:
- Spreading the VMs across NFS datastores, and running NFS storage that meets performance requirement for up to a given number of VMs. NFS seems cheaper to scale, as well as been abstracted a bit more than block level storage so we can move it around as needed.
- Adding more MD3220i controllers/targets. We are concerned though that doing this could have a negative effect somehow in how VMWare handles having lots of targets.
- Swapping all disks from Nearline SAS to SSD. This ought to entirely solve the IOPS issue, but has the obvious side effect of slashing our storage capacity. Also it's still very expensive.
- vSphere 5 has a storage appliance. We haven't researched this much, but it must work well?
The Question
What sort of storage would you run underneath all of that? It wouldn't need to scale to another blade centre, it would just need to provide relatively good performance for all of those VMs.
I'm not looking for "Buy SAN x because it's the best" answers. I'm looking for thoughts on the various SAN technologies (iSCSI, FC, FCoE, InfiniBand, NFS, etc), different types of storage (SATA, SAS, SSD), and methodologies for handling storage for 100s of VMs (Consolidation, Separation, Sharding, etc).
Absolutely any thoughts, links, guides, pointers etc are welcome on this. I'd also love to hear thoughts on the above options we'd already considered.
Many thanks in advance for any input!
Update 5th March '12
Some fantastic responses so far, thank you very much everyone!
Going by the responses to this question so far, I'm beginning to think the following route is the way:
- Tier the available storage to the VMWare cluster and place VM disks on suitable storage for their workloads.
- Potentially make use of a SAN that is able to manage the placement of data on to suitable storage automagically.
- Infiniband looks to be the most cost effective to get the required bandwidth with the hosts at full capacity.
It definitely sounds like it would be worth making use of the pre-sales services of a major SAN vendor to get their take on the scenario.
I'm going to continue to consider this problem for a while. In the mean time any more advise gratefully received!
My organisation is in a position where we are paying quite excessive charges for single homed IP transit form our primary data centre provider. We're at the position now where we're ready to start thinking about switching to direct relationships with transit providers.
The question here goes touches on the subject slightly, but it's not very comprehensive. For example the accepted answer states Cogent and Hurricane Electric are considered cheap providers, whereas Level(3), AT&T, Telia are considered to be better. What it doesn't do is explain exactly why one provider may be better than another.
I am quite familiar with BGP, multihoming and the concept of peering and transit, however what I don't have is an understanding of why some of these providers would be better than others.
To be clear, I'm not asking for a suggestion for a provider, or any names mentioned for that matter. I'm more than happy to go off and do my own "shopping" research. I just have the problem that I need to compare apples to apples, and I don't know what to look for!
If it makes a difference, our organisation is in the UK.
I'm half way through writing a nagios script and I've hit an annoyance with SSH.
According to the man page:
-q Quiet mode. Causes all warning and diagnostic messages to be
suppressed.
Yet if I enable the quiet flag and then pass an invalid port, I still get an error:
$ ssh user@localhost -q -p test
Bad port 'test'
This is a problem, because that will make that message the first line out and that's what is grabbed by Nagios. I need to output something like "Warning|SSH error" after picking up on a != 0 exit code from ssh, but the first line I can output on is going to be line 2.
How can I make SSH TRULY quiet?
Note: I wasn't sure whether to post this question on serverfault, on superuser or on stackoverflow. I went with serverfault as the user base are probably most experienced with cli SSH and cli scripting workarounds.
I need to remotely periodically check if a Linux file system is mounted read only on lots of servers. The only tools at my disposal in this instance are the usual CLI scripting languages (BASH/Perl/Python), SSH and SNMP.
My thoughts so far are:
- Share SSH key and execute the mount command remotely, grep/awk-ing for the RO flag on the given mountpoint.
- Check via SNMP, although I'm not certain if this information is available or how easy that is.
- Attempt to SCP a file across, or remotely touch a file.
The SNMP method is the nicest/cleanest I can think of, however I'm at a loss on how to do this as CLI SNMP seems like a nightmare, especially if I need to run through a decision process to find an index based on a given mountpoint.
Ideally I'd like to pull this off using only BASH (inc. sed/awk/grep etc) and SNMP but need some guidance on where to begin! snmpd is already instaled and configured for v2 on all the servers in question.
The final use of this script will be as a Nagios command.
Many thanks in advance for any help!
Note We already have cacti/nagios set up monitoring 100s of servers, I forget to mention this when originally posting!