Let's say I've got 2 zpools -- one with a database's database files and one with the database transaction logs.
If I do a zfs snapshot -r database@snapshot will the resulting snapshots all be taken at exactly the same time? Or will each pool's snapshot be internally consistent but they won't be consistent across pools?
So I'm tinkering around with ZFS on linux and zrep. I've got 2 VMs on my laptop and I'm running zrep and synchronizing the contents of one filesystem to another.
One unexpected situation is this: If I'm on the "slave" -- the box that's receiving the data -- and I'm continuously reading the contents of a file (such as with sum), if the file is rapidly changing on the master I will get an Input/output error on the slave as the snapshot is getting applied. This does not happen if I'm continuously reading a file that isn't changing in the snapshot.
To be clear -- the "sum" program or any other standard userland program that is reading the changing file on the target filesystem will periodically get an Input/output error and crash.
The ZFS replication itself works correctly -- zrep is just a nice tool for managing the replication process.
I'm a little confused at this behavior -- will reads of files that get updated when a ZFS snapshot is applied cause read errors, or is this a bug in ZFS on linux?
I figured out how to set block quotas in ZFS from the documentation on oracle's site.
I didn't see if / how to setup a quota on how many inodes a user gets, though. I realize that there may not be restrictions on how many inodes a ZFS filesystem has, but when my users create a directory tree 10,000 directories deep each with 300 files, it is kind of a pain in the butt to deal with.
Bonus points if you can set it in the oracle storage appliance...
Let's say I've got a somewhat important database but one where I don't actually care about every transaction. Before you get snarky -- it is a database storing zabbix status data, so I honestly don't care if I lose a minute or two of transactions when the ascii control codes hit the TTY.
If I want to replicate that database to another host, I could do traditional database replication that essentially replays the transaction log across a pipe. There are some issues with that (like I'll have lots of writers to the primary database and the slave will only have one thread inserting the replica data), but I'd also get a "read only slave" for running my reports against. And someone else coming to town would understand what's going on. Both are valuable, I completely agree.
But -- what if I want to be clever? Being clever, I'll run my postgresql database on a ZFS filesystem with compression turned on. And now that my database is running on ZFS, can I just zfssend that stream to another box and either apply it or just archive the zfs send stream for later replay if I decide I need it?
Would the resulting database be usable after a "clean up after an unclean shutdown and replay the transaction logs" on the target machine? Let's say now I also want a read-only slave to run my reports against, could I replay the ZFS, make a snapshot, and startup the database against the snapshot?
Let's assume I have setup a pair of anycasted DNS servers on my internal network using bird and they're just talking OSPF to my core routers. I've found this on the internet and made it happen.
If I setup one to have a higher cost path than the other, sweet, I've just made a cute way to do fail-over without causing my clients any timeouts when switching from one host in their resolv.conf to the other resolver in their resolv.conf.
But that's not good enough for me. I want to distribute my load between my DNS servers. So I set the cost for both to be the same, so the routers now should be doing ECMP and sending 50% of the requests to one and 50% of the requests to the other.
But it turns out my enterprise uses TCP for a large fraction of our DNS -- let's say we've got enormous SRV records and we use DNSSEC because we're awesome.
Now, how do I setup my clever anycast setup to cope with the fact that the TCP requests need 3 or 4 packets to get a request done and now one of those 3 packets is always going to go to the wrong router and getting the tcp connection reset.
Sorry for the naive question; a quick reading of the cisco docs doesn't answer this question...
So I've got a router (say for the sake of argument a 4500 running IOS 15.x)
It has interfaces in 3 different subnets -- 10.0.0.1/24, 10.0.1.1/24, and 10.0.2.1/24 It also has a loopback address of 172.16.0.33
How do I make it so that SSH / SNMP and other administrative traffic works on the 172 address but doesn't work on the IP addresses I wish to only use for L3 forwarding?
Ideally this can be done by disabling the control plane access to these interfaces not just by using an ACL, but whatever, I don't actually care that much as long as it works...
Thanks!
I'm implementing a monitoring system for an existing modest sized data center deployment.
So far I've only gotten to the host / application side of the monitoring equation but I'm noticing what I consider to be an alarming number of Ethernet errors on various hosts. To me, alarming is 3 or 4 per day per host (some have none). When I look at the SNMP counters for the switches, I again see lots of errors on the counters but I'm not graphing those errors (yet).
In my prior environments with many more ports my error rate was approximately zero except for those hosts that had actual problems like duplex mismatches.
None of these interfaces are saturated; they're pushing approximately 40-50 megabytes / sec over gig links.
My gut feeling is that there shouldn't be any errors at all over any interface if everything is working properly but I'm worried that if I pick a fight over resolving these problems I'll just alienate everyone else who believes "it works fine; it's been working for years that way".
Anyone have some good stories / studies / statistics for when to be alarmed at ethernet errors? Or something to indicate how a small volume of errors would affect, say, an iSCSI volume?
Thanks!
Are there any good (reliable / free) ssh clients that work easily with Active Directory authenticated accounts in a Windows environment? Ideally that wouldn't need the Kerberos for Windows package?
I know there are various modified versions of putty that support GSSAPI authentication which in turn allows authentication via Kerberos to remote systems if you've authenticated via Kerberos on the local system (IE you've logged into your windows system using an Active Directory account and that same AD account exists on the remote server you're accessing via SSH)
Are there any other ssh clients that support kerberos / GSSAPI authentication using the local windows AD generated Kerberos credentials?
I found this page which has a list of GSSAPI aware windows clients but I've never used any of them. This page also seems to imply that secureCRT also supports kerberos, but again I've never used it in a kerberized environment.
Any experiences with good ssh clients? Good and free?
My experience with various modified versions of putty is that it works great when accessing Kerberized servers but if you tell it to try GSSAPI and then access a non-kerberized server, it hangs for almost a minute before attempting password authentication. It also relies on the MIT Kerberos for Windows Identity Manager, which isn't a deal breaker for me but when I try to explain how it works to others, their eyes start to glaze over as I describe all the moving parts...
Thanks!
Is there any difference between POE switches and normal switches when budgeting for cooling capacity for a network closet?
Normally, when I'm thinking about cooling; I can look at the power draw of a device and assume that for every watt I dump into the room; I'll need to figure out some way to get that watt out of the room in cooling.
But, with POE switches, they can draw an enormous amount of power, but that power isn't being consumed all in that room. Some of it is going to power the phones or APs (making cooling those "someone else's problem") and some of it is going to heat up the wires.
The question is, do I treat POE switches as though they draw a little bit more than their non-POE brethren? A lot more? Anyone have hard numbers or hard-learned experience?
It seems like no matter how strict and careful we are in our closets when we set them up, they eventually turn into a giant knot.
In part I blame the fact that we've got a rather large number of people who go into the closets and move things around, but I can't really change that.
What I can change is the way we arrange our racks and patch panels and switches.
At the moment, we've got the patch panels in a group (with wire managers between them) and a feet below them we've got our switches.
I'd like to just pull out the cable managers and put the switches directly below the patch panels, and connect the switches to the patch panels with 6 inch cables.
Are there any problems with using 6 inch patch cables?
What's the best way to poke a firewall hole in a border firewall to allow windows updates?
As far as I can tell, the windows update sites are hosted on a content distribution network that can potentially change IP addresses every 30 seconds.
If I "poison" our campus DNS with a static assignment for the sites, eventually I'll be pointing to sites that don't actually host the content anymore.
Are there any IP addresses that host the update content that are guaranteed to never change?
In a more generic sense, how do people configure firewalls to allow access to resources hosted on CDNs where the IPs are going to change constantly? The firewall just sees to/from packets and doesn't necessarily know what url (if https) the request is going to, so the firewall doesn't have a direct way to see that this packet is going to the symantec virus definitions update site while that one is looking at a world-cup stream.
And, in my situation, I don't have the ability to just mandate a specific configuration on the systems on the network. Ah, the joys of working at a university...
Most discussions of ZFS suggest that the hardware RAID be turned off and that ZFS should directly talk to the disks and manage the RAID on the host (instead of the RAID controller).
This makes sense on a computer with 2-16 or even more local disks, but what about in an environment with a large SAN?
For example, the enterprise I work for has what I would consider to be a modest sized SAN with 2 full racks of disks, which is something like 400 spindles. I've seen SAN shelves that are way more dense than ours, and SAN deployments way larger than ours.
Do people expose 100 disks directly to big ZFS servers? 300 disks? 3000 disks? Do the SAN management tools facilitate automated management of this sort of thing?
Has anyone ever used sqlite as the only database for their zabbix server?
I'll be monitoring a uplink ports on a couple hundred switches and a small number of other services such as radius or http servers.
This will be on an oldish opteron server with 2 cpus and 8gb of ram.
Anyone have any suggestions for methods of load-testing wireless networks for enterprise network deployments?
We've got a wide array of different wireless worst-case scenarios to support.
So, we're planning on buying a new generation of 802.11 network and we're not happy with our existing vendor, so it's going to be a forklift upgrade.
Is there any way of simulating these above use-cases without buying 800 laptops and hiring student droids to "pretend" to surf the internet or do homework? When we have bakeoffs, it would be great if we could do things in a controlled and repeatable environment.
I'm aware of the ixia testing devices which seem to kinda do what we want, but without expertise in how to properly do the testing, I'm not sure we'd get it right. So, in addition to alternatives to this thing (or 5 of these things randomly placed around the building), are there any firms that come out and do site-wide testing?
Also, let's say I've got a linux / freebsd box with an atheros wireless radio -- are there any good tools for inspecting existing wireless traffic to give breakdowns of spectrum utilization, packet retransmissions, and so on?
Let's assume you've got a box like one of these with space for 24 SATA disks.
What are the best bits of advice for deploying this?
For instance, should you be greedy and go for the 1.5 or 2tb disks or are they just not reliable enough to be used in an array like this and you should stick with 640gb or 750gb disks instead?
Also, I know that FC (or generically, "enterprise class") disks have a different error recovery strategy than desktop disks. An enterprise disk will fail a read quickly and report to the controller that it wasn't able to read that block, and the RAID controller will quickly regenerate the info from the parity disk and mark the block as bad. A desktop disk, on the other hand, will try and try and try again to get the data, and in pathological cases this may cause a raid controller to fail the whole disk because the read operation times out.
So there are a couple aspects to this question:
What's the best sort of disk to get today? (ie specific disks on the market in Feb 2010)
Generically, what should someone look for when trying to buy something like this that kinda walks the line between enterprise and consumer?
Lastly -- is there anything that can be done with current "consumer" disks to make them more suitable for array use? IE can you use a SMART configuration to change the error recovery strategy used by the disk?
Thanks!
One of the buildings I support recently had an adventure with a broken fire sprinkler. Lots of water everywhere.
One of the "drains" the water used was the vertical risers between network closets. The cable plant in this building has bundles of cat5e as well as conduit with bundles of multimode fiber optic cables. The fiber is standard multi strand plenum rated stuff that terminates in boxes that have the patches to the switches.
As far as I can tell, no water got near the ends of the cables (fiber or copper) but the conduit was saturated, and is likely still saturated because there isn't any air flow to dry the cables out.
My gut reaction is that while it didn't do the cables any favors, it likely also isn't going to cause any problems. A little more reading / googling around leads me to believe that the water may cause problems down the road.
Some pretty pictures so everyone knows what I'm talking about:
Fiber conduit:
Vertical riser, going down:
Vertical riser, going up:
Does anyone have any experience with this sort of damage and how to deal with it? Should we just ask the insurance adjuster to add "pull new structured cable" to the list of things to be replaced?
And, if the opinion is "replace it because it'll start failing randomly over time" please include links that describe the specific failure modes, so I've got some ammo to use with the adjuster.
Border topology for largish enterprise http://www.freeimagehosting.net/uploads/d24ede3b2f.png
Let's say you've got an enterprise with a small internet presence in the form of a DNS server, a web server, and a VPN server, all on the DMZ (in this case, an invisible pair of switches connected directly to the firewalls). There are also 20,000 nodes inside the enterprise network, spread across lots of buildings, with lots of routers and vlans and such. The enterprise is full of very important, very busy people who like to be able to get to facebook without delay... Everything gets natted; the DMZ has static symmetric NATs and everyone else gets to share outgoing IP addresses from a pool set aside for this purpose.
The border is a pair of routers connected to each other and the firewalls via a pair of gig ethernet switches, and to the ISPs via metro ethernet and are connected to the ISPs via metro ethernet. The border routers exchange routes with the ISPs via eBGP and with each other via iBGP. The interior network has diverse routes to any point in the network and uses a dynamic routing protocol to manage failover and route distribution.
The firewalls are connected to the enterprise core via a pair of routers in the datacenter.
My question is this:
Static routes or dynamic routing protocols?
static routes:
Each set of devices has static routes and uses some mechanism like VRRP or HSRP to manage the L2<->L3 failover. The firewalls would point to the virtual address for the border routers for the enterprise default route, the interior routers would point to the virtual address of the firewalls for their default route, and the firewalls would point to the virtual address of the interior routers for 10.0.0.0/8.
dynamic routing:
Use iBGP between the firewalls and the border routers and OSPF / EIGRP between the firewall and the interior routers.
I've seen that people use the static route model far more often than the dynamic routing model. My question is -- why?
What are the best practices for this sort of topology? Or is this just a religious sort of issue?
So here we have an example of why google is scared.... asking google to find the specific recipe for "what is the tcpdump incantation to sniff / filter only for ddns update packets" winds up with a billion pages of stuff not related to what I'm interested in... Lots of stuff about setting up a dns server, though.
so...
Anyone know the specific tcpdump filter you'd use to capture only dynamic dns update packets?
Wireshark and tcpdump both seem to recognize ddns update packets, (I'm using the wireshark example pcap file with ddns update packets from the wireshark wiki). So, at least I can just filter for port 53 traffic, but on this link that's going to be a metric-buttload of traffic.
Thanks! Sorry to ask a 101 type question...
Hopefully this belongs here instead of serverfault...
I've been asked to evaluate smartphones and smartphone management software, including the blackberry (which is likely the winner of the evaluation even before the evaluation gets started).
The environment will be: all data will be on an exchange 2007 server. Users will expect to manage their email, calendar, contacts, meetings, tasks, and notes between their phone and the exchange server.
Sounds good except the blackberry requires some middleware tool such as a BES server of NotifySync.
This leads me to the following question:
At work we're in the early phases of an identity management project. This is all in the context of higher ed where we have a couple thousand faculty / staff and about twenty thousand students.
Has anyone used the Sun LDAP server with an AD domain (kerberos realm) for password storage? Has anyone run an AD domain with a quarter million entries in it before?
One option we have for our identity management is to push active staff onto AD and another copy of the password / identity information to LDAP. We'd need to have a central place to change passwords if we did this so that if you change your AD (or ldap) password the change gets synchronized to the other (or they're allowed to diverge)
The other choice is to have AD be the single authority on passwords, and then we have to have principals for all affiliates (as well as all old affiliates) for a decade or two so we may have a quarter million entities in AD, of which only 20-30k would be accessed with any frequency.
Would AD explode under the load? Are there other ways to keep the passwords between Sun's LDAP and AD synchronized? What are other people's experiences?
