mikewaters's questions

According to the documentation that I could find, daemontools' recommended way to capture stderr (or any other file descriptor) is to redirect it to stdout. If your service has a log/run script, your daemon's stdout will be piped to it.

This obviously means that stderr and stdout are mixed in the output of the logger. Has anyone any experience with capturing stdout and stderr separately in a daemontools-managed process?

In a misguided attempt to recompile ld with debug symbols, I ended up with a /lib64 that was not symlinked to /lib (Debian 64-bit libs are in in /lib/x84_64-linux-gnu). I attempted to reinstall libc6 using apt, which errored out complaining about the above.

I (mistakenly) figured I could just mv /lib64 /tmp && ln -s /lib /lib64; the first command worked, leaving a broken system (/bin/ld not found etc.).

Is there any way to fix this in-place? (i.e. without running a rescue disk)

If I could have posted this anonymously, I would have ... [sigh]

There is a PHP5.3.3 bug that won't allow me to view php-fpm's /status page via Apache2. Is there a way to manually access this data? I cannot upgrade the box (Ubuntu 10.10, php5.3.4 not supported).

I've tried connecting using the technique here: http://inode.co.nz/testing-a-fastcgi-service, but no luck.

I issued the ^z; bg; disown sequence in order to allow me to close an ssh session in which I am running a super-important long-running process. This process writes status output to stderr, and it has continued to do so even after being detached (verified with lsof, the stderr fd is open for r/w).

Is there a way to determine that the process has indeed been disowned (will not recv SIGHUP if the shell recvs one)?

I have large quantities of structured data coming in over the network from disparate sources, and that data is archived in the filesystem, parsed/analyzed, and eventually makes it's way into a database (mysql) to be accessed by a few apps. The data is not relational-friendly, and has to be significantly massaged.

I would like to replace the 'archive to filesystem' step with some other system, one that has equivalent reliability (read: zero data loss) but has features of a database including query and api interfaces. It appears to me that one of the NoSQL products would be perfect for this, as they fulfill the query+api conditions and don't force me to shove the data into a structured format. I really like mongodb, but AFAGTM it (among other nosql products) sacrifices reliability and data integrity for performance. I was just about to set up a mongodb instance with journaling enabled, but wanted to be extra-special-super-sure before I did so.

I am not looking for a DMS or related system, as the data is more like log-ish than file-ish. And obviously I have backups etc.

TL;DR I'm looking for a NoSQL product that can act as an 'archive' in that it will never eat my data.

Thank you. Sorry for the novel.

Is there a simple (linux cli) tool that I can run continuously for a period of time to see if the there are any connection problems between two specific network endpoints over the wan?

One remote site is experiencing slowdowns/drops and I'm looking o figure out which side of the network is at fault.

EDIT

The accepted answer recommends 'mtr'; for future reference, mtr accepts the following relevant options:

--report    
    This option puts mtr into report mode. When in this mode, mtr will run for 
    the number of cycles specified by the -c option, and then print statistics 
    and exit. 
    This mode is useful for generating statistics about network quality.
    Note that each running instance of mtr generates a significant amount of 
    network traffic. Using mtr to measure the quality of your network may result 
    in decreased network performance. 

--report-cycles COUNT
    Use this option to set the number of pings sent to determine both the 
    machines on the network and the reliability of those machines. 
    Each cycle lasts one second. 

1) In your experience, does rsyslog's RELP log transfer format require a significant amount of resources over what plain TCP syslog transfer would use? (cpu, disk i/o, network)

2) IYHO, is TCP syslog transfer so unreliable?

Now that I'm asking, how would you compare these two against UDP syslog transfer?

Thanks!

I recently attempted to reconfigure an existing single-instance Postfix server (multi-homed) to support multiple MTA instances, but failed miserably (read: open relay, nastygram from ISP).

It appears that there are several methods that can be used to accomplish this, and the various (and numerous) secondary sources that I found online were 'all over the map' with respect to which they used/glued together.

Can anyone provide a working configuration (or tips) that use postmulti to manage a multi-instance Postfix setup?

I'm trying to build an application from source on a Debian machine; the source code includes a 'debian' directory with all the files required to build the package (control/rules etc.).

When I run dpkg-buildpackage, it fails because I don't have the required dependencies. OK.

So, I run dpkg-checkbuilddeps, which gives me a very clear list of what the package dependencies are.

Am I now expected to verify the (parse-unfriendly) output of that tool manually? And then install each package myself? It seems insane to me to have such high quality tools that only get you 90% to your goal, so I must be doing something wrong.

How might I get from dpkg-checkbuilddeps to 'done'?

Thank you.

Is there any benefit to using rsyslog's rotation abilities over the standard /sbin/logrotate?

I have a network of disparate remote VPS / "cloud" / colocated servers that I would like to monitor from a central location, using open source software like Zenoss/Zabbix/Nagios/OpenNMS/Opsview/Observium etc... These servers are not necessarily physically located near each other. I have a core network of servers on a reasonably fast circuit, one of which will collect the data.

I need to minimize the load on the remote servers as much as possible (some are small 256MB webservers, for instance) , so I am looking for the solution that presents the remote agent that consumes the least resources.

I would like to be able to monitor processes/network usage/logs/mail queues etc. on each device.

Given my restrictions and architecture, is there a specific product (open source) that you would recommend?

Is there a way to shrink a directory entry?

My dovecot maildir directories have increased in size from the default 4096 to several megs, and it is messing with disk quotas.

The only way that I have found, is to delete and recreate the directory. I am hoping there is a magical function somewhere - I am open to anything, except for hand-coded assembler.

Edit: For posterity, to find dirents>4096:

 find / -type d -size +4k -printf "%s\t%p\t%i\n" | sort -nr

In your experience, have you found there to be any actual problems with having redundant PTR records (same name, multiple IPs) for a given host?

Additionally, would you expect to see corresponding A or CNAME records for each PTR?

EDIT: Wow, thank all of you for the great info!

I'd like to tunnel a large amount of traffic from my local network to a server located in a datacenter, in a way that I don't have to make many changes to my local applications.

The two methods that I have come up with are 1) using a VPN (openvpn), or 2) a persistent SSH tunnel; either way I just need to change the port that my apps use and the traffic should be transparently redirected.

My questions are:

• Which one oh these methods offers the lowest (protocol/network) overhead? My bandwidth is precious.
• Are there other tunneling methods that I am not aware of?

The PE2950 Manual states:

For best system performance, all four, or eight FBDs should be identical memory size, speed, and technology

I currently have 4x2GB in slots 1-4, and would like to use 2x8GB n slots 5/6, leaving room for future upgrades.

What exactly am I sacrificing in not maintaining equality in all 8 slots?

EDIT: adding two more chips is not possible (thanks murisonc); would it be advisable to install 4x4GB (0r 8GB) chips, rather than sticking with 2GB across-the-board (as the manual recommends)?

I have, over the course of the last few days, removed several hundred megs of large text and binary files from a git repository that I had set up recently (I am relatively new to git).

I did this in order to reduce the amount of time it takes to clone this repository on a remote machine.

However, the repo itself is still huge, and it still takes forever to clone. I suspect that git is keeping some manifestation of the large files in order to maintain some history.

How might I remove these objects, to make my repository "lean and mean"?

Thanks!

The Postfix MTA consists of several components/services that work together to perform the different stages of delivery and receipt of mail; these include the smtp daemon, the pickup and cleanup processes, the queue manager, the smtp service, pipe/spawn/virtual/rewrite ... and others (including the possibility of custom components).

Postfix also provides several types of hooks that allow it to integrate with external software, such as policy servers, filters, bounce handlers, loggers, and authentication mechanisms; these hooks can be connected to different components/stages of the delivery process, and can communicate via (at least) IPC, network, database, several types of flat files, or a predefined protocol (e.g. milter).

An old and very limited example of this is shown at this page.

My question:

Does anyone have access to a resource that describes these hooks, the components/delivery stages that the hook can interact with, and the supported communication methods? Or, more likely, documentation of the various Postfix components and the hooks/methods that they support?

For example:

Given the requirement "if the recipient primary MX server matches 'shadysmtpd', check the recipient address against a list; if there is a match, terminate the SMTP connection without notice".

My software would need to 1) integrate into the proper part of the SMTP process, 2) use some method to perform the address check (TCP map server? regular expressions? mysql?), and 3) implement the required action (connection termination).

Additionally, there will probably be several methods to accomplish this, and another requirement would be to find that which best fits (ex: a network server might be faster than a flat-file lookup; or, if a large volume of mail might be affected by this check, it should be performed as early in the mail process as possible).

Real-world example:

The apolicy policy server (performs checks on addresses according to user-defined rules) is designed as a standalone TCP server that hooks into Postfix inside the smtpd component via the directive 'check_policy_service inet:127.0.0.1:10001' in the 'smtpd_client_restrictions' configuration option.

This means that, when Postfix first receives an item of mail to be delivered, it will create a TCP connection to the policy server address:port for the purpose of determining if the client is allowed to send mail from this server (in addition to whatever other restrictions / restriction lookup methods are defined in that option); the proper action will be taken based on the server's response.

Notes:

1)The Postfix architecture page describes some of this information in ascii art; what I am hoping for is distilled, condensed, reference material.

2) Please correct me if I am wrong on any level; there is a mountain of material, and I am just one man ;)

Thanks!

I have a server (debian) with two network interfaces that I would like to host multiple services and domains on; it is not entirely clear to me how the hosts file should be set up. Example:

eth0, bound to WAN interface 1.2.3.4:
   mail.example.com
   www.example.com
eth0:1, bound to WAN interface 1.2.3.5:
   www.other-domain.com
eth1, bound to LAN 192.168.1.123:
   some-clever-hostname

What should my hosts file look like? (including localhost,localhost.localdomain, etc.) Should I use DNS for some of these entries? Which ones?

Thanks!

EDIT: What if I was unable to utilize a DNS server, for instance in a testing environment?

I am looking for an MTA that will allow me (a script, really) to proactively manage its send queue in response to status codes returned by the remote servers I am delivering to.

Basically, for each mail sent I would like to be able to react to the SMTP reply code returned by the remote server, ex. '250 OK', or to any error conditions like connection timeouts.

Additionally, I would like to be able to manage the send queue moving forward based on this information, e.g. 'example.com has timed out the last 5 connection attempts, so no longer queue mail for recipients @example.com'.

I am currently using Postfix and Perl to parse its logs for this information, but I am playing a game of catchup that is prone to errors (out-of-order log entries etc.) and it's starting to get messy (some real ugly regexes ;).

I really don't want to reinvent the wheel and use some language's SMTP library; I would prefer to use a proven/fast/reliable MTA. I am however open to suggestions if what I need just isn't possible.

Thanks for your help!

Is it possible or advisable to use 'regular' not-sanctioned-by-the-server-manufacturer hardware in high end servers? Often these manufacturer-supplied parts have a very high price markup, and I wonder if it's always necessary (understanding that they probably apply more rigorous requirements to this hardware).

For example, Dell sells 300GB 15,000rpm serial-attached scsi drives for a certain server family for almost $600 each, while newegg sells a drive with the same specs for almost half the price http://www.newegg.com/Product/Product.aspx?Item=N82E16822116059.

Do we really need to pay these high markups, especially for disks that are likely RAID-ed and so guarded against catastrophic failure?