monster's questions

My sysadmin would like a proxy whitelist for some old CentOS sever I am trying to update (until now, it was cut off from the Internet).

I asked to have full Internet access, so I can just update everything, and install some more, and then send him the list of URLs that were needed for the update, so he can switch to a whitelist.

Unfortunately, the URLs used are not logged in /var/log/yum.log.

I came upon this question: How to know from which yum repository a package has been installed?

It tells me the "From repo", but this is just a "logical name" which can map to many URLs, and change over time, and "URL" is the "creator/source" of the package, not where it was physically downloaded from.

Is there somewhere else, where I can extract the URLs that yum used to install/update, or maybe can I configure the logging of yum to also log that information?

Currently, updating a package just produces something like this in the logs:

May 18 18:04:07 Updated: 1:busybox-1.15.1-21.el6_6.x86_64

We are replacing an Ubuntu 8.04 server with a Ubuntu 16.04. The server runs a single (non-OS) service that we need. (I'm the dev of that service, and not the sysadmin, which is on holiday this week) The service script looks about like this:

#!/bin/sh

# Load the VERBOSE setting and other rcS variables
. /lib/init/vars.sh

# Define LSB log_* functions.
# Depend on lsb-base (>= 3.0-6) to ensure that this file is present.
. /lib/lsb/init-functions


# ...

case "$1" in
  start)
    umask 002
    #...
    exit $?
    ;;

  stop)
    # ...
    exit $?
    ;;

  restart)
        stop || exit $?
        start
        exit $?
        ;;

  status)
       # ...
       exit $?
       ;;

  *)
    echo "Usage my_service (start|stop|restart|status)"
    exit 1;;
esac

After updating the path to Java ... I got the service to work. But calling:

sudo service my_service status

returns some "standard" systemd output, instead of the code in the "status)" part of the script. Same result if I do this instead:

sudo /etc/init.d/my_service status

I'm not interested in what systemd thinks I want to know/see; I just want it to execute my code instead of it's own.

The output of "status" is parsed by some web-management console, and I don't want to have to change that application to take into account which version of Linux is installed on a specific server. "status" for the same service should output using the same format (IMO), independent of the specific OS version.

How do I tell systemd to "respect" my status command, instead of ignoring it?

Searching for "systemd status" did not get me anywhere, beyond the fact that I learned that systemd seems rather complicated.

If you want failure resilience, you can build several servers, where each can server any clients, and put the data in some SAN or shared DB cluster. But if the clients connect directly to one server, and that server fails, then they get a network error. You can solve that by putting a load-balancer in front of them, either software base in a PC, or specialized hardware. In that case, the death of the individual server does not have to cause a network error in the client.

But then the load-balancer itself could die. So you're not really any better than before, right?

If I assume the DC makes sure the packets get safely to the switch(s) on which my servers are, is there any way to route traffic on one IP such that a hardware failure doesn't cause a lost connection? And could this be done without specialized hardware? Or am I missing some reason why it's not possible?

For the sake of the argument, we'll assume the (non-web) tragic is broken into messages, and that the same messages can be buffered and replayed safely, for every message that was received but wasn't acked by the failing server.

We have a cron job that sends stuff to another company using SCP. The command look like this:

 10 0 * * * ssh USER@LOCAL-SERVER 'scp -q ~/some/dir/* REMOTE_SERVER:/other/dir/' >> ~/log/some.log

This was setup by someone else. Now my problem is that my boss is on the cron mailing list, and despite the ">> ~/log/some.log" cron still sends an email every day, because even if the transfer was successful, it still outputs:

Connection to REMOTE_SERVER 12345 port [tcp/*] succeeded!

And my boss doesn't want to get that email unless an error happens.

The command is really inside a script, which I have omitted to make the make the question simpler. It runs on a RHEL 6 server. But since it's the only line where we talk to REMOTE_SERVER, then it must come from there. And the reason we perform the command on LOCAL-SERVER (Ubuntu 10.04.4 LTS) is because we are on the DB server, which cannot cross the firewall, so we first copy the data to another local server, and upload it from there.

When used without the -q, I also get the following output:

SSH Server supporting SFTP and SCP

I don't want to redirect all stderr output, because I still want to get an email when there is an error.

Until recently, we had at work a small cluster of about 20 small Windows servers (which have now all been virtualized). They were all configured to synchronize with the local time server. It was on an 1Gb sub-network in our own DC. I never got them to be less than about 100ms away from each other, which I consider to be an incredibly big difference.

Is that a normal value? What is a realistic expectation of time difference between machines running on a 1Gb network, and all connected to the same time server, and updating frequently, say every 5 minutes?

I would like to know this as setting timeouts and other parameters in a distributed application requires to take that difference into consideration.

When you install an Ubuntu server, you get a screen to choose if it should have OpenSSH, be a LAMP server, have visualization, ...

There is some console command that shows you the same menu. What is it called?

I have just spent the last 30 min googling all kind of combination of install and setup and configure and command-line without success.

So, to recap, I made a wrong choice in the installation screen, and I want to correct it without installing from scratch, and so I need that command so I can remove that what I don't want to have installed. Since I haven't explicitly chosen any package, the usual package deinstall command will not help. And since it's a server, I need to know the console command to do it.

I would like to use an application API that is not "crash safe"; in other words, there is a high likelihood of the data file being corrupt and unreadable if the application crashes.

The file itself is a "metadata file" and should not get very big: few 100s of MB maximum.

What I want to do is:

1. Force the application to access the file in "direct mode" (no OS caching).
2. Pause updates at regular "checkpoint" intervals
3. Perform a flush() (some data probably got flushed automatically)
4. Now that I know the file is consistent, clone it.
5. If there is an "old clone" delete it.
6. Resume doing changes to the original file.
7. Loop.

Could I use a special-purpose file system that makes some kind of "zero copy" of the file, combined with copy-on-write of the modified sectors of the original file, to get the clone "almost free" (with minimum disk IO)?

Also, can I do the "clone" without having to fork a process? (I don't know if the Linux file API offers a "cp" system-call).

Note: This is a "theoretical" question, as I haven't got that kind of data yet.

If you have a distributed file system spanning a dozen or more servers, and TBs of data, how do you perform backups of that? Local tape drives aren't an option as I am renting the server and don't have physical access to them. The way I see it, I simply must have a backup cluster that is proportional in size to the source cluster. Sending all of that data over the network in parallel would probably saturate it, bringing the throughput down. But the backups all have to be taken at the same time, so doing round robin backups doesn't seem to make sense. One way around this problem would be to just reserve a small portion of the large (in my case) drives, and keep the rest for rotating local LVM snapshots. Unfortunately, that kind of backup would be useless if the server get compromised. Are there any other options to create a point-in-time backup that doesn't kill the network?

[EDIT] SOLUTION:

1) Replicate the entire data set in (near) real time to one large local backup server, so the bandwidth usage, and IO, is spread over the day, and local bandwidth is usually "free".

2) Create the real backup off that machine and send it off site. If you have all the data together, it should be easy to do a differential backup, which saves billable bandwidth.

I have been researching distributed databases and file systems, and while I was originally mostly interested in Hadoop/HBase because I'm a Java programmer, I found this very interesting document about Ceph, which as a major plus point, is now integrated in the Linux kernel.

Ceph as a scalable alternative to the HDFS

There is one thing that I didn't understand, and I'm hoping one of you can explain it to me. Here it is:

A simple hash function maps the object identifier (OID) to a placement group, a group of OSDs that stores an object and all its replicas. There are a limited number of placement groups to create an upper bound on the number of OSDs that store replicas of objects stored on any given OSD. The higher that number, the higher the likelihood that a failure of multiple nodes will lead to data loss. If, for example, each OSD has replica relations to every other OSD, the failure of just three nodes in the entire cluster can wipe out data that is stored on all three replicas.

Can you explain to me why a greater number of placement groups increases the likelihood of data loss? I would have thought it's the other way around.

If you want to have a non-country-specific website in English, what is/are the best time-of-day to run the maintenance? You can also answer by specifying what is the worst time-of-day to run the maintenance, as everything else is then "better". My websites will be more toward leisure then business.

NB: I am aware that this question is subjective, as it can't be defined precisely, but the answers should still be "objective": Country name, and what makes it legally safer.

EDIT: A) I am located in Germany. B) I am NOT looking for a place to offer pirated Software/Media; no binary on my site, except "profile icon".

Hello! I want to start publishing "social" websites / apps, and I found that the biggest initial problem is this: Any and all services I have to depend on, including Domain Registrar, DNS provider, Server/Cloud Provider, CDN Provider, ... even my Insurance Agent, basically say that they can "throw me out" if my website contains "unacceptable" content. It's always phrased in such a way that basically anything can fall under "unacceptable" content. This is very frustrating because you just can't fully control what users post on your "social website", and you so you basically have to expect when you go to bed that your site is going to be gone when you wake up. I've heard a lot of horror stories about this.

Since the "Terms Of Service" of all those providers are foremost to protect themselves from legal actions, and those legal actions depend on the country where they are located, it seems like the first step is to find which country is the "safest" to locate a site. "Safest" being defined as, where I am least likely to get in legal trouble with the local authorities, if some user posts something unacceptable in some way.

The main restriction is that it should also be a "well-connected" country, because there is no point in being "safe", if my users can't get to my sites, or the latency is unacceptable. I am targeting the English speaking people in any country as my future users.