knorv's questions

In a production MySQL environment the following error message is written to /var/log/mysql/error.log every fourth minute:

110723 18:36:02  InnoDB: ERROR: the age of the last checkpoint is 9433856,
InnoDB: which exceeds the log group capacity 9433498.
InnoDB: If you are using big BLOB or TEXT rows, you must set the
InnoDB: combined size of log files at least 10 times bigger than the
InnoDB: largest such row.

I'm not sure how to parse the error message.

More specifically I don't understand how the age of the checkpoint relates to the "log group capacity", and how that in turn relates to the size of rows with large BLOB/TEXT columns.

Basically I'm lost on how to troubleshoot and proceed fixing this problem. My questions are:

• What does the error message mean?
• What am I doing wrong?
• How can I fix it?

I have an application writing to an ext3 directory which over time has grown to roughly three million files. Needless to say, reading the file listing of this directory is unbearably slow.

I don't blame ext3. The proper solution would have been to let the application code write to sub-directories such as ./a/b/c/abc.ext rather than using only ./abc.ext.

I'm changing to such a sub-directory structure and my question is simply: roughly how many files should I expect to store in one ext3 directory while still getting acceptable performance? What's your experience?

Or in other words; assuming that I need to store three million files in the structure, how many levels deep should the ./a/b/c/abc.ext structure be?

Obviously this is a question that cannot be answered exactly, but I'm looking for a ball park estimate.

Given a Linux machine with IPv4 connectivity but no native IPv6 connectivity, what is the easiest way to setup free IPv6 tunneling?

Which tunneling providers are recommended and which steps are needed in the setup?

What is the correct way to enable IP-forwarding in MacOS X?

More specifically, what is the MacOS X command line quivalent of the following two Linux commands:

echo 1 > /proc/sys/net/ipv4/ip_forward
iptables -A FORWARD -j ACCEPT

Assume two IP-addresses in the same /24 network (say 1.2.3.*):

• 1.2.3.X
• 1.2.3.Y

Is it safe to assume that these two hosts belong to the same AS-number? If not, roughly how often is that the case that they do not belong to the same AS?

What are useful rule-of-thumbs with regard to interpreting non-zero vmstat "bi/bo" values? When are the values "okay" and when are they an indication of that the system is under heavy stress?

The question is of course hard to answer in the general case, but I'm looking for rule of thumbs rather than exact rules.

Given access to separate two Xen instances running Linux, is there some way to identify if the two instances are running on the same physical machine?

What are useful rule-of-thumbs with regard to interpreting non-zero vmstat "si/so" values? When are the values "okay" and when are they an indication of that the memory situation is problematic?

The question is of course hard to answer in the general case, but I'm looking for rule of thumbs rather than exact rules.

The Resin configuration (resin.xml) pasted below achieves the following:

1. Binds the built-in http server to port 8000
2. Sets the desired maximum memory allocation (-Xmx512m)
3. Configures the webapp /path/web/root accessible from http://domain.com and http://(www|www1|www2).domain.com
4. Sets up access logging to /path/to/logs/access.log

This is the Resin configuration:

<resin>
  <cluster id="app-tier">
    <server-default>
      <!-- #1 -->
      <http port="8000"/>
      <!-- #2 -->
      <jvm-arg>-Xmx512m</jvm-arg>
    </server-default>
    <!-- #3 -->
    <host id='domain.com' root-directory="/path/web/root">
      <web-app id="/" />
      <!-- #3 -->
      <host-alias-regexp>(www|www1|www2).domain.com</host-alias-regexp>
      <!-- #4 -->
      <access-log path="/path/to/logs/access.log" />
    </host>
  </cluster>
</resin>

I'm switching from Resin to Tomcat and my question is hence:

• What is the "best practice configuration" for Tomcat to achieve the four things outlined above?

In Red Hat the file /etc/redhat-release contains the operating system version data (such as "Red Hat release X.X (Final)").

Is there any similar mechanism in Mac OS X that can be used programatically to get the operating system version data (such as "Mac OS X 10.5.8")?

Consider the following lines from a "ps auxwww" output:

USER       PID  %CPU %MEM      VSZ    RSS   TT  STAT STARTED      TIME COMMAND
root      4262   0.0  0,1    76592   1104 s005  Ss   10:02am   0:00.03 login -pf yo
yo        4263   0.0  0,0    75964    956 s005  S    10:02am   0:00.03 -bash

How do I force ps to expand all commands in the COMMAND column to their fully qualified path names? I want login to be resolved to /usr/bin/login and Bash to /bin/bash.

Is there an equivalent to procfs in Mac OS X? That is - is there a file-based mechanism to easily obtain process information?

I've recently installed a CentOS 5.3 machine which I'm locking down for server usage in a headless environment (no GUI will be used on the machine). The server will be used as a combined web- and database server.

I've disabled xfs and portmap since these will clearly not be needed on the machine.

Below is a chkconfig log which shows the services running on the machine.

Question: Beyond xfs and portmap - which of the services below would you consider disabling? Why?

chkconfig --list | grep 3:on

acpid           0:off   1:off   2:on    3:on    4:on    5:on    6:off
anacron         0:off   1:off   2:on    3:on    4:on    5:on    6:off
atd             0:off   1:off   2:off   3:on    4:on    5:on    6:off
auditd          0:off   1:off   2:on    3:on    4:on    5:on    6:off
autofs          0:off   1:off   2:off   3:on    4:on    5:on    6:off
crond           0:off   1:off   2:on    3:on    4:on    5:on    6:off
haldaemon       0:off   1:off   2:off   3:on    4:on    5:on    6:off
ip6tables       0:off   1:off   2:on    3:on    4:on    5:on    6:off
iptables        0:off   1:off   2:on    3:on    4:on    5:on    6:off
irqbalance      0:off   1:off   2:on    3:on    4:on    5:on    6:off
kudzu           0:off   1:off   2:off   3:on    4:on    5:on    6:off
mcstrans        0:off   1:off   2:on    3:on    4:on    5:on    6:off
messagebus      0:off   1:off   2:off   3:on    4:on    5:on    6:off
microcode_ctl   0:off   1:off   2:on    3:on    4:on    5:on    6:off
netfs           0:off   1:off   2:off   3:on    4:on    5:on    6:off
network         0:off   1:off   2:on    3:on    4:on    5:on    6:off
readahead_early 0:off   1:off   2:on    3:on    4:on    5:on    6:off
restorecond     0:off   1:off   2:on    3:on    4:on    5:on    6:off
sshd            0:off   1:off   2:on    3:on    4:on    5:on    6:off
syslog          0:off   1:off   2:on    3:on    4:on    5:on    6:off
yum-updatesd    0:off   1:off   2:on    3:on    4:on    5:on    6:off

After installing Tomcat 5 what configuration changes/tweaks do you apply, and why?

Be specific: name what files you edit, what settings you change and why you do it.

Some areas of interest:

• General configuration changes
• Best practices with regards to "virtual host handling"
• Performance tweaks
• Stability tweaks
• Security settings

I'm considering adding a cronjob running yum -qy update on a regular basis on some machines that do not get regular maintenance. The goal would be to keep the machines up-to-date with regard to security patches that would otherwise be applied too late. I'm only using the CentOS base repositories.

Questions:

• In your experience - how "safe" would this approach be? Should I expect failed updates once in a while? Roughly how often would this approach require reboots?
• Pros/cons or other gotchas with this approach?
• How are you keeping your machines up-to-date using automation?

Consider the following five machines (named c, h, j, s and u) and their respective vmstat 1 outputs.

Given the information from the vmstat output:

• Which machines would you consider being in a good shape? Why? What's your reasoning?
• Which machines would you consider being in a bad shape? Why? What's your reasoning? How would you go about fixing it? Upgrade memory, CPU, IO-subsystems or something else?

I'm trying to learn how to get an intuitive feel for interpreting vmstat output.

Machine c:

 # vmstat 1
 procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------
  r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
  1  0      4   9456  10692 304020    0    0   193    92  310  131  7  1 80 11  2
  2  0      4  10212  10676 303020    0    0   412    40 1265  514 11  0 80  8  2
  1  1      4   9664  10684 303328    0    0   320    60 1311  545  7  0 84  7  2

Machine h:

 # vmstat 1
 procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------
  r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
  0  2 892948  10084  11352 140248   14    9    67   113  123  184  1  0 81 13  4
  1  1 892948   9828  11360 140420   80    0   232    56 1436  839  1  0 74 24  2
  0  1 892948   9424  11368 140596   88    0   192   780 1405  564  1  0 74 23  2

Machine j:

 # vmstat 1
 procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------
  r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
  0  2    300 152276   7432 616948    0    3   922   795   38   29  8  1 65 25  0
  0  0    300 152096   7436 617136    0    0   184     0  243  421  2  1 77 20  0
  0  0    300 152096   7436 617136    0    0     0    28  160  285  0  0 100  0  0

Machine s:

 # vmstat 1
 procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------
  r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
  0  0  57748 213168  17672 1282888    0    0    23     6    0    0 11  1 79  9  0
  0  0  57748 213224  17676 1282900    0    0    24     0   33  503  1  0 98  1  0
  0  0  57748 213224  17676 1282900    0    0     0     0    7  254  0  0 100  0  0

Machine u:

 # vmstat 1
 procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------
  r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
  0  3 452708   8860   2724 266032   34   16   248   244    5   15  5  3 72 19  1
  0  2 452708  62240   2776 211880  292    0  1952     0  625  749  9  2 70 18  1
  1  1 452708  63752   2832 212320  228    0   816     0  745  821  4  0 67 28  1

How do I configure Varnish so that all responses to requests matching say "*.css" gets cached in the client's browser for 30 days?

What I'm trying to achieve is making Varnish set the correct "Expires:" and/or "Cache-Control" settings so that all CSS:s are cached for 30 days regardless of what the backend says about the client-side cacheability of these objects.

This is a question asked to people who have used OurDelta in production:

• Why did you choose OurDelta over the standard MySQL distribution?
• Pros/cons of OurDelta compared to the standard MySQL distribution?