Florin Andrei's questions

We have several producers and several consumers connected via Kafka - basically, it's batch processing jobs that are created on demand, placed on Kafka in several queues, and the batch processors pick them up via Kafka and process them one by one.

I want to visualize and monitor the lengths of these queues in Kafka. The queue lengths will serve as proxies for the "load" of the overall system. The more jobs waiting in queues, the more "loaded" the system is.

Our Kafka is an AWS MSK cluster. I've enabled Prometheus JMX monitoring, and I'm scraping all metrics every 10 seconds.

Looking at the metrics, there's nothing that appears immediately obvious in terms of queue length. Is that parameter exposed as a metric by default?

If queue length is not exposed by default, what is a good way to collect that metric? Assume I can write a Python script with any libraries installed, and I have full access to the Kafka endpoints from it.

Note: I understand the basic concepts, but I don't have a lot of practical experience with Kafka (I've only used RabbitMQ before), so apologies if my vocabulary is not very precise. E.g. what I called "queue" is apparently called "topic".

We have a bunch of files that contain sensitive information, that need to be accessed by several teams, but need to be unreadable to everyone else. Currently, each team member needs to generate their own GPG key pair and share their public key with the team.

Then whoever has access to the files will add the new person's key to the relevant files. This is a fiddly and time-consuming process. Same when a key needs to be removed from files (say, the person has left the company).

This could be automated, but I do not want to reinvent the wheel. Are there any tools that automate the process I've described?

The goal is to have a data link between VPCs in different regions, and the link should keep working even when an availability zone in one VPC or another fails.

Having a simple VPN tunnel with one VPN instance at each end would not work; if the wrong AZ fails at one end, the tunnel is down.

I was hoping Amazon would allow you to create VPN tunnels from one VGW to another, but that does not appear to be possible yet. Too bad, because it would be an easy way to have a fully redundant interconnect option between VPCs in different regions.

There's also Direct Connect which may allow some redundancy (at least at the AWS level) if properly setup. But it takes time to build a DC link, and it's pretty expensive. It's not something you would do just to test an idea.

In theory, having a pair of VPN instances at each end, with BGP and some kind of heartbeat might work, but the setup must be pretty complex (the instances would have to monitor each other and push route changes into the VPC). Are the any HOWTOs on this topic?

Other ideas?

We're looking at a long-term migration towards the cloud. The plan is to start small, and gradually move less essential parts of the infrastructure into the cloud. All good so far.

Part of this migration includes log files from web servers and whatnot. Keep in mind that the servers are still in datacenters outside the cloud. It should be easy to have a cron job grab log files at the end of each day, compress them, and shove them into Amazon S3, with a possible backup to Glacier. That's easy.

The problem occurs when S3 is the only place where you store the logs, and you want to search the logs for various events. If you don't know the time interval, you may have to download all logs from S3 for a comprehensive search, and that turns out to be expensive - moving your data into the cloud is cheap, getting it out of there is expensive.

Or I could setup an EC2 instance template. When someone wants to do a log search, fire up the instance, download the logs from S3 to it, and grep away. Downloading files from S3 to EC2 is cheap. But the download may take a while; also, again, if you don't know what you're looking for, you need to download a lot of logs, which means using lots of space in EBS.

Another way is to upload logs into DynamoDB or something. Price might be an issue. Another issue is that the logs are completely unstructured Apache and Squid logs, and the like - and so queries might take a very long time.

We're talking 500GB / year of compressed logs, storing up to 5 years.

To me, storing logs in the cloud like this starts to sound like a not very good idea. Maybe just use Glacier as a "tape backup", but keep the logs locally for now, on a couple hard drives.

Which way do you lean?

I need to backup data and config files on this server, daily. I need to keep:

• daily backups for a week
• weekly backups for a month
• monthly backups for a year
• yearly backups after that

All of this accomplished via a shell script run daily from cron.

This is how the backup files should look after 10 years of running:

blog-20050103.tar.bz2
blog-20060102.tar.bz2
blog-20070101.tar.bz2
blog-20080107.tar.bz2
blog-20090105.tar.bz2
blog-20100104.tar.bz2
blog-20110103.tar.bz2
blog-20120102.tar.bz2
blog-20130107.tar.bz2
blog-20130902.tar.bz2
blog-20131007.tar.bz2
blog-20131104.tar.bz2
blog-20131202.tar.bz2
blog-20140106.tar.bz2
blog-20140203.tar.bz2
blog-20140303.tar.bz2
blog-20140407.tar.bz2
blog-20140505.tar.bz2
blog-20140602.tar.bz2
blog-20140707.tar.bz2
blog-20140728.tar.bz2
blog-20140804.tar.bz2
blog-20140811.tar.bz2
blog-20140816.tar.bz2
blog-20140817.tar.bz2
blog-20140818.tar.bz2
blog-20140819.tar.bz2
blog-20140820.tar.bz2
blog-20140821.tar.bz2
blog-20140822.tar.bz2

Running Postfix on Linux.

I know how to create IP aliases.

I know how to create multiple smtp transports and bind them to various IPs:

smtp_bind_address=X.Y.Z.Q

I know how to assign a transport to a specific recipient domain:

# cat transport
yahoo.com    yahoo:
gmail.com    gmail:

What I don't know is how to get Postfix to use multiple transports for a single recipient domain, each transport bound to a different IP. If that was possible, I could create several IP aliases, bind a transport to each, and spread delivery to a specific domain over all those aliases.

The reason I want to do this is that some ISPs are exceedingly slow to take in emails. 75% of our emails are delivered very quickly, but then there's this straggler ISP who has a lot of recipients but takes a very very long time on the intake because they are throttling everything down a lot, per sender IP.

AnyConnect works fine with Windows XP.

On Ubuntu Linux 9.10 32 bit, I downloaded anyconnect-linux-2.4.0202-k9.tar.gz, installed it, and tried to connect to the same ASA firewall. It failed with:

  >> state: Connecting
  >> notice: Establishing VPN session...
  >> error: AnyConnect package unavailable or corrupted. Contact your system administrator.
  >> notice: Connection attempt has failed.
  >> state: Disconnected
  >> state: Disconnected

Then I downloaded anyconnect-linux-2.4.0202-k9.pkg from cisco.com and installed it on the ASA's flash:

foobar# show flash
--#--  --length--  -----date/time------  path
   75  8192        Nov 12 2009 05:02:58  log
  154  4181246     Dec 31 2002 16:08:18  securedesktop-asa-3.2.1.103-k9.pkg
  155  398305      Dec 31 2002 16:08:40  sslclient-win-1.1.0.154.pkg
   79  8192        Oct 24 2009 13:15:24  crypto_archive
  157  13934592    Nov 12 2009 04:52:54  asa805-k8.bin
  158  11491880    Nov 12 2009 04:56:48  asdm-623.bin
  159  3191813     Nov 12 2009 06:41:18  anyconnect-win-2.4.0202-k9.pkg
  164  5199492     Nov 24 2009 13:58:12  anyconnect-linux-2.4.0202-k9.pkg

255426560 bytes total (216465408 bytes free)

Tried again with Linux - fails exactly the same way. The ASA log says:

Group <DfltGrpPolicy> User <foo> IP <X.Y.Z.K> WebVPN session terminated: User Requested.

I'm out of ideas. Any suggestion?