davidparks21's questions

Using a self-hosted GitLab instance I can configure the GitLab container registry under a repository successfully. The repository is part of a GitLab group.

It would be more logical, organizationally, if the containers were stored as part of the group rather than as part of a repo (which I would just dedicate to the purpose of storing docker containers for the group).

It's not clear to me if I just don't know how to configure the GitLab group to be a container registry, or whether this just isn't possible.

I'm trying to configure rclone to access a google cloud bucket (which happens to be managed by terra.bio. My rclone config reads as follows:

[terra]
type = google cloud storage
object_acl = private
bucket_acl = private
token = {"access_token":"XXXXX","token_type":"Bearer","refresh_token":"XXXXX","expiry":"XXXXXX"}

I've tried setting authenticatedRead and private but get the same result.

The error message I see is:

2022/08/13 18:24:44 Failed to copy: googleapi: Error 400: Cannot insert legacy ACL for an object when uniform bucket-level access is enabled. Read more at https://cloud.google.com/storage/docs/uniform-bucket-level-access, invalid

We've got Apache mod_proxy performing the function of a reverse proxy to a specific host:ip currently.

We would like to be able to take a url like https://domain/pathA and https://domain/pathB and look up the appropriate host:port in a table or via an HTTP rest service call. Each pathX will have a different place the proxy should get the information from.

We would like to avoid hard coding this information into the Apache config because it will change often.

I'm running a kubernetes job, deleting it, then starting it again in rapid succession.

I'm using the following command to get the pod related to my job so I can run kubectl logs $POD.

kubectl get pods -l job-name=myjob --output=jsonpath='{.items[*].metadata.name}'

But right after deleting a job the above command returns two pods, the one that has yet to be removed, and the new one.

kubectl describe jobs/myjob shows the correct pod name, I could parse it out of the text from there so I get the unique pod name for the most recently started job, but that seems hacky.

Is there a better way to get the pod name from a specific job?

I need to run NFS within an LXC container (meaning I can't use the kernels NFS driver).

Ergo I need a user space NFS server.

• UNFS3 - seems like the obvious choice, but it doesn't compile on Ubuntu 12.04 and has no support
• NFS-GANESHA - another option, but I failed to get it to compile, the list of dependencies seemed to go on indefinitely and then it finally just failed with indecipherable make errors.

Are there any options that are actively maintained and have a user community? Or that just plain work?

I'm using a dell c6100 and can connect to the web based IPMI management tool, and I can get console redirection running via the java applet.

But many keystrokes (especially enter) generate 2 or more keystrokes to the server. This is making installations almost impossible as it quickly skips past many steps.

Anyone experienced the same? Have any ideas?

I want to run 2 LXC containers under a host, and I want each to have their own IP on the network, let's say the host has 10.10.1.10 and the containers have 10.10.1.11 and 10.10.1.12.

Does my setup require that I configure 2 bridged interfaces in the hosts /etc/network/interfaces file, something like this:

auto br0
iface br0 inet static
   address 10.10.1.11 #ip of our host
   netmask 255.255.255.0 #subnet of our host
   network 10.10.1.0 #network of our host
   broadcast 10.10.1.255 #our broadcast address
   gateway 10.10.1.1 #default gw for our host to access the internet
   bridge_ports eth0
   bridge_fd 9
   bridge_hello 2
   bridge_maxage 12
   bridge_stp off

auto br1
iface br0 inet static
   address 10.10.1.12 #ip of our host
   netmask 255.255.255.0 #subnet of our host
   network 10.10.1.0 #network of our host
   broadcast 10.10.1.255 #our broadcast address
   gateway 10.0.1.1 #default gw for our host to access the internet
   bridge_ports eth0
   bridge_fd 9
   bridge_hello 2
   bridge_maxage 12
   bridge_stp off

Or is there another way to configure the host more like a switch and allow the individual containers to configure their own IPs (that would be safer so we're not always changing the interfaces file whenever we move containers around).

I have a Sonicwall NSA 3500 I'm configuring. It supports 1.5 Gbit of state-full traffic, but a single WAN port only supports 1 Gbit of traffic.

The NSA series firewalls don't support link aggregation. Could I, would I, should I configure round-robin WAN load balancing across two ports (which it does support)?

I have the ports to do it, both WAN ports would arrive at the same upstream switch which has a 10Gbit trunk to the router.

Why not right? Famous last words... so I ask...

Update

It seems that this is not possible because the firewall complains that the two WAN ports must be on different subnets. Confirmation from anyone that I understand this correctly?

I'm setting up a rack of servers which will have 2 webservers and 10 internal servers that provide back end application support (migrating from an AWS environment). We'll have virtual machine instances running on the boxes.

In most enterprise network configurations I've worked with they dual home the webservers so one NIC sits on the DMZ network and one sits on an Internal (non-internet routable) network.

Are the security benefits really sufficient to warrant two networks and dual homed hosts in the DMZ for a smaller 12-server configuration? The application is a mission critical web app.

One last benefit to consider is that separate networks reduce the risk that if the internal network were to saturate it's bandwidth, the DMZ network could be left unaffected (1Gbit interfaces, and we can get 2Gbit's into the firewall which supports 1.5Gbit statefull throughput).

So ultimately this is what we're talking about I think:

We just ordered a sonicwall NSA 4500 firewall and I am preparing a network diagram for its arrival.

• We have 12 servers in a newly setup rack, a 48-port switch, and a Sonicwall NSA 4500 firewall
• I want 2 VLANs, a DMZ for public traffic and an internal network for which the firewall will provide NAT
• The DMZ VLAN will connect to the firewall PORT 1
  • All servers in the DMZ VLAN have public internet-routable IPs (they’re webservers, each box has a 2nd NIC connected to the internal network to talk to the internal boxes, DB/Application servers)
• The internal VLAN will connect to firewall PORT 2

My question is configuring the firewall with these VLANs.

I’d like to guarantee the bandwidth to the DMZ VLAN isn’t affected by what the internal VLAN will do (the internal VLAN has the potential to saturate its 1Gbit link at times, that shouldn’t affect the DMZ-user traffic).

So from the router can we have 2 ports configured:

1. Firewall port 3 is configured with a public internet routable IP and provides the NAT translation to the internal VLAN on port 2 (these two ports are segregated off in their own VLAN on the firewall)
2. Firewall port 4 has a separate connection to the upstream gateway, but in this case I’m confused about configuring port 4, should it have an IP in this case? I think not because the DMZ VLAN boxes all are configured with their own public IP, in this case the firewall is just forwarding the traffic correct?

Am I even correct in assuming that I can/should have 2 outbound links for each of the 2 VLANs (private and DMZ)?

I'd appreciate the help on this 2nd point of confusion, my primary expertise is in application architecture rather than network architecture.

I've got a very simple java 7 app that reads from a proprietary file format (Hadoop sequencefile) sitting on disk xvdb and creates millions of 2-20kb files on disk xvdf.

  ByteBuffer byteBuffer = ByteBuffer.wrap(imageBinary, 0, ((BytesWritable)value).getLength());
  File imageFile = new File(filePath);
  FileOutputStream fos = new FileOutputStream( imageFile );
  fos.getChannel().write(byteBuffer);
  fos.close();

Running iostat -d 30 shows that we're doing more than double reading on the disk than writing. There is no other activity on this volume than the application above which is only writing to this disk.

  Device:            tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtn
  xvdap1            0.40         0.00         3.07          0         92
  xvdb             19.90       828.67         0.00      24860          0
  xvdap3            0.00         0.00         0.00          0          0
  xvdf            988.93      3538.93      1385.47     106168      41564

mount options:

  /dev/xvdf on /mnt/ebs1 type ext4 (rw,noatime,nodiratime)

I have a 3-level directory structure defined by 2 hex digits as such:

0A/FF/2B/someimagefile.gif

I have 300M small files in 1.5TB of compressed files that will populate these directories (we will have more files to come in the future, so I chose the directory structure to keep the mass of files from crashing a typical extX filesystem).

Unpacking these files moves at 1MB per second (or ~18 days to unpack). Ouchie!

I guess it was slow because I was creating the directory structure and then the files (done from Java APIs). So I set out to just create the directory structure alone in a bash loop.

The directories alone is about a 5 day task at current rate.

Any ideas on improving the speed that this moves?

UPDATE

One part of the puzzle is solved, using perl, rather than bash, creates the directories over 200 times faster, now it's an operation that give you a coffee break, not an extended weekend off.

But file creation is still extremely slow, even without needing to create the directories.

We putting up a rack of 12 (maybe 24-30 later) servers (moving off AWS) and we have only a modest level of networking skill on our team (an out of date CCNA admin).

In looking at two possible switches, I wonder if there's a reason we'd go with the recommended FastIron enterprise level switch rather than the seemingly easier to configure DLink switch listed below.

D-Link DGS-1210-48 Web Smart 10/100/1000Mbps Gigabit Switch With 4 Combo SFP Slots

  vs.

FastIron GS 648P

The D-Link switch appears to have all of the necessary features to support our need (VLAN support, trunking, the basics), and it has a remote management web-based UI, where the FastIron, I believe, only has console access (we don't have a KVM as all the servers & firewall also have an IP based management console).

Any reason to fear going with the D-Link option in this case?

I'm looking at buying a Foundary BR-FGS624P 24 port switch which will suit our purposes now, but thinking into the future I might like to stack the switch when we add a few more servers to our rack.

The FGS624P-STX model supports stacking, but I don't see many on sale at eBay, so I'm wondering if the non-stacking model can be upgraded to support stacking simply by buying the 10GiE ports (2 maximum).

Specifically I'm looking at this model:

http://www.ebay.com/itm/Foundry-BR-FGS624P-24-Port-E-FE-GE-4xSFP-Gigabit-Ethernet-Switch-RPS-FGS-600W-/390490417051?pt=US_Network_Switches&hash=item5aeb0ae79b

The show mod readout seems to suggest to me that it is stackable when/if I buy the 10GiE ports for it.

BR-FGS624P Router#show mod
    Module                                           Status Ports Starting MAC
S1: FGS-24G 24-port Management Module                OK     24    0012.f27d.6a00
S2:
S3:
S4:
BR-FGS624P Router#show ver
  SW: Version 04.3.02bT7e1 Copyright (c) 1996-2008 Foundry Networks, Inc.
      Compiled on Sep 21 2009 at 16:04:58 labeled as FGL04302b
      (3268159 bytes) from Primary FGL04302b.bin
      BootROM: Version 05.0.00T7e5 (FEv2)
  HW: Stackable FGS624P
==========================================================================
STACKID 0: SL 1: FGS-24G 24-port Management Module
      Serial  #: CH34061240
      P-ASIC  0: type D804, rev 01
==========================================================================
  400 MHz Power PC processor 8245 (version 129/1014) 66 MHz bus
  512 KB boot flash memory
 8192 KB code flash memory
  128 MB DRAM
The system uptime is 1 minutes 24 seconds
The system : started=warm start   reloaded=by "reload"

BR-FGS624P Router#show flash
Compressed Pri Code size = 3268159, Version 04.3.02bT7e1 (FGL04302b.bin)
Compressed Sec Code size = 1919636, Version 02.6.00T7e1 (/tftp/fgs02600.bin)
Compressed BootROM Code size = 416213, Version 05.0.00T7e5
Code Flash Free Space = 1048576
BR-FGS624P Router#show chassis
Power supply 1 (NA - AC - Regular) present, status ok
Power supply 2 not present

Fan 1 ok
Fan 2 ok
Exhaust Side Temperature Readings:
        Current temperature : 28.5 deg-C
        Warning level.......: 85.0 deg-C
        Shutdown level......: 90.0 deg-C
Intake Side Temperature Readings:
        Current temperature : 23.0 deg-C
Boot Prom MAC: 0012.f27d.6a00
Management MAC: 0012.f27d.6a00
BR-FGS624P Router#

My system crashed while I was in a nano session with unsaved changes.

When I log back in via SSH I see the nano process still running when I do a ps.

davidparks21@devdb1:/opt/frugg_batch$ ps -ef | grep nano
1001     31714 29481  0 18:32 pts/0    00:00:00 nano frugg_batch_processing
1001     31905 31759  0 19:16 pts/1    00:00:00 grep --color=auto nano
davidparks21@devdb1:/opt/frugg_batch$

Is there a way I can bring the nano process back under my control in the new terminal?

Or any way to force it to save remotely (from my new terminal)?

I've set up openvpn using tun. I can ping both ways between client & server. But I can't ping from the client to 8.8.4.4 (as a test).

I've added nat with:

iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE

But things don't appear to route from the tun0 interface to eth0.

tcpdump on tun0 shows that packets are coming in (in this case ping to 8.8.4.4 as a test):

12:21:55.956564 IP 10.8.0.10 > google-public-dns-b.google.com: ICMP echo request, id 512, seq 17153, length 40

But tcpdump on eth0 doesn't show anything other than my ssh traffic.

I have a ping looping from the vpn client, and I see that in my tcpdump on tun0, but I don't see anything "google" in my tcpdump on eth0.

Any thoughts? I can run this ping fine from the openvpn host. iptables are empty except for the MASQUERADE action, with defaults to accept.

I wonder if I can write a script that will monitor for a change in a file and execute some action when the change is detected.

Detailed explanation:

1. OpenVPN writes its status to a file every 1 minute.
2. I need to parse this status file and take action.
3. OpenVPN truncates the status file before writing to it.
4. I tried writing to a named pipe, but I get undesirable (but not fatal) errors in the app when it fails to truncate the pipe.

I'm trying to use the pam_exec.so PAM module to execute a script which needs to know the username/password coming from the application (OpenVPN in this case).

I have a script that executes printenv >>afile, but I don't see all the environment variables that the man pages states that pam_exec.so exports (namely PAM_USER I think), I only see the following:

PAM_SERVICE=openvpn
PAM_TYPE=auth
PWD=/usr/local/openvpn/bin
SHLVL=1
A__z="*SHLVL

I do successfully pick up the password off of STDIN and output it with this same script. But for the life of me I can't get the username.

Any thoughts on what I should try next?

I'm setting up a PAM service for OpenVPN, such that the OpenVPN PAM module can authenticate a username/password to an external service that I will implement using a shell script.

The OpenVPN PAM module states approximately the following documentation:

plugin openvpn-auth-pam.so "openvpn name USERNAME password PASSWORD"

While "USERNAME" and "PASSWORD" are special strings which substitute to client-supplied values, it is also possible to name literal values to use as PAM module query responses. For example, suppose that the login module queried for a third parameter, "domain" which is to be answered with the constant value "mydomain.com":

So now I created an openvpn PAM configuration in /etc/pam.d/openvpn:

auth required pam_exec.so expose_authtok /tmp/outputenv

Which got me as far as executing a script and getting the password from stdin, brilliant, I'm 95% of the way there. But I don't have access to the username. I've checked the environment variables and anything I can think of, but I can't work out how I could pass BOTH the username AND password to my script for external authentication.

Ideas?

I have an app that launches an authentication helper (my script) and uses STDIN/STDOUT to communicate.

I want to re-direct STDIN and STDOUT from this script to two named pipes for interaction with another program.

E.g.:

SCRIPT_STDIN > pipe1
SCRIPT_STDOUT < pipe2

Here is the flow I'm trying to accomplish:

[Application] -> Launches helper script, writes to helpers STDIN, reads from helpers STDOUT (example: STDIN:username,password; STDOUT:LOGIN_OK)
[Helper Script] -> Reads STDIN (data from app), forwards to PIPE1; reads from PIPE2, writes that back to the app on STDOUT
[Other Process] -> Reads from PIPE1 input, processes and returns results to PIPE2

The cat command can almost do what I want. If there were an option to copy STDIN to STDERR I could make cat do this with a command (assuming the fictitious option -e echos to STDERR rather than STDOUT):

cat -e PIPE2 2>PIPE1 (read from PIPE2 and write it to STDOUT, copy input, normally going to STDERR to PIPE1)