swami's questions

I need to transfer massive amounts of data to my server from different client systems around the world as fast as possible. My organisation has two redundant lines from different service providers (150 Mbps + 150 Mbps) which are not used at night. They are mapped to different public IP addresses. I have been given permission to use both lines in parallel to get the maximum possible throughput.

How do I configure the network settings (on my server) for a 2nd parallel connection? (I'm on CentOS). I'm using a java based file transfer tool (fdt) that transfers data over sockets.

The client (uploading the data) will run this tool on their system by specifying a public IP address (corresponding to Provider 1 or 2). This public IP is mapped to a private IP address on my server (by our company firewall/router). This works well. However, I now want the client to run two instances of the tool - one for each provider (thereby doubling the transfer bandwidth). We have two public IP addresses (one corresponding to each provider). So on my side (on the CentOS server), I have created another private IP address, and asked our IT team to map the 2nd public IP to that.

There will be two fdt instances running in server-mode on my server. I need each one to receive data from the respective client instances

I tried adding the 2nd IP address as an alias, with netmask = 255.255.255.255 but unable to ping it. What is the right way to achieve this.

I will be going on a 100 day (global) tour where we will be generating 1TB of video data per day. To mitigate risk of data loss we plan to drop one copy of hard disks with local volunteers in each country en route, who will upload the data from their home connections back to our server.

We don't want to open up the server to the whole world, and would like to restrict access to just those IPs that are uploading data to us, but the volunteers will mostly have dynamic IP addresses. How do we do this?

We are making a small application that will manage the data transfers, so we can do some custom logic in that.

EDIT: I just found out about Dynamic DNS services (like noip.com). Would it be fair to assume that most people's home routers would support DDNS configuration? Our infra person says that thay can configure our firewall whitelist to be based on this instead of IPs). This sounds like a simple, secure solution. Any thoughts?

In an object storage solution, how is the object data actually written to the underlying storage volumes? i.e. In what format is it written? Does it use an intermediate file system, or does it access the block storage directly? Or is it different for each implementation?

Suppose I have some data stored on one on-premise object storage platform and I want to migrate them to another, by just pointing the application at the disks - not having to copy all the data over. example: ceph to minio. Would that be possible? i.e. do they use some standardized way of storing the underlying data?

What happens if I lose the object storage middleware and I just have the physical volumes (disks). Would I be able to retrieve or understand the data in any way?

Finally, can an object storage solution be based on tape volumes (since they are lower cost than disk)? Assuming that I'm not concerned about lower latency of having to load/unload tapes.

I want to write data directly to an unmounted raw EBS volume (with no file system) using system calls.

1) Can I can do this via the EC2 instance to which the EBS is connected?

2) Can I provision raw EBS volumes without having an associated EC2 instance and read/write to them via an API?

I'm running the latest version of CentOS 7 (kernel version 3.10.0-1062.4.1.el7.x86_64) I have specified a custom firewalld zone, in which I have specified a port and a source ipset. I am expecting that both of these criteria should be applied. However, I am able to connect to the system (from an allowed source IP), but from a different port. Is this how the zone configuration is supposed to work?

Here is my zone config:

<?xml version="1.0" encoding="utf-8"?>
<zone target="ACCEPT">
  <port port="8080" protocol="tcp"/>
  <source ipset="authorized-ips"/>
</zone>

Here is my ipset config:

<?xml version="1.0" encoding="utf-8"?>
<ipset type="hash:ip">
  <!-- Bob's system -->
  <entry>192.168.123.123</entry>
  <!-- Bill's system -->
  <entry>192.168.123.124</entry>
</ipset>

If I connect using port 8008 or 8009, then I am still able to access the application via apache. But I have not explicitly allowed these ports through the firewall!

The output of running: firewall-cmd --list-all --zone=ACCEPT

Error: INVALID_ZONE: ACCEPT

I need to regularly transfer large amounts (several TBs) of video files between two servers on a 10G network. I want to decrease the overall time it takes to finish the job.

I am running CentOS7. I have two servers (A and B) each having a 10GbE card with two ports, and I have a 10GbE switch.

I have assigned IP addresses as follows:

• Host A/port 1: 172.x.1.1
• Host B/port 1: 172.x.1.2
• Host A/port 2: 172.x.2.1
• Host B/port 2: 172.x.2.2

I have created 2 VLANs (100 and 200) on the switch:

• 100: (172.x.1.1, 172.x.1.2)
• 200: (172.x.2.1, 172.x.2.2)

I want to run parallel file transfers between A and B using both ports of both the 10GbE cards simultaneously.

I have managed to make this work, however, I have another host on VLAN 100 (172.x.1.3), which now cannot be accessed from either A or B

Is there any extra configuration I need to do on the servers or the switch to make this kind of setup work?

Note: I have very fast RAID on both servers (so disk IO shouldn't be the bottleneck)

Note 2: I already tried bonding/teaming the NICs. This method doesn't work for me. All modes except balance-rr just send all packets down one connection. And I don't know if my switch will support balance-rr. Tried but no luck