Hooman Bahreini's questions

I am not able to connect to my server using FTPS. I noticed that port 990 is not open. When I try to connect to my server on port number 990 I get the following:

telnet myip 990
Connecting To myip...Could not open connection to the host, on port 990: Connect failed

I am able to telnet to the same server on port 21.

How can I open port 990 on my ubuntu server?

The firewall is inactive:

ubuntu@ip: sudo ufw status
Status: inactive

This is what I get for iptables:

ubuntu@ip:sudo iptables -L
Chain INPUT (policy ACCEPT)
target     prot opt source               destination

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination

This is what I get from apparmor:

ubuntu@ip: sudo aa-status
apparmor module is loaded.
22 profiles are loaded.
20 profiles are in enforce mode.
   /sbin/dhclient
   /snap/core/11993/usr/lib/snapd/snap-confine
   /snap/core/11993/usr/lib/snapd/snap-confine//mount-namespace-capture-helper
   /usr/bin/lxc-start
   /usr/bin/man
   /usr/lib/NetworkManager/nm-dhcp-client.action
   /usr/lib/NetworkManager/nm-dhcp-helper
   /usr/lib/connman/scripts/dhclient-script
   /usr/lib/snapd/snap-confine
   /usr/lib/snapd/snap-confine//mount-namespace-capture-helper
   /usr/sbin/tcpdump
   lxc-container-default
   lxc-container-default-cgns
   lxc-container-default-with-mounting
   lxc-container-default-with-nesting
   man_filter
   man_groff
   snap-update-ns.amazon-ssm-agent
   snap-update-ns.core
   snap.core.hook.configure
2 profiles are in complain mode.
   snap.amazon-ssm-agent.amazon-ssm-agent
   snap.amazon-ssm-agent.ssm-cli
2 processes have profiles defined.
0 processes are in enforce mode.
2 processes are in complain mode.
   snap.amazon-ssm-agent.amazon-ssm-agent (814)
   snap.amazon-ssm-agent.amazon-ssm-agent (1034)
0 processes are unconfined but have a profile defined.

The server is an EC2 instance, I also have defined the following rules in the security group:

I have setup an FTPS server on ubuntu using vsftps

Here are the FTP configurations: sudo vim /etc/vsftpd.conf

listen=NO
listen_ipv6=YES
anonymous_enable=NO
local_enable=YES
write_enable=YES
dirmessage_enable=YES
use_localtime=YES
xferlog_enable=YES
connect_from_port_20=NO
# pasv_address=13.55.13.221 # <-- this is the public IP (note that this line is commented)
# pasv_promiscuous=YES
idle_session_timeout=1200
data_connection_timeout=3600
accept_timeout=500
connect_timeout=5000
allow_writeable_chroot=YES
chroot_local_user=YES
user_sub_token=$USER
local_root=/home/$USER/uploads
pasv_enable=YES
pasv_min_port=40000
pasv_max_port=50000
userlist_enable=YES
userlist_file=/etc/vsftpd.userlist
userlist_deny=NO
secure_chroot_dir=/var/run/vsftpd/empty
pam_service_name=vsftpd
rsa_cert_file=/etc/ssl/private/vsftpd.pem
rsa_private_key_file=/etc/ssl/private/vsftpd.pem
ssl_enable=YES
allow_anon_ssl=NO
force_local_data_ssl=YES
force_local_logins_ssl=YES
ssl_tlsv1=YES
ssl_sslv2=NO
ssl_sslv3=NO
require_ssl_reuse=NO
ssl_ciphers=HIGH
utf8_filesystem=YES

The Server is an EC2 instance on AWS and we are not using a NAT box.

When I try to connect the FTP server using FileZilla it works fine.

When I give the server url to the client, the client gets the following error:

Here is the Log File: Connection timed out - "connect(2) for 172.31.39.144:42176

The strage thing is that the IP shown in the error is the private IP of the EC2 instance. The Client connects to ftp.mydomain.com which resolves to the public IP of the instance. So I am not what's going wrong on the client that tries to connect to private IP of the EC2 instance? I am also not sure why FileZilla client is able to connect to the same server with no issue?

I am trying to use elasticsearch-certutil to generate a certificate

sudo /usr/share/elasticsearch/bin/elasticsearch-certutil cert -out /usr/share/elasticsearch/config/elastic-certificates.p12 -pass ""

I am getting the following error:

java.security.KeyStoreException: Key protection  algorithm not found: java.security.UnrecoverableKeyException: Encrypt Private Key failed: unrecognized algorithm name: PBEWithSHA1AndDESede

I have seen this post which recommend downgrading Java version, but I don't know how to do this?

The other suggestion is to edit: sun.security.pkcs12.PKCS12KeyStore and add the following:

keystore.pkcs12.keyProtectionAlgorithm=PBEWithSHA512AndAES_256

But again I am not sure how to do this?

This is how I installed Java on my Ubuntu server:

sudo add-apt-repository ppa:openjdk-r/ppa
sudo apt update
sudo apt install openjdk-8-jdk openjdk-8-jre

and this is my Java version: /usr/lib/jvm/java-1.8.0-openjdk-amd64

I have schedule some crontab jobs to scrape a number of websites.

I have set some cron jobs to run the scrapers at 1 AM, scraper_1 starts on 1:01, scraper_2 starts on 1:03 and scraper_3 starts on 1:05

Each scraper may take 3 to 6 mins to complete, so there are some overlapping time between running scrappers.

# start on 1:01
01 01 * * * cd /home/ubuntu/jobscrapers/scraper_1 && scrapy crawl spider_1 >> /tmp/scraper.log 2>&1

# start on 1:03
03 01 * * * cd /home/ubuntu/jobscrapers/scraper_2 && scrapy crawl spider_2 >> /tmp/scraper.log 2>&1

# start on 1:05
05 01 * * * cd /home/ubuntu/jobscrapers/scraper_3 && scrapy crawl spider_3 >> /tmp/scraper.log 2>&1

All of these scrapers are written using Scrapy and they use Selenium and Chrome Web Driver.

The code runs fine on my development machine (windows)... but recently I am getting some occasional errors on the production machine (Ubuntu)

For example an scraper run fines for some time and then it crashes with the following error:

selenium.common.exceptions.WebDriverException: Message: unknown error: session deleted because of page crash from unknown error: cannot determine loading status from tab crashed (Session info: headless chrome=86.0.4240.111) (Driver info: chromedriver=2.41.578700 (2f1ed5f9343c13f73144538f15c00b370eda6706),platform=Linux 5.4.0-1029-aws x86_64)

Is this because 2 scraper are running at the same time? Does crontab create a new thread for each scraper (webdriver)?

Updated question

The issue was that there was no space left on the server...

I realized the problem by accident, the scrapy log was not helpful. Was there other logs that I should have checked to point me to the actual issue?

I have written a small Python scraper (using Scrapy framework). The scraper requires a headless browse... I am using ChromeDriver.

As I am running this code on an Ubuntu server which does not have any GUI, I had to install Xvfb in order to run ChromeDriver on my Ubuntu server (I followed this guide)

This is my code:

class MySpider(scrapy.Spider):
    name = 'my_spider'

    def __init__(self):
        # self.driver = webdriver.Chrome(ChromeDriverManager().install())
        chrome_options = Options()
        chrome_options.add_argument('--headless')
        chrome_options.add_argument('--no-sandbox')
        chrome_options.add_argument('--disable-dev-shm-usage')
        self.driver = webdriver.Chrome('/usr/bin/chromedriver', chrome_options=chrome_options)

I can run the above code from Ubuntu shell and it execute without any errors:

ubuntu@ip-1-2-3-4:~/scrapers/my_scraper$ scrapy crawl my_spider

Now I want to setup a cron job to run the above command everyday:

# m h  dom mon dow   command
PATH=/usr/local/bin:/home/ubuntu/.local/bin/
05 12 * * * cd /home/ubuntu/scrapers/my_scraper && scrapy crawl my_spider >> /tmp/scraper.log 2>&1

but the crontab job gives me the following error:

Traceback (most recent call last):
  File "/home/ubuntu/.local/lib/python3.6/site-packages/scrapy/crawler.py", line 192, in crawl
    return self._crawl(crawler, *args, **kwargs)
  File "/home/ubuntu/.local/lib/python3.6/site-packages/scrapy/crawler.py", line 196, in _crawl
    d = crawler.crawl(*args, **kwargs)
  File "/home/ubuntu/.local/lib/python3.6/site-packages/twisted/internet/defer.py", line 1613, in unwindGenerator
    return _cancellableInlineCallbacks(gen)
  File "/home/ubuntu/.local/lib/python3.6/site-packages/twisted/internet/defer.py", line 1529, in _cancellableInlineCallbacks
    _inlineCallbacks(None, g, status)
--- <exception caught here> ---
  File "/home/ubuntu/.local/lib/python3.6/site-packages/twisted/internet/defer.py", line 1418, in _inlineCallbacks
    result = g.send(result)
  File "/home/ubuntu/.local/lib/python3.6/site-packages/scrapy/crawler.py", line 86, in crawl
    self.spider = self._create_spider(*args, **kwargs)
  File "/home/ubuntu/.local/lib/python3.6/site-packages/scrapy/crawler.py", line 98, in _create_spider
    return self.spidercls.from_crawler(self, *args, **kwargs)
  File "/home/ubuntu/.local/lib/python3.6/site-packages/scrapy/spiders/__init__.py", line 19, in from_crawler
    spider = cls(*args, **kwargs)
  File "/home/ubuntu/scrapers/my_scraper/my_scraper/spiders/spider.py", line 27, in __init__
    self.driver = webdriver.Chrome('/usr/bin/chromedriver', chrome_options=chrome_options)
  File "/home/ubuntu/.local/lib/python3.6/site-packages/selenium/webdriver/chrome/webdriver.py", line 81, in __init__
    desired_capabilities=desired_capabilities)
  File "/home/ubuntu/.local/lib/python3.6/site-packages/selenium/webdriver/remote/webdriver.py", line 157, in __init__
    self.start_session(capabilities, browser_profile)
  File "/home/ubuntu/.local/lib/python3.6/site-packages/selenium/webdriver/remote/webdriver.py", line 252, in start_session
    response = self.execute(Command.NEW_SESSION, parameters)
  File "/home/ubuntu/.local/lib/python3.6/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
    self.error_handler.check_response(response)
  File "/home/ubuntu/.local/lib/python3.6/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: Chrome failed to start: exited abnormally
  (unknown error: DevToolsActivePort file doesn't exist)
  (The process started from chrome location /usr/bin/google-chrome is no longer running, so ChromeDriver is assuming that Chrome has crashed.)
  (Driver info: chromedriver=2.41.578700 (2f1ed5f9343c13f73144538f15c00b370eda6706),platform=Linux 5.4.0-1029-aws x86_64)

Update

This answer help me solve the issue (but I don't quite understand why)

I ran echo $PATH on my Ubuntu shell and copied the value into the crontab:

PATH=/home/ubuntu/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
05 12 * * * cd /home/ubuntu/scrapers/my_scraper && scrapy crawl my_spider >> /tmp/scraper.log 2>&1

Note: As I have created a bounty for this question, I am happy to award it to any answer which explains why changing the PATH solved the issue.

I have setup an AWS EC2 instance (Ubuntu Server).

This is the content of /etc/resolv.conf

nameserver 127.0.0.53
options edns0
search ap-southeast-2.compute.internal

now I connect my EC2 instance to a VPN, using OpenVPN. OpenVPN make the following changes to /etc/resolv.conf

#
# Generated by OpenVPN 3 Linux (NetCfg::DNS::ResolvConfFile)
# Last updated: 2020-08-25 21:39:32
#
search somenetwork.com ap-southeast-2.compute.internal

# OpenVPN defined name servers
nameserver 121.98.0.1
nameserver 20.92.1.1
nameserver 20.92.1.2
nameserver 20.92.1.3

# System defined name servers
nameserver 127.0.0.53

# Other system settings
options edns0

After connecting to the VPN, my internet stops working. How can I resolve this issue?

Update:

As suggested in the comment by Thomas Ward, I connected to the VPN (which adds the above named server lines to resolv.conf) and then I am not able to browse the internet... but I can still ping 8.8.8.8. If I comment out the lines from resolv.conf then I can start browsing the internet.

I also tried the suggestion in the answer by Ravindra Bagale and ran the command:

ls -la /etc/resolv.conf

this is the output:

-rw-r--r-- 1 openvpn openvpn 244 Oct 10 05:05 /etc/resolv.conf

I want to move all the files from parent folder to child folder... and ignore the existing files in the child folder, i.e. if img1.jpg exists in both parent and child folder, it should not be replaced in child folder.

I can move the files like below:

mv ./parent/*.jpg ./parent/child/

but don't know how to tell Ubuntu to ignore the existing files?

I am using s3fs to mount an s3 bucket on my Ubuntu server:

sudo apt-get install s3fs

sudo vim /etc/passwd-s3fs // <--put user access key and secret key in this file

sudo chmod 640 /etc/passwd-s3fs // <-- change permission to password file

sudo vim /etc/fuse.conf // uncomment user_allow_other

Now mount the s3 bucket:

sudo s3fs -o allow_other s3-bucket-name /home/myuser/s3

I want to make sure that the bucket is mounted automatically after reboot, how can I do this?

I am learning bash and am not able to understand what is going wrong with the output redirection in the following example:

I have a file called myfile.txt with the following content.

Practice makes Perfect

I am going to use tr command to replace P with p:

cat myfile.txt | tr P p

This does what I want, now I am going to put the result back into the original file:

cat myfile.txt | tr P p > myfile.txt

But after executing the above command myfile.txt is empty... why is this happening?

Update:

If I send the output to a different file, then it works as expected:

cat myfile.txt | tr P p > anotherfile.txt

I want to download and install fuse, I am following this tutorial

Download and compile latest version of fuse source code. For this article, we are using fuse version 3.5. Following the set of command will compile fuse and add fuse module in the kernel.

cd /usr/src/
wget https://github.com/libfuse/libfuse/releases/download/fuse-3.5.0/fuse-3.5.0.tar.xz

I have executed the above 2 commands, but seems that it has not worked - output:

HTTP request sent, awaiting response... 302 Found
Location: https://github-production-release-asset-2e65be.s3.amazonaws.com/48296177/f5f40080-6080-11e9-831e-924025a26137?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20190819%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20190819T032718Z&X-Amz-Expires=300&X-Amz-Signature=9daad2fc48ac63c665e7bb3c4af49d43b4d9f04170b13e31cd134f8f4bf37c72&X-Amz-SignedHeaders=host&actor_id=0&response-content-disposition=attachment%3B%20filename%3Dfuse-3.5.0.tar.xz&response-content-type=application%2Foctet-stream [following]
--2019-08-19 03:27:18--  https://github-production-release-asset-2e65be.s3.amazonaws.com/48296177/f5f40080-6080-11e9-831e-924025a26137?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20190819%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20190819T032718Z&X-Amz-Expires=300&X-Amz-Signature=9daad2fc48ac63c665e7bb3c4af49d43b4d9f04170b13e31cd134f8f4bf37c72&X-Amz-SignedHeaders=host&actor_id=0&response-content-disposition=attachment%3B%20filename%3Dfuse-3.5.0.tar.xz&response-content-type=application%2Foctet-stream
Resolving github-production-release-asset-2e65be.s3.amazonaws.com (github-production-release-asset-2e65be.s3.amazonaws.com)... 52.217.36.188
Connecting to github-production-release-asset-2e65be.s3.amazonaws.com (github-production-release-asset-2e65be.s3.amazonaws.com)|52.217.36.188|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1021676 (998K) [application/octet-stream]
fuse-3.5.0.tar.xz: Permission denied

Cannot write to ‘fuse-3.5.0.tar.xz’ (Success).

I cannot see any code being downloaded in: /usr/src

I have added the following crontab job:

sudo crontab -e

58 * * * * ubuntu /home/ubuntu/backup/mysqlbackup.sh
>/home/ubuntu/backup/log/backup.log

I have tried both root and ubuntu users.

When I run the command:

sudo crontab -l

And when I run:

systemctl status cron

But mysqlbackup.sh is not executed. Any idea what is the problem or how can I diagnose what is going wrong?

Note I have added only minutes to test the script

I have created the following shell scrip to backup MySQL DB, zip it, and then copy it to an s3 bucket:

#vim /home/ubuntu/backup/mysqlbackup2.sh

#!/bin/bash
## Backup mysql DB, zip it and then copy it to s3 bucket

mysqldump -hhostname -uusername dbName -p'p@ssW0rd' > db.sql

if [ $? -le 1 ] 
then
    # zip the file and copy it s3 bucket
    sudo gzip -9 db.sql
    s3cmd put db.sql.gz s://mys3bucket/
else 
    echo "Fail to backup MySQL DB" >> "backup.log"
fi

It does everything fine and the backup is copied to s3 bucket. But I cannot understand the output of shell script:

I understand the password warning but why does it show: '[' 0 -le 1 ']'? Is there anything wrong in my if condition?

I want to create a shell script to automatically backup MySQL DB. I will later copy it to an s3 bucket.

I have created the following shell script:

#vim /home/ubuntu/backup/mysqlbackup.sh

#!/bin/bash
## Specify the name of the database that you want to backupbackup

# Database credentials
USER="user1"
PASSWORD="password"
HOST="hostname.compute.amazonaws.com"
DB_NAME="db1"

#Backup_Directory_Locations
BACKUPROOT="/home/ubuntu/backup"
TSTAMP=$(date +"%d-%b-%Y-%H-%M-%S")
S3BUCKET="s3://s3bucket"
#LOG_FILE="/home/ubuntu/backup/log/dump.log"

mysqldump  -h <HOST>  -u <USER>  --database <DB_NAME>  -p"$PASSWORD" > $BACKUPROOT/$DB_NAME-$TSTAMP.sql

Then from the command line, I run the script:

sudo bash -x ./mysqlbackup.sh

And it fails, telling me:

HOST: No such file or directory

I have followed the instructions here to set up a Galera cluster. The instruction says I need to disable appArmor:

Disabling AppArmor

---

By default, some servers—for instance, Ubuntu—include AppArmor, which may prevent mysqld from opening additional ports or running scripts. You must disable AppArmor or configure it to allow mysqld...

So I followed the instructions and executed, the following commands:

sudo ln -s /etc/apparmor.d/usr /etc/apparmor.d/disable/.sbin.mysqld
sudo service apparmor restart

I have completed the cluster configuration. But I am not sure if I have correctly disabled the AppArmor, because when I run:

sudo aa-status

I get:

... some more output here
2 processes have profiles defined.
1 processes are in enforce mode.
   /usr/sbin/mysqld (1938)
1 processes are in complain mode.
snap.amazon-ssm-agent.amazon-ssm-agent (1295)
0 processes are unconfined but have a profile defined.

I see mysqld is in enforce mode... what does this mean? Does it mean AppArmor is disabled for MySQL? Is it possible to disable AppArmor all together?