Home / ubuntu / Questions / 1038403
Accepted
Ali Majed HA
Asked: 2018-05-21 06:41:14 +0800 CST

Problem in reading from socket

  • 772

I have a spark streaming program, that reads data from a socket I have craeted using:

nc -lk 9999

The program reads the data from socket and exclude the "Error" messages. When I write manually in socket, it works fine.
I have created a python script that prints "Error" messages frequently. I will save the result in a file using:

stdbuf -oL python my_script.py &>> my_file.txt

and read the file from the socket:

nc -lk 9999 | tail -f my_file.txt

Every thing is ok, the socket will read data from file while the file is being updated on the background, But the problem is that my spark program doen't capture the "Error" messages.
As a summary: when I write manually "Error" messages in socket, spark capture them, But it won't capture "Error" message generated by python script from socket.
Actually the program doesn't work if I read file from socket instead of typing in it.
What is the difference?

command-line python files sockets

1 Answers

  1. Best Answer

    The command you typed

    nc -lk 9999 | tail -f my_file.txt

    means: Take the output of netcat and pipe that to tail -f my_file.txt. But tail doesn't accept any input, it merely watches the file my_file.txt. Try

    tail -f my_file.txt | nc -lk 9999

    instead, so that the output of tail is fed to nc.

    • 1