Home / ubuntu / Questions / 472202
Accepted
Abdennour TOUMI
Asked: 2014-05-26 19:11:03 +0800 CST

Get intersection between two files using command line

  • 772

What is the most efficient way to get the intersection between two files , there is : 

sort file1 file2 | uniq -d
grep

3 Answers

  1. Best Answer

    Use grep command line as following: 

    grep -Fx -f file1 file2
    • 3

  2. If you want to combine the contents of two files then run the below command,

    awk 'NR==FNR' file1 file2
    • 0

  3. Let file1 = 'A\nA' and file2 = 'A\nA\nA'. Is the intersection just 'A' or 'A\nA'? In other words, is the number of times the same line is appears in both the files relevant?

    If it is just 'A' this will work:

    sort <(sort file1 | uniq) <(sort file2 | uniq) | uniq -d

    The problem with the approach given in the question is that even if a line doesn't exist in both files, if it is repeated in one it will appear in the result. So, we need to remove repetitions within files first using sort fileX | uniq.

    If it is 'A\nA' then this will work:

    sort <(sort file1 | awk '{if ($0 == prev) c++; else c=0; print c, $0; prev = $0;}') <(sort file2 | awk '{if ($0 == prev) c++; else c=0; print c, $0; prev = $0;}') | uniq -d | cut -d' ' -f2-

    I can think of a few more one liners (with awk, sort, uniq and cut) but none of them is much simpler than this. This, of course, doesn't mean there aren't any simpler solutions.

    • 0