how to unique a file row-wise?

Question

[sharmatina189059]

Hello all I have a input file like this:

3 0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  786 0  0
19 0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  787 0
1 0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  786 0  0  0  0
0 9 0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  787 0  0  0  
11 0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  786 0  0  0  0

I want to make it unique row-wise. The output , i want to get should look like

3 786 0
19 787 0
1 786 0
9 787 0
11 786 0

Can anybody help me out?

[finswimmer]

Hello sharmatina189059,

how is this question related to bioinformatics? Is the order of the numbers per row in the output important? If so, how is the relationship to the input?

fin swimmer

[sharmatina189059]

It is actually a frequency table I got it using Prophecy (Emboss module). I need to check the number of isolates in which these variations exist. That's why I want to make it sort so that I can read it easily. Can you please help? I could not understand how to make it unique row-wise?

Answer 1

[DareDevil]

use awk

awk '{ n=split($0,a,FS); $0=""; j=1; delete u; for (i=1; i<=n; i++) if (!u[a[i]]++) $(j++) = a[i]; print }' < file.txt

Using while loop

while read -r line; do echo "$line" | grep -o '[^ ]*' | sort -h | uniq | paste -s; done < file.txt

Answer 2

[shenwei356]

Using GNU datamash and parallel.

Sending every input line to parallel, which calls sed for converting space-delimited values to tab-delimited, and datamash for transposting column-wise data to row-wise, sort -u is for sorting and saving unique values.

cat t.txt \
    | parallel --pipe --recend "\n" -N 1 -k  \
        'sed -r "s/\s+/\t/g" | datamash transpose | sort -u  | datamash transpose '
0       3       786
0       19      787
0       1       786
0       787     9
0       11      786

[sharmatina189059]

Thanks for the reply

perl -MList::Util=uniq -alne 'print join " ", sort { $a <=> $b } uniq @F' out_new_prophecy > perl_out_prophecy

This also works!!