Merging multiple tsv files

Question

[Nobody]

Hi!

i have a multiple tsv files, do you have any methods to merge them in a single file with only header ?

[Pierre Lindenbaum]

head -n 1 file1.txt > merged.txt
for F in file*.txt ; do tail -n +2 ${F} >> merged.txt ; done

or if your header is at the top after a sort

cat file*.txt |sort | uniq > merged.txt

[Nobody]

i want to do it with python

[cpad0112]

use tsv-append and a simple header aware concatenation is (copy/pasted):

$ tsv-append -H file1.tsv file2.tsv file3.tsv

All the available options can be read from here: https://github.com/eBay/tsv-utils/blob/master/docs/tool_reference/tsv-append.md

[Nobody]

it works with python ?

[JC]

hard to know what exactly you have and what you want, you can use cat to concatenate files

[lieven.sterck]

if you need to merged them (column-wise , we don't have enough info to figure that out), have a look at the linux paste command

Answer 1

[tothepoint]

I think this will work for you. Python code to merge tsv files in to one with only header.

#merge_tsv_files
from glob import glob

filename = 'merge.tsv'

with open(filename, 'a') as singleFile:
    first_tsv = True
    for tsv in glob('*.tsv'):
        if tsv == filename:
            pass
        else:
            header = True
            for line in open(tsv, 'r'):
                if first_tsv and header:
                    singleFile.write(line)
                    first_tsv = False
                    header = False
                elif header:
                    header = False
                else:
                    singleFile.write(line)
    singleFile.close()

[Nobody]

thank you so much this is what i wanted to do exactly