Please help with removing spaces from fasta file

Question

[seta]

Hi all,

I'm dealing with a fasta file with spaces at the end of line, which caused the problem. I didn't find a suitable way to remove them. Please kindly tell me the appropriate command for removing them?

Answer 1

[Antonio R. Franco]

Is that you have an space or a lack of the end of line code?

If your data are tab separated, and you have an space only at the end of the lane, you can do the following

cat file.fasta | tr -d " " > newfile.fasta

But notice that this will get rid of all spaces, including those at the middle of the lane.

Answer 2

[dschika]

sed 's/ *$//g' in.fasta > out.fasta

will remove only spaces at the end of lines. To remove tab or space use:

sed 's/\s*$//g' in.fasta > out.fasta

[Jean-Karim Heriche]

Note for sed on Mac OS X, you have to use [[:space:]] instead of \s:

sed "s/[[:space:]]*$//g" in.fasta > out.fasta

Answer 3

[Jean-Karim Heriche]

Not a bioinformatics questions, you should try Stack Overflow for this, but here is a quick answer in perl:

perl -i.bak -pe 's/\h+$//' sequences.fa

[seta]

Thanks. I tried the command, but the whole sequences within the file was removed, so that grep -c ">" file1.fa returned 0

[harold.smith.tarheel]

Try the following:

perl -i.bak -pe "s/\s+$/\n/;" sequences.fa

Note that this will remove all trailing whitespace characters from each line (including newline), and replace with a single newline.

[Jean-Karim Heriche]

What's your perl version ? The \h character class was introduced in perl 5.10.

[seta]

It's v5.18.2.

[Jean-Karim Heriche]

Not sure why it didn't work for you. I tested it with 5.12 and 5.20 and it worked fine.

Answer 4

[biocyberman]

Oh my gawk!

All previous solutions would risk modifying your fasta header as well. This one will not.

gawk 'BEGIN{line=0}{ if ($0 !~/^>/ && $0 ~/ +/ ) {gsub(/ +/, //); line++} print}END{print line" lines with white spaces treated" > "/dev/stderr"}' myfasta.fa >output.fa

If you only want to remove the spaces at the end of the lines:

gawk 'BEGIN{line=0}{ if ($0 !~/^>/ && $0 ~/ +$/ ) {gsub(/ +$/, //); line++} print}END{print line" lines with white spaces treated" > "/dev/stderr"}' myfasta.fa>output.fa

[dschika]

It's true that the solution with sed could also alter the fasta header.

But: have you ever been in a situation where removing whitespaces at the end (!) of the header would mess up something? I hope not ;)

[biocyberman]

To be fair, that is at low probability :-)

[Jean-Karim Heriche]

Yes, they could alter the header but only by removing white space from the end of it (the $ sigil anchors the match at the end of the line). The problem reported was with white spaces at the end of lines, whether the problem was limited to non-header lines wasn't specified.

[biocyberman]

I was just being paranoid and want to present gawk-based solution :-)

Answer 5

[Atu]

Hi,

I think you could make use of the python rstrip() string method. Just call it while reading your fasta file, and it will handle the the white spaces as you want.

for line in open('path_to_fasta_file'):
    print line.rstrip()

Copy the code into a file, say my_script.py, and run

python my_script.py

There you go

[Tej Sowpati]

Wouldn't this also strip the newline characters?

[Atu]

Yes,

any trailing character will be removed (white spaces plus newline character), but newline characters will be added again by the print fuction. So the output FASTA should be well-formed.

Happy New Year!