script to organize files

Question

[Galileo]

Hello,

I have thousands of files that I would like to organize into project directories automatically. I have a "file metadata table" as a tabulated text file with two columns: (1) file name, (2) project the file belongs to. How can I easily execute:

• make a directory per project name
• mv files to corresponding directory according to metadata

I believe this is something super easy to script for people that do this routinely. I've tried writing something in bash and perl but without success.

Following this operation my goal is to have another script navigate each directory and run a bioinformatic pipeline.

Thank you very much!

Answer 1

[jv]

A bash example

while read file project
do
  # check if project dir exists, if not create it
  if [[ ! -d $project ]]
  then
    mkdir $project
  fi
  # move file to project dir
  mv $file $project
done < metadata.txt

Revise accordingly for your file and project paths

[Galileo]

Thanks jv, this seems to work but read added '$'\r' at the end of filenames of the second column and thus mv cannot find the file. Any ideas how to remove '$'\r' so the file name is recognized?

[Galileo]

I found a solution: used dos2unix on the files that have '$'\r' at the end :)