Dear biostars I really need your kindly help, please save me from this hardship situation

Question

[zizigolu]

Hello,

For more than a month I am trying many tools for finding DE miRNAs in fungi but error, error, error

online tools don't support fungi and local tools were mostly written in perl needed mirdeep2 algorithms

believe my failure is not because of my laziness, I tried whatever I could

although being too rude with a great expectation, please someone help me to install mirdeep via TeamViewer or whatever

I asked institute admin, he tried his best but my problem was not solved

thank you in advance

[lakhujanivijay]

It is often helpful if you share the error message with the community.

[zizigolu]

actually each software with its own error but mostly saying miRDeep ERROR :: system args failed: 6400

[WouterDeCoster]

I understood that an earlier installation was succesfull, but that (after trying to reinstall) nothing works anymore. You also installed your own perl interpreter if I remember correctly. Somewhere, when reinstalling, something probably got wrongly configured and causing your problems.

I would try to remove everything I installed (including perl and perl modules) and start over from the beginning. Because it used to work, right?

[zizigolu]

you all right, I ran on replicate for Aspergillus fumigatus successfully but in the second replicate (before re-installing) only mapper.pl worked and quantifier.pl did not (too weird). then I re-installed but this time even mapper.pl does not work anymore

[5heikki]

In perl scripts you generally define which perl to use in the shebang line. Did you do that?

[zizigolu]

thank you,

you mean in mirdeep2.pl script I should change #!/usr/bin/perl to another directory?

[GenoMax]

If you installed your own perl (if the default one on the cluster is not one you can use) then you would change that line to the path of where this new perl program is (e.g. /home/izadi/bin/perl).

[zizigolu]

thank you

I only downloaded perl-5.24.0 and unzipped in

/usr/data/nfs6/izadi/perl-5.24.0/

in /usr/home/izadi/ there 2 perl folders; perl which is empty and perl 5/bin contains only cpanm

[5heikki]

Perhaps you should download e.g. latest perl source, extract, actually read the README file, google everything that you don't understand, and only then proceed with your stuff.

[zizigolu]

I performed like so

wget http://www.cpan.org/src/5.0/perl-5.24.0.tar.gz

tar -xzf perl-5.24.0.tar.gz

 cd perl-5.24.0

 Configure -des -Dprefix=$HOME/usr/data/nfs6/izadi/

   make
    make test

All tests successful. Elapsed: 1242 sec u=4.69 s=1.59 cu=230.89 cs=25.09 scripts=2248 tests=851140

       make install

now in /usr/data/nfs6/izadi/ a folder was created name perl that is empty but I noticed mirdeep2.pl in perl-5.24.0 folder, what should I do now?

[GenoMax]

Please don't try to install things that are not necessary. They are making your life difficult. Your system perl which is at 5.22 should be fine for miRDeep2.

Start with a fresh download of miRDeep2 code. Open the README in that folder. Start at section 2 in installation, since we know you can't get perl install.pl method to work. Follow each step sequentially (2.1, 2.2 etc) until you complete them all. It is natural that you are starting to feel desperate by now but trying to do this methodically is the only way.

[zizigolu]

ok thank you

but from previous post where you advised me to follow 2.1 Sample Installation, I did and each step with its own error stopped me from ranfold to all

anyway thank you

[GenoMax]

If you had an error at each step then you need to address those errors sequentially. Moving to step 2.2. is not going to make the problem with 2.1 go away magically. You may be able to skip some steps (e.g. if you have bowtie installed already then use that PATH and go to the next app).

[zizigolu]

thank you

I will try the steps again and ask help for each error in a new thread.

[GenoMax]

Followed the steps in miRDeep2 README section 2 to verify that everything is working as included (on linux mint). So you should be able to do it. You would need to install Font-TTF-1.05 package from CPAN for the PDF-API2 to work.

[zizigolu]

thank you for paying attention

fortunately admin installed Font-TTF package from fedora repository just yesterday.

[zizigolu]

sorry for the below instruction from example installation

**for attach the miRDeep2 executable path to your PATH

        (echo 'export PATH=$PATH:your_path_to_mirdeep2/src' >> ~/.bashrc)**

did I perform right?

[izadi@lbox200 mirdeep2_0_0_8]$ setenv PATH

$PATH\:/usr/data/nfs6/izadi/mirdeep2_0_0_8/src/

[izadi@lbox200 mirdeep2_0_0_8]$ setenv PATH $PATH\:/usr/data/nfs6/izadi/mirdeep2_0_0_8/

[izadi@lbox200 mirdeep2_0_0_8]$

[GenoMax]

That echo command won't be useful since you are not using bash shell.

You could do

$ bash

to enter a new bash shell. The system prompt will return but you should now be in a new bash shell. Confirm by echo $SHELL

That will make it easy to follow the instructions from README as is.

What does echo $PATH show now?

[zizigolu]

[izadi@lbox200 mirdeep2_0_0_8]$ echo $PATH /usr/local/sparky/bin:/usr/local/moltemplate/src:/usr/local2/app/MathWorks/MATLAB:/usr/local2/app/Wolfram/Mathematica:/usr/local2/app/intel/Compiler/11.1/059/bin/intel64:/usr/local2/app/abaqus:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:.:/usr/local2/scripts:/usr/data/nfs6/izadi/mirdeep2_0_0_8/src/:/usr/data/nfs6/izadi/mirdeep2_0_0_8/:/usr/data/nfs6/izadi/bowtie-1.1.2/ [izadi@lbox200 mirdeep2_0_0_8]$

[GenoMax]

That look ok. So on to step 2.

[zizigolu]

sorry,

in the next step this happened

[izadi@lbox200 ViennaRNA-2.2.10]$ configure --prefix=/usr/data/nfs6/izadi/mirdeep2_0_0_8/

checking for Perl dynamic library extension... .so

checking for perl... /usr/bin/perl

checking for perl module ExtUtils::Embed... no

configure: error: Require ExtUtils::Embed to proceed

then I performed so

wget http://files1.directadmin.com/services/all/perl_modules/ExtUtils-Embed-1.14.tar.gz

tar xvzf ExtUtils-Embed-1.14.tar.gz

cd ExtUtils-Embed-1.14

perl Makefile.PL

make make install

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! ERROR: Can't create '/usr/share/man/man3'

Do not have write permissions on '/usr/share/man/man3'

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! at -e line 1.

Makefile:694: recipe for target 'pure_perl_install' failed

make: * [pure_perl_install] Error 13

[izadi@lbox200 ExtUtils-Embed-1.14]$

[GenoMax]

Same problem. You don't have root privileges so you can't write to /usr.

See if these instructions help in installing the modules "user space". They add another complication : http://blogs.perl.org/users/marc_sebastian_jakobs/2009/11/how-to-install-perl-modules-from-cpan-in-the-unix-user-space.html

BTW: I did not need to install ExtUtils but looks like you don't have them. Why are you using your mirdeep directory as prefix for ViennaRNA package. Use the directory for ViennaRNA.

[zizigolu]

thank you for your sincere effort to help me to solve my problem

in step 2 I got confused and could not figure out what to do

tomorrow I will start my trying from step 2

thanks again

[GenoMax]

According to the README miRDeep2 can use any perl in 5.x series. So the system perl in your case should be fine. What does perl -version show?

[zizigolu]

thank you

[izadi@lbox200 bin]$ perl -version

This is perl 5, version 22, subversion 2 (v5.22.2) built for x86_64-linux-thread-multi

[WouterDeCoster]

So you tried reinstalling, but without first removing everything? (It's not sure that that will work, but it might)

I tried to install mirdeep2 a few days ago to see whether I also had problems, but it went perfect and test data worked without issues.

[zizigolu]

I removed mirdep2 folder and unziped again but this time when running mapper.pl or wherever saying

[izadi@lbox200 bin]$ mapper.pl 1.fq -e -h -m -v -p af -s sample.fa -t Sample.arf

Please run the install.pl script first before using the miRDeep2 package

The install script is located in /usr/data/nfs6/izadi/mirdeep2_0_0_8/ so just do

cd /usr/data/nfs6/izadi/mirdeep2_0_0_8/

perl install.pl

and when I ran perl install.pl telling

Installing PDF-API2 now

randfold was/is not installed properly

bowtie was/is not installed properly

pdf was/is not installed properly

RNAfold was/is not installed properly

Please run the install.pl script again to check if

everything is properly installed.

like an infinite circle :( :(

[Farbod]

Dear F, Hi

If you intend to use miEDeep and miRDeep2, you must have reference genome.

Do you have such genome for your "fungi" ?

[zizigolu]

yes I have from ensemble

[Farbod]

So I did not get the point that what part of your new "post" has changed.

Is it a new "post" or you have just remove something from old one ?

[zizigolu]

sorry, Farbod I did not get what you mean and which post you mean

[Farbod]

Fereshteh Jan,

you have asked this question before, now I can not find the new part of it.

is it an unsolved old question or it is a new (which part is new) one ?

[zizigolu]

yes, Farbod I removed my email. although I can't run mirdeep2 yet :(

[Farbod]

exactly, I was searching your email! ;-) of course it seems that contacting with email is forbidden and insecure here.

[zizigolu]

actually I think here is secure but I don't like my mentor knows I am solving my problems barely by biostars :(

might be she suddenly come across this post

[Farbod]

I understand.

So, your mentor wants you to install mirdeep2 correctly or to find DEGs of your fungi miRNAs?

[zizigolu]

no she only asked me via a good pipeline with detailed description give her DEGs table. but I only performed so by online tool which she did not accept :(

Answer 1

[Persistent LABS]

Hi Angel, I dont know if there is any specific reason that you are using miRDeep2. But miRDeep* claims that it is better (in predicting novel miRNAs) than miRDeep2 (http://www.australianprostatecentre.org/research/software/mirdeep-star) and has GUI. You can try this. Additionally, there is a web-server oasis2.0 (https://oasis.dzne.de/index.php) that can be used for miRNA identification, DE study and Target prediction. Though your organism of interest is not listed there, there is an option "All Genomes of miRBase v21" under Reference Genome drop-down. I am not sure, if you can use this option.

[zizigolu]

thank you

actually I tried about all tools I found by googling, some didn't support fungi, some gave me something like bugs. I noticed many working based on mirdeep that is why I am trying that

[zizigolu]

I checked Oasis, uploading a fastq file took 3 hours but the problem was the results that my reads were trimmed more than 98% and only about 1 percent mapped. I tried with Illumina Small RNA 3' Adapter and unknown-adapter options separately but the trimming rate was the same.

[Persistent LABS]

Did you assess the quality of reads before and after trimming (for example using FastQC)? How was that? Can you share here? Meanwhile, since you have tried a lot many tools, did you try miRDeep*. It has UI. It can support your genome too. It has that option.

[zizigolu]

thank you, actually since yesterday I am involved in trying Oasis but I will try mirdeep. I tried Oasis with Solid 3 prime, illumina 3 prime, unknown-adapter and even I used my cleaned reads, but the counts of reads and statistics were the same. I emailed maintainer they replied that my reads are too short while they are not short

[WouterDeCoster]

I don't particularly like the result you posted of adapter content (although I'm not familiar with small RNA-seq). What it says to me is that after 20 nucleotides sequencing 70% of your reads already ran into the adapter. Meaning that after removing adapters, reads are rather short! Or perhaps this is normal for small RNA-seq. Not sure.

[zizigolu]

you know, I have already mapped them by bowtie2 and mapping rate was Ok. actually me too can't figure out specially after one month confusion and googling.

[Persistent LABS]

Looks okay since miRNAs have length ~21 nt on average. What happens after trimming? Can you please share: basic summary, adapter content, sequence quality score and sequence length distribution after trimming?

[zizigolu]

sorry, mirdeep asks a mi-RNA gff file from mirbase which is not such a file for fungi

https://i.imgsafe.org/7d73f04621.png

https://i.imgsafe.org/7d7109e0d6.png

https://i.imgsafe.org/7d746ab0bb.png

https://i.imgsafe.org/7d742867e8.png

[Persistent LABS]

After trimming, reads look okay. Now this is regarding the requirement of known miRNA annotation in gff3 (from miRBase) for miRDeep*. If I remember correctly, I was able to run without this annotation file. If that does not work, you try with an empty file with name knownMiR.gff3 and keep it wherever the tool manual suggests. The known annotation file is required to identify known miRNAs in the organism. If this file is kept empty, all identified miRNAs will be novel for this organism. However, later on you can match the identified miRNAs with all the miRBase entries, and would be able to assign an miRNA id.

[zizigolu]

thank you, I tried to build index for aspergillus based on readme file but after building for 2 chromosomes process stuck. anyway I continued with these indexes. first I did not use any gff that I gave error then I used an empty gff that process finished without error with an empty result as I provided shot.

https://i.imgsafe.org/89622e0baa.png

https://i.imgsafe.org/896308c2a5.png

https://i.imgsafe.org/8963acf4cb.png

[Persistent LABS]

I would suggest you to write to miRDeep* author Jiyuan An [e-mail]), and give every possible information like your experiment, the steps that you followed in miRDeep* and observation. He is nice person. He replied to me several times to resolve my queries. I am sure he will help you.

[zizigolu]

I am so thankful for your kindly help and sharing your experiences, I emailed him. as I read some times in this forum local tools are more robust because give us more flexibility then I hope to learn how to install tools like mirdeep2.

[Persistent LABS]

Welcome Angel. We are here to help each other. Once you are done with your analysis, please share whatever Jiyuan suggests you. It will help the community to understand the tool and analysis process better.

[Persistent LABS]

Hello Angel, I remember the case when I was trying to index Ensembl mouse genome (chromosomes were distributed in individual files like 1.fa, 2.fa etc) using build_bwt_index in miRDeep*. I was having some issues. Jiyuan had suggested me to rename the entries as well as individual chromosome files. For example, if I have 1.fa which contains >1 blah blah as header, he suggested to rename the header as chr1, and file name as chr1.fa. If you have some spare time, try this out.

[zizigolu]

thank you, I performed so, this time indexes were built for three chromosomes. then I put chromosomes separately after each error till all indexes were built. I am not sure but might be this is a memory problem or because I am in windows and linux might be work that I did not try yet. anyway with all indexes mirdeep did not work. in biostars I read without gff we should try mireap which I tried and got error about perl packages. in mirdeep manual I read something about changing fastq header, did you change the header of your fastq files before uploading them in mirdeep?

https://i.imgsafe.org/9df8608894.png

[Persistent LABS]

No, I did not change the header in fastq file. May I request you to share the public URL of your genome fasta file? I would like to try to create index files myself.

[zizigolu]

thank you, sure

ftp://ftp.ensemblgenomes.org/pub/release-32/fungi/fasta/aspergillus_fumigatus/dna/

Answer 2

[Nestor Wendt]

There are some docker containers for mirdeep2.

[zizigolu]

thank you,

https://i.imgsafe.org/a43cc57fb3.png

:(

[GenoMax]

If the cluster you have access to in Germany runs docker then you could use the containers there.

[zizigolu]

thank you, good point. I will try

[Farbod]

you can use "filtershekan" for checking it out.

[John]

Wow, and I thought paywalls were a pain in the ass :(

[Farbod]

I think that using this "ass" -contained idiom is not very . . . !

[John]

I don't mean to offend :) I could remove "ass" and swap it for another body part, but "pain in the elbow" sounds weird. Regardless, then your post would still contain the word, and this thread would literally become half-ass'd.

[zizigolu]

sorry John, I did not get you at all. you mean my post is irrelevant with biostars?

[WouterDeCoster]

That's most likely not what he meant. He reacted on the fact that docker is blocked in Iran, and that that must be very annoying -> a pain in the ass means something causing irritation or annoyance and as such is not directed to you.

[zizigolu]

thank you, I thought he is complaining about my long thread :) :) :)

[John]

Ah, no no - I love long threads :D I'm just feeling sorry for the people who cannot access websites like Docker. That's really unfair :(

[zizigolu]

thank you John for your sympathy, a kind biostar

[Ram]

behind is a little less unsavory

[WouterDeCoster]

An elbow in the behind?

Answer 3

[Persistent LABS]

I am using the command line version of miRDeep* on Ubuntu 16.06. Please take appropriate action for your UI version.

 ## Build index
    ## ===========
    $ cd /home/srikantverma/miRDeepStar/build_bwt_idx_v32

    $ cd genome

    $ mkdir Aspergillus_fumigatus
    $ wget ftp://ftp.ensemblgenomes.org/pub/release-32/fungi/fasta/aspergillus_fumigatus/dna/*

    #Only Aspergillus_fumigatus.CADRE.dna.chromosome.*.fa.gz files were kept in this directory
    $ ls
    Aspergillus_fumigatus.CADRE.dna.chromosome.I.fa.gz    Aspergillus_fumigatus.CADRE.dna.chromosome.IV.fa.gz  Aspergillus_fumigatus.CADRE.dna.chromosome.VI.fa.gz
    Aspergillus_fumigatus.CADRE.dna.chromosome.II.fa.gz   Aspergillus_fumigatus.CADRE.dna.chromosome.MT.fa.gz  Aspergillus_fumigatus.CADRE.dna.chromosome.VII.fa.gz
    Aspergillus_fumigatus.CADRE.dna.chromosome.III.fa.gz  Aspergillus_fumigatus.CADRE.dna.chromosome.V.fa.gz   Aspergillus_fumigatus.CADRE.dna.chromosome.VIII.fa.gz

    ## Decompress the files
    $ gzip -d *
    $ ls
    Aspergillus_fumigatus.CADRE.dna.chromosome.I.fa    Aspergillus_fumigatus.CADRE.dna.chromosome.IV.fa  Aspergillus_fumigatus.CADRE.dna.chromosome.VI.fa
    Aspergillus_fumigatus.CADRE.dna.chromosome.II.fa   Aspergillus_fumigatus.CADRE.dna.chromosome.MT.fa  Aspergillus_fumigatus.CADRE.dna.chromosome.VII.fa
    Aspergillus_fumigatus.CADRE.dna.chromosome.III.fa  Aspergillus_fumigatus.CADRE.dna.chromosome.V.fa   Aspergillus_fumigatus.CADRE.dna.chromosome.VIII.fa

    ## change the header and file name. For example,  Aspergillus_fumigatus.CADRE.dna.chromosome.I.fa should have >chr1 as header, and file renamed to chr1.fa
    $ ls  
    chr1.fa  chr2.fa  chr3.fa  chr4.fa  chr5.fa  chr6.fa  chr7.fa  chr8.fa  chrMT.fa

    $ cd /home/srikantverma/miRDeepStar/build_bwt_idx_v32
    $ java -jar -Xmx4g build_bwt_idx.jar Aspergillus_fumigatus
    chr1
    chr2
    chr3
    chr4
    chr5
    chr6
    chr7
    chr8
    chrMT

    $ ls genome/Aspergillus_fumigatus/
    FM.chr1.idx  FM.chr5.idx  FM.chrMT.idx    SA.chr3.idx  SA.chr7.idx   bwt.chr2.idx  bwt.chr6.idx   chr1.fa         chr3.fa         chr5.fa         chr7.fa         chrMT.fa
    FM.chr2.idx  FM.chr6.idx  FM_profile.idx  SA.chr4.idx  SA.chr8.idx   bwt.chr3.idx  bwt.chr7.idx   chr1.media_tmp  chr3.media_tmp  chr5.media_tmp  chr7.media_tmp  chrMT.media_tmp
    FM.chr3.idx  FM.chr7.idx  SA.chr1.idx     SA.chr5.idx  SA.chrMT.idx  bwt.chr4.idx  bwt.chr8.idx   chr2.fa         chr4.fa         chr6.fa         chr8.fa
    FM.chr4.idx  FM.chr8.idx  SA.chr2.idx     SA.chr6.idx  bwt.chr1.idx  bwt.chr5.idx  bwt.chrMT.idx  chr2.media_tmp  chr4.media_tmp  chr6.media_tmp  chr8.media_tmp

## Copy Aspergillus_fumigatus index directory to miRDeepStar's genome directory
## ============
$ cp -r /home/srikantverma/miRDeepStar/build_bwt_idx_v32/genome/Aspergillus_fumigatus /home/srikantverma/miRDeepStar/MDS_command_line_v35/MDS_command_line/genome/
$ cd /home/srikantverma/miRDeepStar/MDS_command_line_v35/MDS_command_line/genome/Aspergillus_fumigatus
$ ls
FM.chr1.idx  FM.chr5.idx  FM.chrMT.idx    SA.chr3.idx  SA.chr7.idx   bwt.chr2.idx  bwt.chr6.idx   chr1.fa         chr3.fa         chr5.fa         chr7.fa         chrMT.fa
FM.chr2.idx  FM.chr6.idx  FM_profile.idx  SA.chr4.idx  SA.chr8.idx   bwt.chr3.idx  bwt.chr7.idx   chr1.media_tmp  chr3.media_tmp  chr5.media_tmp  chr7.media_tmp  chrMT.media_tmp
FM.chr3.idx  FM.chr7.idx  SA.chr1.idx     SA.chr5.idx  SA.chrMT.idx  bwt.chr4.idx  bwt.chr8.idx   chr2.fa         chr4.fa         chr6.fa         chr8.fa
FM.chr4.idx  FM.chr8.idx  SA.chr2.idx     SA.chr6.idx  bwt.chr1.idx  bwt.chr5.idx  bwt.chrMT.idx  chr2.media_tmp  chr4.media_tmp  chr6.media_tmp  chr8.media_tmp

$ mkdir miRBase
$ cd miRBase
$ touch knownMiR.gff3
$ touch mature.fa
$ touch hairpin.fa

Now, you can proceed with identification of miRNAs.

## Run miRDeep*
## ============
$ cd /home/srikantverma/miRDeepStar/MDS_command_line_v35/MDS_command_line
$ java -jar -Xmx2g MD.jar -g Aspergillus_fumigatus your_dir/input_files.fastq

[zizigolu]

thank you,

I tried point by point in linux but the same with windows :(

[izadi@lbox200 build_bwt_idx_v2]$ java -jar -Xmx4g build_bwt_idx.jar Aspergillus_fumigatus

Chr.MT

Chr1

Chr2

Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: -1

at bowtie.create_BWT_Index.write_index(create_BWT_Index.java:212)

at bowtie.create_BWT_Index.createIdx(create_BWT_Index.java:75)

at build_bwt_idx.Build_bwt_idx.main(Build_bwt_idx.java:25)

[izadi@lbox200 build_bwt_idx_v2]$

https://i.imgsafe.org/a136f5cb4b.png

by your suggestion I have already taken DE miRNAs using Oasis