In my view, there could be three different reasons why a normal sequence search, e.g. with BLASTP, against the annotated protein coding genes in a genome would fail to identify an ortholog when there should be one:
- The region of the DNA within which the gene is located is missing in the genome assembly
- The gene finding pipeline failed to annotate the gene despite it being there
- The protein sequence has diverged too much for you to be able to identify the ortholog
To convincingly prove that a gene really does not have an ortholog in a given genome, you need to somehow justify that neither of these three are likely to be the case.
To rule out point 1, you need to evaluate the genome assembly. Has the genome been assembled into complete chromosomes, or does it consist of a large number of contigs. In the former case, it is fairly unlikely that big chunks of DNA would be missing, but in the latter case you will have difficulties ruling out that the gene of interest might simply not have been sequenced.
To rule out point 2, you could do what you suggested yourself: run BLASTX against the genomic DNA sequence. If you do not find a good hit to your query sequence, it is fairly unlikely that the ortholog was not identified due to errors in the gene finding process.
To rule out point 3, you need to show that the gene of interest does mutate faster than other genes for which you are able to find an ortholog. You can do this by analyzing orthologs across a number of species and estimating mutation rates (or simply looking at percent identity). If your sequence does not appear to diverge faster than other sequences for which you managed to find orthologs, sequence divergence is unlikely to be a concern. As a side note, if divergence is an issue, you might want to look at gene synteny as a way to identify the ortholog.
If all of the three reasons for why an ortholog might not be found can be ruled out, you can reasonably argue that there appears to be no ortholog. Otherwise, you can only conclude that you could not identify one.
Generate some generic PCR primers! Bench science has its place at times :->
@Alastair: thanks for the suggestion! It is certainly one way to go in many cases. Unfortunately not in my case as the genes I'm interested in have never (even remotely) been found in any other species. So designing degenerate primers is even worse than looking for a needle in a haystack. Basically one of the referees on my manuscript wants proof they do not exist in my negative control data set...