First, this is formally okay. The SAM specification says: "Bit 0x4 is the only reliable place to tell whether the segment is unmapped. If 0x4 is set, no
assumptions can be made about RNAME, POS, CIGAR, MAPQ, bits 0x2, 0x10 and 0x100
and the bit 0x20 of the next segment in the template."
In other words: if the "segment unmapped" bit is set, then the "read mapped in proper pair" bit doesn't mean anything anymore.
One scenario where this can happen, is if you align read pairs with BWA and one read maps to a chromosome end and goes over the end (e.g. circular chromosomes like chrM or bacterial genomes). That read is then set to unmapped by BWA, even though all the other flags are kept as they were and even the mapping positions are kept in the SAM output. See also Paired-End Sam Flag Mismatch During Filtering post.
what is your aligner ?
I use BWA 0.5.9 and samtools 0.1.7
Can you provide the read pair so that we can try to reproduce this output? What version of samtools are you using?
they look like:
CWS01-ILLUMINA:22:FC:3:1:1954:990 87 ENSMMUT00000038271 647 29 7S40M29S = 571 -116 ATCCCACTAAAAATCTTTGAGATAGGACCTGTACTTACCCTGTAAATAACCCCGCTTACTCTCTCTCTCTCCCCTN ############################################################################ XT:A:M NM:i:3 SM:i:29AM:i:29 XM:i:3 XO:i:0 XG:i:0 MD:Z:13A12T4T8 CWS01-ILLUMINA:22:FC:3:1:1954:990 163 ENSMMUT00000038271 571 29 76M = 647 116 GTCTATTACGGACAATGCTCAGAAATCTGTGGCGCAAATCACAGCTTCATACCAATTGTCGCAGACCTAATCCCAC IDIGBGG@8@>BBFBBGE@D<EG4GFF<F@ECF?CGGGGDG<+GDGGGIFHIBHEGEGBGE@DGB,<=?=EIGGBB XT:A:U="" NM:i:3="" SM:i:29="" AM:i:29X0:i:1="" X1:i:0="" XM:i:3="" XO:i:0="" XG:i:0="" MD:Z:41T17T5G10<="" p="">