Hey, thanks for posting. Other opinions than mine are very welcome.
Some issues that I have identified over the years:
1, lack of reproducibility of results
This is compounded by poor study design, which itself relates to any number of things:
- no statistical power
- imbalanced sample groups
- failure to control for sources of bias, including age, gender, sampling time, ethnicity, et cetera
- untested software that contains bugs
I could add here the fact that every instrument that we use has windows of specification in which the instrument is meant to be operated, and they also have error rates. No NGS instrument, for example, can faithfully sequence any sample of DNA - error will always exist.
2, lack of appreciation of biological variability and how to best capture this
This is mainly for expression, ChIP, metabolomic, and proteomic studies
3, slow and costly clinical trials
Clinical trials are very costly and take many years to conduct. Most go nowhere.
4, No 'translational' mechanisms in place
In most cases, there are no official mechanisms / systems in place such that data from a research setting can be readily used in a clinical environment, or, if one exists, validation obviously has to take place, and this may be governed by national and / or international law. If something new is introduced, I notice that it is usually a 'local' change to the health system, i.e., in a hospital unit / department, and not something that is global.
------------------------------
On a side note, I have noticed research change even in the time during which I have been in it (or maybe it is that I have changed). I notice researchers exhibiting greater signs of stress and with less focus on the end goal of the very research that they are conducting. Most do not ponder on what their results could do in terms of improving a health service. Many are also focused intensely on publications and winning the next grant to simply stay in the job. This is a vicious cycle that is ultimately eroding quality research.
I should finally add that I believe more tests will be produced in the next few years. In fact, in certain countries (e.g. Brazil) where there is less regulation, many tests are enrolled straight into practice from research settings. This may prove dangerous, though.
Kevin
Good points. Let's add points 5) apriori probability. If your system detects something with a 99% accuracy (1% FPR) and the general population carries the disease at <= 2%, then your new test is no better than blind chance!
6) no practical workflow. I've seen new techniques touted that detect short term issues, but take days to process or analyze. The test result is obsolete by the time it's ready. Imagine a heart-rate monitor that takes two days to give a result. Pointless clinically, even if it's great fun academically or scientifically.
Any new test needs to be better than what we had before, and affordable, and quick, and error-proof. Machine learning on high-dimensional RNA doesn't satisfy any of that.
Thanks for the detailed explanation. Really helpful. I am on a project that aims to distinguish lung cancer vs cirrhosis using DNA methylation via liquid biopsy. Can you share some nice review articleS regarding DNA methylation based classifer. I am pretty new in this area.
I am not sure. Probably the best thing is to use a search engine. In this case, I would search for keywords:
ncbireviewmethylationclassifier