An international panel of scientists that I’m a member of is today issuing a ‘call to action’ – aimed at eliminating the use of substandard research tools in biomedical research.
This call is coupled with the launch of a new community-based, 'TripAdvisor-style' website designed to help scientists choose better-quality research tools – and to avoid potentially serious errors in biomedical studies.
In an article published today in the prestigious journal Nature Chemical Biology, the expert group warns that many researchers are unwittingly using poor-quality chemical probes, leading to mistaken and misleading conclusions being drawn from research studies.
Furthermore, these errors are wasting both time and money, and are causing delays in getting new treatments to patients.
As an active researcher in chemical biology and drug discovery myself – as well as being responsible for a cancer research institute – this is a problem that I feel very strongly about. As scientists funded by the public, all of us in the academic community have to hold ourselves accountable for the quality of the research we carry out.
We owe it to the public and to patients not only to do the very best science we can – illuminating our understanding of fundamental biology – but also to avoid delays in speeding effective treatments for cancer and other conditions.
Yet the expert panel that I am part of – involving researchers from non-profit institutions, plus biotechnology and pharmaceutical companies – identifies numerous examples of research studies using poor-quality and outdated research tools.
Urgent action is needed.
So what is going on? What are these chemical probes? Why are many of them substandard? What is so difficult in rectifying the situation? And what is the remedy?
The rise of chemical probes
First the definition.
The ‘probes’ we are talking about here are chemical compounds that are of relatively small size – scientists refer to them as ‘low molecular weight’ – so not big molecules like proteins (such as antibodies) or nucleic acids (such as DNA).
These are substances used by researchers to investigate or ‘probe’ (hence the term ‘chemical probe’) biological processes in cells that control normal functioning and how this goes wrong to cause disease.
Chemical probes – sometimes also called chemical ‘tools’ – play a vital role in biomedical science, including cancer research. The best probes are designed to interact with a particular ‘target’ molecule in the cell of interest – usually a protein – resulting in inhibition (or sometimes activation) of a function of that target protein. By perturbing a protein’s behaviour in this way, high-quality chemical probes can help determine the biological role of that particular protein.
In addition to use in pure or basic research another common application of chemical probes is to predict the effects that a future drug might have on the protein of interest, and the consequences (beneficial or adverse) of that in the body – and so help decide if a possible therapeutic target molecule is ‘valid’ enough to embark on a long and costly drug discovery programme.
By acting as prototype drugs, chemical probes also help to establish whether it is technically feasible to produce a real drug that acts on the target – known as determining the target’s ‘druggability’. Whether a target is druggable relates to the presence on the protein of a cavity or docking station into which the chemical probe or drug can bind.
As an alternative to chemical probes, researchers can use genetic technology – such as RNA interference and CRISPR – to investigate the function of a protein in a cell, organism or disease. Chemical tools provide a complementary approach to such genetic manipulation – and have certain technical advantages, such as better control over the extent and duration of the alteration of protein function. Using genetic and chemical approaches in parallel can provide further understanding of – and often extra confidence in – a biological discovery and the validity of a drug target.
Over the last decade the academic research community has dramatically increased its efforts to produce chemical probes acting on a wide range of target proteins. This is in large part because DNA sequencing technology has massively expanded our knowledge of the genome, and so the development and use of new chemical tools to uncover the functional roles of the gene products – usually proteins – in both healthy and diseased cells has become imperative.
In addition, because of the availability of large numbers of uncharacterised potential drug targets arising from genome sequencing many academic groups – including our own at The Institute of Cancer Research, London – have become more directly involved in discovering both chemical probes and new medicines.
The discovery of chemical probes and drug candidates is also aided by increased access to screening technologies for chemical compounds in academia as well as industry.
Many probes emerging from these efforts have fulfilled expectations – acting as powerful research tools to understand biology and providing seeds to spur the development of new medicines.
But as the use of chemical probes has increased dramatically, it has become clear that many such tools in widespread use have significant limitations – and are often compromised by fatal flaws.
The scourge of poor-quality chemical probes
As numerous examples in our call-to-action article show, many chemical probes being used today are of limited or poor quality and are often employed incorrectly – generating misleading results.
It is quite common to see researchers continuing to use out-of-date probes for investigating a target protein, when much better chemical probes exist.
In many instances, chemical probes may affect proteins other than those claimed – so-called ‘off-target’ effects.
And in extreme cases – unfortunately not at all uncommon – chemicals that are claimed as useful probes may be quite indiscriminate in their actions, affecting a very large number of proteins in the cell and rendering them essentially useless for biomedical research.
Those using these poor-quality probes risk coming to the wrong conclusions in their research and confusing themselves and others.
This isn’t a theoretical risk – it is happening very frequently, and in some cases having serious consequences.
In one case, the flawed use of a probe led to the failure of a major phase III clinical trial in which more than 500 patients were treated. There had been many years of research in the lead-up to the trial, but the researchers had failed to pick up that the drug being developed (called iniparib) did not work in the way researchers assumed it did – to block what is in fact a very good cancer target called PARP – and instead acted indiscriminately. I will return to this problem later.
It was to prevent this kind of calamity that several years ago I established an approach known as the Pharmacological Audit Trail.
Using the audit trail ensures we understand how the drug is affecting a particular target or biochemical pathway and that the evidence for this is very robust – as we showed with our preclinical and clinical studies with our PI3 kinase inhibitor pictilisib (previously known as GDC-0941), now being developed by the major biotechnology company Genentech.
Application of chemical probes is increasingly one of the one of the major methods researchers are using in early preclinical work – so it is essential they are fully aware of the required quality and potential drawbacks of the probes they employ.
So what makes a good chemical probe and what are the common limitations and fatal flaws?
Fitness factors and flaws in chemical probes
Writing with my ICR colleague Ian Collins, we previously provided recommendations and guidelines for high-quality chemical probes. While cautioning against overly restrictive rules that might stifle innovation, we advocated use of a ‘fit for purpose’ approach in the form of a series of ‘fitness factors’ to be considered when assessing chemical tools.
More recently with another ICR colleague Julian Blagg we discussed the desired properties and limitations of chemical probes – with a focus on target validation in cancer.
As a result of important contributions from many other academic and industrial researchers – such as this and this – and recommendations such as those used by the Structural Genomics Consortium – a consensus has emerged that to make useful research tools chemical probes must meet certain minimum quality criteria.
The current consensus guidance for high-quality probes, reflected in today’s Nature Chemical Biology article, places a major emphasis on:
- Potency – achieving the effect on the desired target with low concentrations (small amounts) of the chemical probe, so as to reduce the likelihood of undesired off-target effects
- Selectivity – ensuring that any effects of any other proteins, especially related ones, occur only at concentrations 30-fold or ideally 100-fold higher than those on the desired target
- Broad profiling – checking extensively for lack of interaction with an ‘industry standard’ panel of ‘off-targets’ such as this one
- Careful controls – further minimising the risk of reaching false conclusions from off-target effects by using two different chemical probes with very distinct chemical structures, as well as variants with small changes that lead to loss of the desired activity (inactive analogues).
In addition, chemical probes need to dissolve well and be stable in water, penetrate into cells, show evidence of target engagement and biochemical pathway modulation in cells, and be readily available to researchers in a pure form – for example from trusted specialist commercial vendors.
Additional characteristics are needed if the chemical is to be used for research in animals, which requires features that are closer to those of a drug – such as distribution to tissues and reasonable half-life.
In fact, it’s becoming increasingly clear that in some respects the criteria for high-quality chemical probes often need to be more stringent than for a drug to be used in patients – especially in regard to selectivity. This is because it’s necessary for some drugs to act on several targets (known as ‘polypharmacology’) in order to deliver the desired clinical benefit.
In contrast a high-quality chemical tool should ideally show strong potency and very high selectivity for the claimed target – with minimal off-target effects – so that it can be used with confidence by researchers to probe for the importance of that particular target in cells and organisms.
It is often not appreciated that a considerable amount of work is required to achieve a truly high-quality probe – chemical compounds identified by high-throughput screens are usually just the starting point and if used ‘as is’ can be the source of many of problems discussed here.
Although the flawed probes identified in the current call-to-action article – and in the previous sources provided above – have many limitations, it is often the problem of insufficient selectivity that really lets them down.
In extreme cases, like the example of iniparib above, so-called chemical tools may interact with a very large number of cellular proteins and other targets such as DNA.
There are several common causes of such promiscuity: reacting randomly and irreversibly with cell constituents; clumping together of individual probe molecules to form ‘aggregates’; and undergoing oxidation/reduction chemistry (‘redox reactions’). Such indiscriminately active compounds are referred to appropriately enough as PAINS (Pan Assay Interference Compounds).
Many of us have wasted time, effort and money on such chemical con-artists and they should be banished to the wilderness. In fact, such PAIN compounds can often easily be identified by expert medicinal chemists and chemical biologists, and a web-based tool is now available to help exclude them.
Preaching beyond the converted
Although the guidelines and recommendations developed so far have been useful, they clearly haven’t been sufficient to change widespread behaviour in the use and especially abuse of chemical probes.
I think a major reason for this is that even the most conscientious researcher may not be able to keep abreast of all the emerging literature about probe specificity and the new tools that become available on the market.
It’s easy for a scientist who is inexperienced in using chemical probes to be influenced by historical use, since the continued misuse of many older probes continues to plague the published literature. Search engines and commercial listings from vendors will inevitably bias the uninitiated towards out-of-date tools.
An example of this is the still widespread use of an old chemical tool known as LY294002 – an early inhibitor of the important cancer target PI3 kinase. It was originally described as a PI3 kinase inhibitor in 1994 and has been used in around 30,000 scientific publications.
Although LY294002 was undoubtedly useful as an early ‘pathfinder’ probe, research going back over a decade has shown it has major limitations. In particular it is flawed in the key fitness factors of potency and selectivity – inhibiting many other proteins at the concentrations used to inhibit PI3 kinase. We discuss this in more detail here.
In the meantime, a large number of more selective and well-characterised PI3 kinase inhibitors have become available. An authoritative source has recommended the use of newer chemical probes such as our PI-103 as a PI3 kinase/mTOR inhibitor and the more recent pictilisib/GDC-041. The availability of these and other new potent and selective inhibitors has obviated the need for LY294002 as a chemical probe, and it should be discarded as a selective research tool. Yet entering ‘LY294002 and PI3 kinase’ in the search engine Google Scholar returned as many as 1,100 documents for 2014–2015 alone. And commercial vendors continue to sell LY294002 as a PI3 kinase inhibitor.
So how can researchers ensure they are choosing the appropriate probes and using them correctly? That’s where the expert panel’s new ‘TripAdvisor-style’ resource, called the Chemical Probes Portal, comes in. It is designed to be a one-stop shop for expert and up-to-date information.
The independent Portal site is endorsed by the Structural Genomics Consortium, US Broad Institute and the ICR and initially seed-funded by the UK's Wellcome Trust. It will crowdsource information from experts – allowing researchers to share their experiences and views about chemical probes, and their colleagues to access the most up-to-date comparative information, so they can select the best possible probe for their needs.
We have also produced a checklist for researchers who use chemical probes – which can be found in Box 1 of our Nature Chemical Biology paper. This has key questions about the quality of the probe that any researcher should ask themselves.
In addition, we have recommended use of the checklist and Portal for journal editors and reviewers of academic publications and grants. Where new chemical probes are generated, selection of reviewers experienced in the development and application of such tools is essential.
And we urge commercial vendors to provide accurate and up-to-date information on the properties of probes such as their selectivity and other fitness factors, and to make available negative control compounds.
We can’t ignore the uncomfortable truth that a significant number of scientific papers are coming to incorrect conclusions. But the hope is that our community-led resource will make a real difference to the quality of future research studies.
We need much greater rigour and critical caution in the nomination and application of chemical tools. Researchers should always be sceptical and adopt the maxim of caveat emptor – let the buyer beware.
As our article makes clear in its title, we need to maximise the promise and minimise the peril of chemical probes.
Let’s make sure that all future research using chemical probes is gold standard – because gold-standard research is what we should all expect, and what the public and patients with diseases such as cancer deserve.