Raise Standards for Preclinical Cancer Research

Jump to original

One of the requirements in science is that you can repeat your experiments and get the same results. In cancer research this has been a big problem. Many of the experiments conducted in cancer research cannot be repeated, or what is known as replicated.

Since medications are tested and then offered to patients it's important to know that they work. If an experiment on a new medicine is conducted one time and shows signs of helping to stop the spread of the disease, it's possible that that means the drug works, but it is not certain. The only way to know if the drug works is to repeat that experiment. Because cancer drugs are often not properly tested in repeated experiments, many of them fail to help patients. In medicine, cancer medications have some of the worst rates of success because of this problem.

Imagine you're the general manager of a baseball team recruiting new players. Would you ever select a player for your team who you've only seen hit one home run? Probably not. You'd want to find players who regularly hit home runs when they're up at bat. The same should be true when selecting a medication. You want to know that it will work over and over, not just in one experiment.

When we make a new drug, our goal should be to make sure the drug targets the cancer cells and stops them from spreading, without causing more harm to the patient. To figure this out, we need to do a lot of tests over a long time. Because of this, our experiments need to be done carefully and repeated, so that we can be sure they work and aren't harmful.

Unfortunately the problems with cancer medication testing goes beyond this lack of replication. Scientists are quick to accept the information from an experiment even if there are a few small mistakes in how the experiment was done. Lately, people have been talking about this problem more, because making new drugs can be very expensive and we keep finding out that the drugs we thought would work, don't.

Amgen, acompany that makes cancer medications, set out to retest the medications often used to treat cancer in hopes of learning which medications actually worked. The two primary investigators from Amgen were C. Glenn Begley and Lee Ellis. They looked at 53 papers that had been published in medical journals and set out to retest them in hopes of finding the results could be validated, or shown to work. What they found was frightening. After retesting they found only six of the 53 experiments produced the same results..

Using our baseball idea, this would be like if you picked 53 players for your roster and when you went into your first game you learned out that only six of your 53 players actually knew how to play the game, the other .

Experimenting on living subjects can be very hard. Animals bring lots of unknowns to the table, so there are many reasons why one set of animals may respond to a treatment and then the next set may not. This is why we need to repeat the experiment several times to make sure it works they way we think it should, rather than just getting lucky the first time.

The scientists from Amgen, Begley and Ellis, set out to try to understand why they were getting different results when they copies the original 53 experiments. In these cases, they tried to talk to the original scientists, to find out what went wrong.

In six studies that produced the same results as the papers had indicated, the scientists noticed that detail and control was very important. In the studies that didn't, they found that sometimes, the original scientists only showed one experiment that worked and didn't mention that parts that didn't work. Those scientists tended to allow sloppy results because the results agreed with what they wanted to be true. The scientists want the medication to work, so they have what is known as a bias, or a kind of pressure to show that it works, even when it does not. Unfortunately, the problems found by Amgen, the medication company,are similar to those found by others. A lot of the time, other scientists can't repeat the experiments and get the same results. Last year, a team at Bayer HealthCare in Germany reported that only about 25 percent of the studies they looked at were good enough to be used. Most of these studies were on cancer. Some of these studies were also looked at by a group at Amgen.

What makes this problem worse, is that when a study is accepted in the medical community it is assumed to be correct. Scientists assume the experiment worked as it should and they then move along in the scientific process of testing other ideas using the foundation that came from the earlier studies.

One way to think about this is to think about a house. If we find a house we want to buy, but it's a bit too small, we might plan to buy it and then build on another floor. We are assuming the house is strong and that the foundation of the house will be able to hold this new upper level. But what if the builder put the wrong material in the foundation or he didn't test the strenth of the materials he used? When we go to build on top of the house the whole thing might come tumbling down because the foundation of the house was not built properly. This is exactly what happens in science. The first researchers in a field or looking at a specific medication lay the foundation. If their work is not solid then everything that comes afterwards will also not be correct.

It's important to say that these problems are found in all scientific research. These studies were mostly looking at cancer research. But the fact that most of the published studies couldn't be replicatedby scientists or by medical trials shows that there's a big problem in that industry.

To fix this, we need to make the early stage of cancer research better. There are problems in the work coming out of colleges and universities as well as the medical industry that need to be fixed. It will take a lot of effort and the desire to make things better. Most importantly, scientists need to do a much better job being careful about their work raising the standards for these studies.

One big problem in developing cancer drugs is the misuse of early stage data from cell lines and animal models. These problems have been talked about a lot and are well known. For example, often the cell lines used don't properly show what happens in humans. Also, there are problems in understanding how drugs move and work in the body, and in choosing what outcomes to measure. Also, early testing rarely uses markers that could help choose which patients will likely benefit from a drug later on in clinical trials.

Cancer researchers need to be more careful in the early stages of study. Because of the difficulty in simulating the human body in early stage research, we also need those reviewing the research should demand more precision.

Cancer researchers will have to make the tough, slow, and expensive transition towards new tools for research. They will need to use stronger models for predicting tumour behaviours and improved verification strategies. Also, early efforts to find markers that help identify suitable patients should be required at the beginning of drug development. All in all, the scientific process demands high standards of quality, ethics, and thoroughness. In the end, the people who plan, do the experiments, and then share the results are the ones who are responsible. They have to make sure their experiments are designed and done correctly. They also have to make sure that what they tell others about their work is all correct. When we do science, we have to be super careful, honest, and hard working.

So why do some scientists end up sharing wrong, incomplete, or missing information? Well, the world of science can sometimes make them think this is okay. Researchers, especially, feel a lot of pressure to do really well so they can get money for their work, get good jobs, and get promoted. They worry that their work isn't going to be perfect. Because of this, they might only share the best pieces of their work, or even change some of their data to fit what they originally thought.

Science is the pursuit of knowledge. If we want to learn more about something we must be careful about how we're learning. We need to be thoughtful about problems with the information we have and look for better, more serious ways to test our ideas. Instead of expecting every piece of research to be flawless, we should be happy with real and repeatable results. This way, scientists won't feel as though they have to present a perfect story every time.

Even though the people who review or edit for science journals share some of the blame, the scientists must be responsible for the data they create, check, and share. We should always remember that the goal of science is to make life better. The better we do at research, the more we can help people.

Earlier in the history of medical science, people weren't very careful with how they tested new medical treatments. But things have changed a lot since then. So if we can change once, we can change again.

Making sure that all cancer research is reliable and can help in real treatments will be really tough. But we must never forget that patients should be our focus. Our work is funded by the public and by charities. Therefore, we should keep trying new things, keep making progress, and keep bringing new treatments that will make life better. Even though a lot of research papers are written each year, we've not had as many success stories as we had hoped for. That's why it's important to follow a process that is open, and that helps patients on a regular basis.

--------- Original ---------

Jump to original

Add intro:

Prior to testing new cancer treatments on humans, scientists must first demonstrate in the lab that the treatment will actually work. This is typically done using animals, such as mice, or with cell cultures made from human tissue know as cancer cell lines. Once the treatment is shown to be effective and safe in the lab, it can then be tested on humans. This process is referred to as “preclinical research”.

Unfortunately, most big discoveries in cancer research can't be duplicated. As a result, the success rate of the drugs developed is remarkably low. This is partly because the cells of animals we use in the labs aren't similar enough to human cells. It's also due to the poor quality of the studies, and selectively choosing the data, making the treatment appear more effective than it really is.

When trying to create new drugs, we count on the “preclinical” research findings. Preclinical just means before they've been tested in a wide range of people, before they're available for a doctor to prescribe to patients. Preclinical usually refers to the testing phase before the drugs are approved for public usage. We can't test how well a drug works on humans until we've been studying it for many years We need to be sure that our early lab tests are very thorough and accurate. These tests need to account for how different every case of cancer is, and how different every patient is.

We'd like to think that we can trust the information in preclinical research papers,. but sadly, we can't. Even though this problem has been known for a long time and has been talked about a lot, it hasn't been resolved, and is still a big issue. The cost of developing drugs is very high, and a lot of clinical trials fail, so we need better therapies.

In 2012, researchers at the biotech company Amgen tried to check whether the results in 53 landmark research papers could be repeated. They knew that some results might be difficult to replicate because some of the papers were chosen for testing new ideas or strategies against cancer. Out of the 53 papers, only six experiments, of 11 percent of the total tested, were found to produce the same results as the original scientific researchers had found. This was shockingly lower than they expected..

One explanation for the failure to replicate could be that the original researchers did not take share all of the ll factors in their experiments or did not include all the data they collected. It's not essential to include all information in a research paper, but it is increasingly common for researchers to leave out important information, this is especially true in cancer research. If a scientist attempts to replicate an earlier experiment they need to know exactly how the original experiment was conducted and all the significant data points used at that time, without that knowledge the research has to guess and this may lead to mistakes.

Separately, a team from Bayer HealthCare in Germany conducted a similar exercise retesting foundational studies that were widely accepted as validated. They found that only about 25% of published early-stage studies were good enough to proceed with next steps. Cancer research made up 70% of the studies they looked at. Some of the studies Bayer tried to replicate may have also been studies that Amgen tried to replicate. Amgen has kept all the studies they retested a secret, so no one knows which ones were found to be irreplicable and which ones were valid.

We can be sure that some of the irreproducible studies went on to form the basis for an entire field of study, with hundreds of extra papers adding to the first paper's ideas. But the original studies were never checked to make sure their ideas had merit. Even worse, some of these studies led to trials on human patients, meaning that many people were subjected to treatments that probably wouldn't work. Since only the original scientists knew that Amgen was working to reproduce the results they'd found, it was up to the scientists to tell the public that their work was no longer credible. To date, none of the scientists whose work was found to be irreproducible have come forward making it highly probable that their work is still being relied on by medical professionals who do not know the work is not trustworthy. This is terrifying.

However, this doesn't mean that all cancer research is bad. There are some examples of research that have quickly and accurately led to benefits for patients. In 2011, several new cancer drugs were approved, based on solid early-stage data. Still, the fact that industry and clinical trials can't confirm most of the results from published papers on possible new treatments suggests a big, general problem. This problem was recognized by many researchers Amgen researchers talked to, both in academic and industrial settings.

As for how to improve early-stage cancer research, there are clear issues both in academic and industry settings with how the research is done and shared. To fix these, we'll need a lot of commitment and a willingness to change how research is conducted. We'll need to demand higher standards for completing and sharing these studies.

There's also a big problem in developing cancer drugs when it comes to how we use and interpret early-stage data from cell lines and animal models. These models aren't perfect and have multiple limitations that have been widely recognized. We need to be more careful in how we carry out these studies, and try to better mimic the human body. Reviewers and editors should demand more careful work.

We should conduct early-stage studies as if they were clinical studies, including keeping researchers in the dark about the different research groups. All experiments should be checked using the right controls. They should also be repeated, ideally by different researchers in the same lab. We should present all the data in the final paper, even including when a drug didn't work.

We shouldn't publish studies based on a single cell line or model. We should use a variety of cancer cell lines that are similar to the patients we hope to treat. Cancer researchers need to make the tough, slow, and expensive move to new research tools, as well as more reliable tumor models and better checking strategies. We should also start trying to figure out who's likely to benefit from a drug right when we start developing it.

Remember, science demands the highest standards. At the end of the day, the people doing the research, the labs where they work, and the schools or organizations they work for should make sure everything is done right. They're the ones who are responsible if an experiment is designed badly, if they don't have enough strong evidence, or if they're leaving out important information. Doing science correctly requires the highest standards and ethics.

We need to build a better system.

Why do people end up publishing info that is wrong, selective, or not repeatable? Well, the way academia and peer-review works might actually be encouraging this unintentionally. Researchers need lots of publications in big journals to get money, a job, promotions, or job security. Editors of these journals, people reviewing the work, and grant committees often prefer findings that are simple, clear, and complete - basically a 'perfect' story. So, researchers feel tempted to only share certain information or data, or even change it to fit their hypothesis.

But there are no 'perfect' stories in biology. Sometimes, the stuff we don't know can lead to more research. For example, if a certain treatment only works on some cells, this can help figure out why it doesn't work on others. People who are reviewing the papers and grants need to be okay with these 'imperfect' stories. They should reward the work that can be repeated with exact same results. This way, scientists won't feel like they need to tell 'perfect' stories to advance their careers.

Even though editors, reviewers and grant committee members are partly responsible for these flaws, researchers should admit when they're wrong and take responsibility for their work. As scientists, we need to remember why we're doing this research: to help make patients' lives better.

The current lax attitude towards creating and analyzing pre-clinical-trial data is similar to what was going on in clinical research about 50 years ago. But change is possible. Improving preclinical cancer research to the point where it can be repeated and then used in successful clinical trials will be really hard. But we need to remember that patients are at the center of all our efforts. If we forget this, it's easy to lose our sense of purpose and urgency.

Cancer researchers get money from taxes and donations, which is being wasted. But most importantly, patients rely on us to come up with new treatments that will make their lives better. Even though hundreds of thousands of research papers are published every year, there have been few clinical successes given the amount of money that's invested. We need a system that will allow for a clear discovery process that consistently leads to significant patient benefits.

Recommendations to improve:

• Researchers should have more chances to present negative data. Negative results can be just as informative as positive ones. So, all findings - regardless of the outcome - should be included in conferences and publications.

• Editors of scientific journals need to take an active part in promoting this culture change. They could do this by making sure negative data can be easily found in search engines.

• There should be ways for researchers to report unethical behavior without fear of harming their careers.

• There should be better communication throughout the whole process, from preclinical researchers to physicians to patient advocates.

• More credit should be given for teaching and mentoring, rather than just for getting papers published in journals.

• New research tools must be provided for all researchers, such as better access to large cell-line collections, better characterization of cell-lines, and better tumor models.

Homeschool:

Homeschooling Curriculum: Learning Through Preclinical Oncology Research

Objective: This homeschooling curriculum aims to educate students about the importance of research methodologies, reproducibility and validation of scientific findings in preclinical oncology research.

Course Duration: This course will run for eight weeks.

Course Requirements: Basic understanding of biology and an interest in medical research.

Week 1: Introduction to Oncology and Preclinical Research

Objective: Understand the basics of preclinical research in oncology, carcinoma types, treatment and importance of cell lines and animal models.

Week 2: Quality of Published Preclinical Data

Objective: Appreciate the relationship between data quality and oncology trials. Understand target biology in drug development.

Week 3: Clinical Endpoints and Patient Survival

Objective: Learn about clinical endpoints in cancer research and how it differs from other disciplines.

Week 4: Importance of Robust Preclinical Studies

Objective: Understand the importance of generating robust data sets capable of withstanding challenges of clinical trials.

Week 5: Reliability of Preclinical Studies

Objective: Discuss the variability and reproducibility of data in preclinical research.

Week 6: Confirming Research Findings & Landmark Studies

Objective: Learn about efforts made to confirm published findings via case study analysis.

Week 7: Irreproducible Research and Its Implications

Objective: Understand the impact of irreproducible research on drug development, clinical trial failures and need for effective therapies.

Week 8: Resolving Discrepancies and the Importance of Collaboration

Objective: Discuss the role of collaboration in resolving discrepancies in research findings. Discuss the role of investigators in research integrity and miscommunication.

References will be provided for each weeks topic to allow students to study the subjects in depth. At the end of the course, an assessment will be given to gauge the students' understanding and apply what they have learned to real-world scenarios.

Assessment: The students will be assessed through weekly assignments, quizzes and a final project that requires them to examine a real-life preclinical study and discuss its reproducibility and validity.

Disclaimers:

This homeschooled course is composed of complex and advance topics. It is recommended that students have a prior understanding of biology and elementary oncology before they start. Curriculum Title: Understanding Scientific Research & Reproducibility in the Healthcare Industry

Lesson 1: Introduction of Pharmaceutical Research

Objective: Students will understand the significance of preclinical studies in pharmaceutical research and the problems associated with lack of validation.

Lesson 2: The Consequences of Non-Reproducible Research

Objective: Students will study cases where non-reproducible preclinical papers triggered an entire field of false research and clinical trials.

Lesson 3: The Exceptional Cases

Objective: Students will learn about successful instances of research translations into clinical benefits. This lesson also introduces the students to the systemic problems in the pharmaceutical sector.

Lesson 4: Improving Research Quality

Objective: Students will learn about getting the preclinical environment and the need for raising the bar of reproducibility in performing and presenting preclinical studies.

Lesson 5: Addressing the Challenges in Cancer-Drug Development

Objective: Students will gain knowledge through reviewing examples of erroneous usage and misinterpretation of preclinical data in developing cancer drugs.

Lesson 6: The Limitations of Preclinical Cancer Models

Objective: Students will be informed about the limitations of preclinical cancer models, and the widespread acknowledgement of these limitations in research sectors.

Lesson 7: The Need for Rigour in Preclinical Studies

Objective: Students will understand the necessity for more rigorous approaches in preclinical studies to ensure the robustness of research.

Lesson 8: Study Validation Techniques

Objective: The focus of this lesson is on the strategies for proper validation of experiments, including blinding of investigators, systematic use of controls, repeatability of experiments, and complete representation of data.

Lesson 9: Diversification of Models Used in Studies

Objective: Students will understand the importance of using diverse, well-characterized cell lines and models in studies and the need for biomarkers in patient selection during drug development.

Lesson 10: Maintaining Highest Standards in Scientific Process

Objective: Wrapping up the curriculum by emphasizing quality, ethics, and rigorous standards in the scientific process while carrying out research.

Project:

Have your child develop a mini research project based on the learnings from this homeschooling curriculum, encouraging them to apply the concepts and practices emphasised in the lessons. Curriculum Title: Emphasis on Quality in Home-School Science Education

Week 1: Understanding Responsibility and Accountability in the Scientific Process

- Explore the concept of responsibility within a scientific investigation

- Discuss various instances of accountability, focusing on poor experimental design or the lack of proper data

- Emphasize the significance of ethics and rigor in scientific processes

Week 2: Building a Stronger System

- Delve into possible reasons for publishing erroneous, selective, or irreproducible data

- Discuss the pressures faced by researchers in their career development

- Understand the allure of the "perfect" story and the pitfalls it presents in an unbiased analysis

Week 3: Imperfections in Science

- Learn why there are no perfect stories in biology and explore real-life examples

- Discuss how missing gaps in research can present new opportunities

- Understand why it's significant to reward reproducible and partially imperfect results

Week 4: The Importance of Data Accuracy and Creditability

- Reiterate the responsibility of data generation, analysis and submission resting on investigators

- Discuss the ultimate purpose of research to improve lives

Week 5: Understanding Research Rigor in Historical Context

- Trace the evolution of rigor in preclinical data generation and analysis over the past 50 years

- Draw parallels between past challenges in clinical research and present issues

Week 6: Improving Preclinical Research

- Understand the importance of reproducibility in preclinical cancer research and discuss challenges involved

- Learn about the contributions of cancer researchers and their impact on patient lives

- Discuss the importance of transparency and focus in the field

Week 7: Recommendations for Improving the Reliability of Studies

- Understand the importance of presenting negative data

- Discuss the potential influence of journal editors in cultural change within the scientific community

- Understand the role of search engines in reporting research findings. Curriculum: Homeschooling Children with Inbox Weekly

Week 1:

- Introduction to Homeschooling: Understand the basic principles and benefits of homeschooling.

Week 2:

- Setting Up Home Classroom: Learn how to create an effective learning environment at home.

Week 3:

- Establishing a Daily Routine: Plan a structured routine that balances your child's academics, hobbies, and recreational activities.

Week 4:

- Enroll for External Programs: Learn how to sign up for supplementary educational resources like 'Nature Briefing: Cancer'. Discuss the importance of learning about various fields, even those not typically covered in standard curriculums.

Week 5:

- Implementing Technology in Education: Understand the usefulness of online resources and how to use them effectively.

Week 6:

- Use of Standard Educational Resources: Learn how resources from verified sources like 'https://verify.nature.com' can aid in a child's development.

Week 7:

- Introduction to Digital Media: Teach your child about how digital media works, covering aspects like URLs, image sourcing, etc.

Week 8:

- Practicing Cybersecurity: Educate children on the importance of internet safety.

Note: Assignments will be primarily online but will require students to apply their knowledge in a practical setting. Both parents and children will be equipped with the necessary tools and skills to make homeschooling a success. Parents will receive weekly updates and detailed instructions via inbox.

--------- Original ---------