Connect with us

Author: Kelly Ritch

[Interview] Moving the Chess Pieces: A Merck Exec’s Take on the Art of Staffing Clinical Trial Projects

We recently talked with Adam Kinsey, Executive Director, Clinical Sciences and Study Management—Oncology at Merck, about staffing clinical trial operations as part of our new interview series Conversations in Clinical Trial Readiness, by ArcheMedX

We wanted to share this excerpt from the conversation with Adam:


When you are working on the beginnings of a clinical trial project, how do you know how to staff up?  How do you plan resources?


In a rapidly changing environment like oncology, you could have a program shut down or a program ramp up in a very quick time frame. That’s definitely something that someone at my level is constantly looking at. Do we have enough people in the right roles of study manager versus clinical scientist? Do we have enough people to support what it looks like we’re going to have in 6-12 months? Sometimes, companies go out as far as 18 months.

That’s the big picture, but individual assignments to studies is more of an art form. Say that I’ve got a new study coming up that’s very complex and challenging, and a new therapy area to us. Am I going to assign that to a newer, greener person? Probably not. You factor in those things, balancing the work that people already are assigned to and what’s coming up in the future.

One of the things that’s always hard to manage is CRA resources to do the feasibility and site qualification piece. It’s always tough to assign people to that task, because you don’t even know which of the sites that they’re assessing you’re actually going to go forward with. You don’t make the permanent assignment of CRAs until later.

Ideally, the people who are doing the site assessing, and giving the recommendations on whether to select the site or not, are the CRAs who are going to take that site forward, train the staff, get them up and running, and work with them. That’s sometimes harder to accomplish; That’s two very different tasks. You may go to 50 sites in the U.S. to do qualification and feasibility, but you’re only going to be selecting 25. Well, I can’t assign CRAs to service all those 50 sites long-term, because I don’t know where the 25 are going to actually come from.

Get a monthly digest of hand-picked industry content. Sign up for The ArcheMedX Clinical Research Newsletter.


You referenced evaluating fit in a complicated oncology study. The number of people involved in a clinical trial is huge. And there’s the turnover to consider. So, are you looking at fit, or preparedness to be working on the trial, during the trial? If so, how you’re monitoring that?


There’s a set of metrics around monitoring performance. One example is protocol deviations. You might look across the sites that you have on a study and see that a patient is not supposed to be enrolled; that their liver enzymes are elevated. You see that there’re several patients who have violated the criteria. Those violations are supposed to be recorded by various people who observed them, and logged in the clinical trial management system.

Out of that clinical trial management system, you can get trending data on where protocol deviations are occurring — and begin to see outliers. You can overlay that with data like, “Are we seeing that there’s a concentration of higher levels of deviations with the list of sites that a particular CRA is site managing?” That’s from a numeric quantitative standpoint. You might also look at monitoring visit report turnaround time and action item closures. There are a variety of different metrics that you might use, all of which connect to the clinical trial management system.

There are also quality control visits, in which clinical research managers go out to sites to assess them. They take out with them a series of questions that are very quantitative, in some sense, but qualitative in other senses to check on the health of that site. You can then compare that to the signals that you’re getting from the monitoring visits.


Let’s say there’s an investigator meeting, and you’re getting everybody involved with the trial trained up on the protocol. Is there anything that might happen during this stage that indicates that a CRA, or any individual, doesn’t have a grasp of the content matter of the protocol? Does that also get cross-checked with some of the metrics you look at?


There is always a set of questions that you have to complete in order to get credit for that class. It’s generally not that sophisticated; it’s basically just testing that you paid attention. We have a learning management system through which people are given a curriculum for a study, and they have to take courses X, Y, and Z. That system tracks whether you did that, and did it on time.

We haven’t gotten sophisticated with really testing someone’s true grasp of the key messages, or establishing a correlation between people who really paid attention and a reduction in protocol deviations. I haven’t seen anything like that, but it would be really nice.



Do you think an understanding of which parts of the training the individuals have grasped, and how well, could help indicate risk?


I definitely think so. You’ve heard of concepts like quality by design and risk-based monitoring, which apply age-old risk management concepts to clinical trials. That’s focusing on reducing errors that are really going to matter to the overall purpose of your study, rather than just focusing on anything that might go wrong.

What are the risks that are really going to be critical to reduce in your program? I think that with any study, you should be making sure that the people who are working on it really know what they’re doing.


Absolutely. One way to address that — and help support the art of resource assignments — is to employ a digital training environment, one that can analyze unique behavioral data to identify leading indicators of staff readiness and provide actionable information to managers so they can more effectively deploy their resources.

What are the risks that are really going to be critical to reduce in your program? I think that with any study, you should be making sure that the people who are working on it really know what they’re doing.

Learn how ArcheMedX enables peak clinical trial readiness by delivering personalized learning experiences that generate actionable insights.

Continue Reading

We Need Less Guessing In Our Clinical Trial Site Selection Process

When sponsors and contract research organizations (CROs) make decisions about which sites to include in a clinical trial, they too often rely on outdated historical data and show little confidence in the additional data the sites provide to them.

For example, when completing the traditional site feasibility assessment sites may guess or inflate the number of patients that they believe they can enroll. Sponsors and CROs anticipate this response and typically adjust the projections to fit their subjective expectations for each clinical trial site. They simply don’t have faith in the data that the sites report.

This has been the state of clinical trial site selection. There’s little confidence and lots of guessing.

Don’t Bet on Past Trial Site Performance.

Fortunately, we live in a digital age and more and more sponsors are employing new methods to collect and analyze a wider variety of data: data that should inspire confidence. However, there’s a warning commonly offered up by the investment industry: “Past performance is not always indicative of future results.” Think about that when you’re considering sites for your next clinical trial.

Currently, the pharmaceutical industry is investing heavily in mining EHR data and patient claims databases to identify viable patient populations at prospective sites. They then combine this analysis with past historical data when evaluating clinical trial sites — data that might pertain to a different location, a different study team, or a different technology. There are other factors subject to change as well. Given this, just how relevant are those historical data for you and your team planning a future study?

Certainly historical data are a valid input, but one that needs to be balanced with a thorough understanding of the current state of affairs at the prospective site. How confident and capable is the current study coordinator? How committed are the principal investigator and sub investigators to recruiting subjects for your study vs. the five other studies they already oversee?

How can we get this type of insight?

Changing the Clinical Trial Site Selection Paradigm

Is there a way to generate additional data that can complement and contextualize the historical analysis or even replace the traditional feasibility survey with a more accurate, confidence based assessment?

The answer is yes. We can leverage proven digital technologies and behavioral analysis to evaluate site feasibility in a more novel way that assesses the confidence, capability, and interest of site personnel.

Sponsors and contract research organizations (CROs) can deliver an interactive experience around the protocol synopsis and key questions to personnel from prospective sites.

As the personnel engage with the content and questions (at the time and place of their own choosing) the sponsors and CROs learn about them. Specifically, they learn what site personnel strengths and weaknesses are. They learn how engaged and interested the clinical trial site is around the particular study. They learn what key concerns or questions sites might have in operationalizing the protocol. And, ultimately, they learn whether or not site personnel are sufficiently prepared and motivated to conduct a successful study.

This approach can provide valuable behavioral insights that help to inform the selection process.

Time to Move On?

Pharma has yet to fully embrace the power of digital technology (and behavioral data), and the industry’s clinical trial operations in particular retain a lot of out-of-date practices. (For example, most feasibility assessments use basic spreadsheets and many still rely on paper surveys) But considering the vast amounts of time and money involved in getting a clinical trial site up and running — as well as the time and money lost when sites must be closed because of failure to perform— it seems worthwhile to consider changing the status quo.

Learn how ArcheMedX enables Sponsors to re-imagine site feasibility to assesses the confidence, capability, and interest of site personnel.

How Might We Improve the Value of the Principal Investigator Meeting?

The principal investigator (PI) meeting is a well-established (and expensive) tradition in the pharmaceutical industry. And, in our era of global programs, you might need to host multiple PI meetings: one for the Americas, one in Europe, one in Asia. Yet, no matter how many meetings you host, you will most likely face a recurring and costly set of challenges.

First, can you get the right players in the room?

  • Despite your best efforts, up to 60% of Investigators may fail to attend the meeting…and that’s before we consider the impact of frequent staff changes over the duration of the study.

Second, how many attendees can sit through eight hours of PowerPoint presentations without losing interest or even falling asleep?

  • As it happens, numerous brain studies show that the average attention span of an adult learner is only 10 minutes. Imagine how often attendees will become distracted if they are not interested in the topics being shared or not actively engaged in some way?

Third, have you heard of the forgetting curve?

  • Even if you run a flawless meeting, most attendees will forget 90% of what they learned before they ever screen the first subject.

This means the real costs and challenges of getting everyone up to speed are much greater than the initial budget allocated for the investigator meeting.

Can this model be modernized? Made more efficient? Yes, of course it can. If we are open to new ideas, we can transform the traditional investigator meeting into a far more effective experience that lays the groundwork for a successful study, one with fewer missteps and less waste.

Get a monthly digest of hand-picked industry content. Sign up for The ArcheMedX Clinical Research Newsletter.

Let’s Prepare Content Based on What Attendees Already Know.

In preparing content for PI meetings, we typically start by guessing what attendees know and don’t know. We also guess about what content they will find challenging and what they will find easy. There’s no reason to play this guessing game today and waste valuable time during the meeting. Instead, invest in assessing your meeting attendees’ knowledge before finalizing meeting content — and make time spent at the meeting both more productive for you and more engaging for attendees.

How might this be accomplished? Some sponsors distribute a survey or ask for feedback after sharing a protocol synopsis. However, these informal approaches yield minimal feedback and insight from a limited number of potential attendees. A more novel, digital-based approach can be designed to engage PIs and Study Coordinators around content and critical questions ahead of time, providing valuable insight to meeting planners and content developers.

Let’s Properly Train Those People Who Won’t Attend the Investigator Meeting.

Not everyone who must perform at a high level on a clinical study will attend the investigator meeting. There will be scheduling conflicts, competing priorities, and travel constraints. Because the meetings are expensive, the number of people who can attend may be limited. Consider, too, that over the course of a program there will be staff turnover. New sites may be added. How will the newcomers be trained? How effective will the second-tier training be? While there are reasons to employ a train-the-trainer model, one industry insider described the approach as akin to the children’s game of “whisper down the lane”.  There’s a clear problem with consistency.

Instead, why not capture those PI meeting activities in an enduring program that others can access as necessary on their own time and from their own locations? In addition to guaranteeing accuracy and consistency, such programs can be structured to provide insights that let sponsors and contract research organizations (CROs) know exactly how well material has been understood, where the learners’ strengths and weaknesses lie, and how well prepared the learners are for participation in the study.


Remember That Learning Is A Journey.

A PI meeting occurs at a moment in time. That “moment” could be close to the time when investigators will sit in front of their first subjects, or it could be many months prior. As mentioned earlier, numerous studies on adult learners tell us that 90% of what is learned will be forgotten after one week. How much of the information presented at your PI meeting will be retained months later when it must be applied?

Incomplete or lost learning leads to problems like protocol deviations and early high screen failure rates. Many of these missteps could be prevented if we accept that learning is a journey that should be personalized and reinforced over time.

Imagine if you could identify where participants were most at risk and then nudge them to dive deeper into those critical study objectives until they mastered the topic.  Imagine if you could enable learners to increase and solidify their knowledge in the days and weeks that follow the meeting instead of forgetting most of what they learned. Imagine if you could give sponsors and CROs continuing insight into how well prepared the team is.

It’s Age-old But Brand-new.

Preparation. Education. Analysis. Feedback. They are all simple, timeless techniques. But paired with today’s most advanced software and systems, they can deliver never-before-seen levels of clinical trial readiness for sponsors and CROs. You might even decide that you really don’t need those PI meetings after all.

Need more help with your investigator meeting during the pandemic? Check out our new blog on how to perform a virtual investigator meeting in just four steps!