Discussants:

Moderated by Leeza Osipenko from Consilium scientific. Nick, over to you on your two minutes your idea.

Till Bruckner: Each of the panellists is going to give a quick pitch of only two minutes on one idea of how clinical trial registries can be improved. And after that we'll have a discussion 

Nicholas DeVito: So, before we go on for spending the next hour talking about what's wrong with the registry system I just want to say, first, how really excellent, the registry system is. It is a great accomplishment that it even exists in the form it exists in, and it really allows a level of transparency and overview of the clinical research environment globally, that we really wouldn't have otherwise. I think it's worth recognizing that that's the case, and that's why we're so interested in making it operate better and more efficiently to serve the purpose, it was intended to serve. So we've landed in a situation where, to various stakeholders, registration is seen as a chore, or a checkbox, something that needs to be overcome on the road to doing your trial and getting it published in a good journal. And that's really a shame when you think about the value it should hold throughout the entire process of conducting, and eventually reporting a clinical trial. And because it's sort of taken on that stigma where it’s just become a thing you need to overcome to do at the beginning, we see really poor quality data on the registries, which makes stakeholders further down the line, like journals, take it less seriously as a tool for sense checking and transparency and then they value it less, which then leads the investigators to value it less, and their institutions to value it less, and so on and so forth and it really creates a negative feedback loop that undermines the entire system. So intervening to break that negative feedback loop and ensuring that registries are used in a meaningful way, and maintained in a meaningful way throughout that cycle by all the stakeholders within that cycle is really central. So we've sort of gotten the registration part down. Now we need to do all the other parts of keeping those registries maintained, getting results onto those registries, and ensuring those registries are sustainably funded and can offer a minimum level of functionality across the entire registry system, which is currently definitely not the case across all 17 primary registrants, for sure. That would be sort of my very high level and broad pitch on how to fix the registry system.

Perrine Janiaud: So, like was mentioned before, I am a registry user. I have dived into the registries through covid evidence where we're tracking all covid related trials. In the emergency of the pandemic we’ve seen an overwhelming number of trials being registered and the situation evolved extremely rapidly, and it was really hard to keep track of what's planned, what’s ongoing, what’s completed? And this was because even in this very short time information was quickly being updated. And so I was wondering, as users. There are some registries which makes huge efforts as sending reminders email about their starting date completion date, posting their results,  but this should maybe be done on a broader level, maybe at the WHO level or try to get all registries to do such simple automatic reminder emails. That would be my first point.

 My second point is a consequence of long frustrations, dealing with registries, with trials being registered in multiple registries for various different reasons: because of local requirements, but also we’ve seen with those huge international collaboration with platform trials, adaptive trials, being launched through the pandemic. We had multiple registrations within the same registries, for the same master protocol, and some of them were extremely hard to track down, it was not that clear. So my question would be, why not try to work with all the registries? The WHO has put in place a unique trial number, why not try to have the registry make it a mandatory field within their own registries, so that we can hopefully track those multiple cross registrations, and really answer the question how many trials are being initiated? Because it's really hard to know exactly what's going on in a very evolving situation like a pandemic especially. So that would be my points. 

Gayatri Saberwal: The unsolvable problem. And that is that all of us have done all these studies where we find problems within a record: missing data, wrong data, internally inconsistent data. We find problems between registries for the same record. Registry and publication registry and FDA etc. Now if you look at it, the data, around a given trial is in many other locations as well. So, each PI and there may be multiple PIs on a trial has data somewhere on the bit. The sponsor has the East fees or ethics committees have, plus all these other, the registry the publication maybe the even preprints regulator etc. Now I know this may not at all be feasible. But is it possible to have one place where everything exists, from start to finish. So if I register, everything from the protocol from the beginning ,from the get-go, putting all the data in one place and then registries draw down that data to themselves. Regulators draw the data from there, because we not only have data, everywhere, we also have data being updated or not at different rates in different places. But if there was one place then everything would be updated there. So it would probably be possible to do a comprehensive audit. If there was let's say a multinational trial or a trial running in, you know, various places, even in one country, to do a comprehensive audit of every place where data related to that trial is, it will be close to impossible I suspect because so much of it would be confidential, the ethics committees, the sponsor, the regulator: they may not give us any data. But if there was this one place, which again would have to have controlled access up to a point and then maybe public access. Maybe it's just science fiction, but I thought I'd put it out there.

Marc Taylor:  I’d just like to start with a sort of yes but reply to Gayatri, because I guess our experience in the UK has been that, things started moving better when there was good research management, and you start to see information systems joined together so that there's an end to end, feel about it, and also that the PIs feel as though they are being supported by the research management function in their institution. So, it seems to me, if you like, in each country that there's a way forward. But it's a way forward that involves quite a lot of different actors. Now let me turn to my feeling about this, which is that it's not the registry system that needs fixing so much as the research environment around it. And this is if you like an international version of the same point of view because the big step forward as far as the UK was concerned, was when the Health Research authority started to link the business of registration and reporting to its responsibility for the interests of the public and patients in reliable research outputs. So, that's a question about results being as important as the mechanics of registration. I just said the mechanics of registration were pretty important. But the feeling that this is all about the punch line seems to me to be fundamental to progress with the registry system. And, I mean, the US registry is a splendid thing, and a gift to the world, but it's really bizarre, when you look at the registry movement as an international enterprise that it's still so reliant on the kindness of the US people.  What I would really like to see is some other champions I mean India for example, a very good example, is a growing importance in vaccine production and all kinds of medicines production, but it still has a regulatory environment, which fields about 20 years out of date, and it doesn't require publication of results or data it's all about disclosure to the regulator. So there's a big step there, it seems to me, and I'd love to see Indian scientific leaders and preferably politicians championing the public health interest in transparency, both at home and as real partners for the other people at the World Health Organization, who have an interest in understanding in this. I'm not trying to divert attention from everything that needs doing in the countries that support the regulators who are on the World Health Organization's trusted list, but it just seems deeply unsatisfactory that we don't have that the Government of India aspiring to have its own regulatory system and its research management join in that international community that promotes transparency in the public interest.

Thomas Wicks: It's hard to come a bit later on in the conversation. I've heard some really good points that I would have loved to have made, proving your idea for example of sending out reminders is great. My perspective is a lot on how we help people engage, support, or submit data to the registries and do so understanding what's required. Some very basic ideas I was thinking were what could registry do that cost virtually nothing. One is if everyone could really think through the FAQs and the support materials that they provide. Simple things like some registries are good about providing a list of “here are all the data fields you will have to fill in when you register a study, when you disclose results.” Each one of those fields with a brief description. Simple things like is it a regulatory requirement. If so, yes, then link to that the latest version of the regulation. Often, it's not linked, or the links are broken, out of date, it's just sort of basic things like that, some essential statistics, the number of trials that are posted. The earliest trial that was posted, things like when did the registry go live, just so we can get a general sense of how far back it goes. And then finally, and this is where it costs perhaps a little bit more, more of an investment is making it easier to get data in, and for researchers to get data out. Many of the websites, I know some colleagues who do research, have to scrape data using sort of essentially hacking the website to scrape the data instead of having a download function, clinical trials.gov does it extremely while others do it very poorly. Some even require codes that you enter before you can see this so it's really sort of public, but you can't use it. That seems insufficient. I'll leave it at that for now maybe as the conversation continues I have quite a list.

Leeza Osipenko: Excellent Thomas so Deborah, a difficult position being last but please go ahead with your comments.

Deborah Zarin: It's good that I'm last because I feel like the grumpy old lady here. I'll just say that between 2005 when I joined clinical trials.gov right before the journal editor’s policy took effect, and about 2015, my team and I spent an inordinate amount of time going around the world going to WHO meetings, going to London meeting with Mark, meeting with ISRCTN folks. People went to Australia, people went to India, all in an effort to prevent the chaos that we see now. The chaos is the direct result of WHO policy, which was to encourage the formation of new registries in different geographic regions around the world. We were trying to argue that this was, needless duplication of infrastructure and would lead to world chaos. At one DIA meeting when I was speaking it got turned Deborah's chaos they started talking about Deborah’s chaos. So the things: unacknowledged duplicate registrations and multiple registries, all that. It's because there are so many registries that have variable amounts of funding and variable amounts of staffing, and there's no solution. The idea that a unique identifier is going to solve this is actually silly if you think about it, no offense but the duplicate registrations happen because person A has no idea that person B already registered the trial somewhere else, or that there are multiple PIs, they're not registering it on purpose. If they are registering it in multiple registries and it's the same person doing it, They include the unique, the registry IDs. It's a very simple issue to find an NCT number in ISRCTN, or to find an ISRCTN number in clinical trials.gov. That's not the problem. The problem is when the different investigators or sponsors, don't realize that you know person A registered in the Indian registry, and I'm coming here to register it sometimes under a different title with different data elements in another registry. So, if you didn't know that's happening, you're certainly not going to use the unique identifier. Each registry has a unique identifier so it's easy to link them if they exist. So, that's my grumpy preface which is that we saw this coming, we knew it was going to happen. And it's annoying.

Back to some other points, though, what can you do now? It’s not too late. I mean the registries could in fact consolidate. We have old PowerPoints that show how you could divide the world into geographic regions have primary registries maybe three or four sort of like gen bank. So there's gen bank which has, I think, a registry in the US and the UK and in Japan and every night they exchange data, they collaborate closely they meet frequently, and you can find gene sequences uniquely. There's not a problem of unacknowledged duplicates really. Whereas the world registry system is not like that. So I think the first thing is you need fewer registries or fewer so called maybe meta registries or come up with some name that would agree to come up with a way to exchange data and make sure that if a trial is being done in India, then, if that's where the coordinating centre is or where the sponsor is, then maybe that should be the primary registry, and anyone else who sees that registration should ask for the Indian registry number.  It's easy enough, if clinical trials  come in from Japan, we should be able to ask for that ID number so that's not that hard. Back to some other issues that till asked us to address. I agree with Mark Taylor that registries are real, there's a difference between the registry and the policies that encourage or require registration, or the policies that encourage or require results reporting. And it's the sort of holders of those policies that have the leverage, right so clinical trials.gov can only do so much about a sloppy registration. There are rather strict quality control standards like clinical trial.gov, but if a sponsor does something that's not perfect, but barely meets the quality control standards, clinical trials. gov can nudge, but really the sponsor can say we don't care. But what needs to matter is that the journal editors who have their policy care, the FDA, that has enforcement power and NIH that has enforcement power needs to care, the employers, the academic medical centres, as Mark was saying: somebody needs to care If you do a sloppy job registering your trial. And if nobody in your incentive structure cares. you're going to go the least problematic course and do it sloppy and with registries there’s limit to how much they can do. Now, people like Till, and Ben Goldacre and Nick DeVito can call this out, they can have their report card so they can call it out, which matters somewhat, but they'll all tell you that certain groups still don't care because in their daily incentive life, nobody cares that they're on the bottom of Nick DeVito’s report card. And so, you still need research funders sponsors academic medical centres to actually care about the quality. I'll stop there, but I can, I can rant on cue anytime. 

Leeza Osipenko: This is a call to the audience, please feel free to put your comments into the chat or put your questions for our experts, so we'll try to address them. But now I would like to hear from you to reflect to each other's points. Maybe you've already started doing this and your responses. But please go ahead. If you would like to support or propose any other solutions.

Thomas Wicks: Just something that Deborah said earlier around the unique identifiers, this  idea that maybe we need to have a global identifier and then everything would be solved, even on a more modest scale, if we could at least agree that there's a certain uniqueness that has to be observed with protocol IDs. It's not so much in pharma companies but I see academic research studies posted, which have a running number 001 002 003, and there's a ton of them or they use a date because that’s the data the studies start. It's not really that good because depending on where you're looking, there'll be 60 studies that have the same ID there's no uniqueness to it at all. So if there could be just some thought paid into a form of doing it, even if it's within the organization and they use the first three letters of the protocol title and then 001, anything like that would allow more assurance that were kind of comparing apples to apples and find those links studies

Deborah Zarin: Thomas I would just interject that I would propose that instead of adding a new unique identifier to the world, if it's in the US, make the NCT number on the front page of your protocol, if it's in the UK make it the ISRCTN number on the front page of your protocol and actually use that internally as well as externally.

Thomas Wicks: The only thing is some researchers don't. Some studies, depending on where they get posted, it would be great but even I have found people have inconsistent NCT numbers across studies registered in Europe where they list the NCT number on CT Gov. They don't match up, In some cases.

Gayatri Saberwal: Yeah, product is added. We once tried to do a mock registration with clinical trials.gov, and they said you have your registry in India so why don’t you register there. Now, that's fine and I think others have also tried and been told the same thing. I think right now the number of trial records in clinical trials.gov is about roughly 60% of all the 18 17 plus one. For your idea of there being two or three registries globally. I presume it takes quite a bit of money to you know handle process every record. I know budgets can be increased but there's always reluctance to do that. I'm fairly pessimistic about the Indian registry, some of you would have seen that it's been in the news recently we've been. They have been threatening to close it down so I wouldn't expect a huge investment, it may happen, but I doubt it. So if all these other records were to suddenly, someone should request that we put our trials into clinical trials.gov. How do you see them as being receptive to that,  given the financial burden and also, I don't know if the, sort of motivation to register high quality trials, is there, I mean I see that's bad record.

Deborah Zarin: So, first of all, I clearly don't speak for the National Library of Medicine. I guess I never did but I used to pretend to and now I definitely don't. With that being said, I think the time period when you tried to register, there are plenty of Indian based trials on clinical trials.gov right. But, in an effort to try to organize the world. We did go through a time where we were trying to encourage people to register first, say in India, or first in China, and then carry that idea of you also wanted a clinical trials.gov record. That was an effort to help things not to deter people from registering clinical trials.gov. sounds like it didn't work that way. So, I'm sorry about that. From my point of view when I was at clinical trials. gov, we were getting maybe 500 registrations per week. Doubling that to say 1000 per week would of course involve more staff because of the quality control issues, but it wouldn't involve a doubling of the budget. The infrastructure is there, the IT infrastructure doesn't change. And it seems more efficient globally, to have local organizations, able to track down the ethics issues do with education, do support for registrations say, I mean there were countries, like Israel, for example, unbeknownst to us and clinical trials.gov at the time, literally passed a law requiring registration on clinical trials.gov and they created a portal that is an Israeli based portal. Other countries have since done that. So, I don't know about the current people at clinical trials.gov but I think in general the feeling was the best resource for the world including for people in the United States is the most comprehensive resource you can get just like PubMed takes you know journals from anywhere.

So I think that that would be a conversation that they would be interested in having. Unfortunately, most of the organizations that have their own registries feel very strongly that they want their own registers for a variety of reasons. Also, people don't want the United States to take over and I don't blame them but that would be why we have places like Australia, get a chunk, and have places get resources, get big chunks, and have other registries maybe feed into them.

Leeza Osipenko: We have a comment from Nicholas, and then from Mark, and there's also a very interesting comment in the chat from Judith. So if one of you wants to read it and address that.

Nicholas DeVito: This whole argument of centralization versus a decentralized registry system is difficult. It's one I've gone back and forth on quite a bit because I think in an ideal world, the system that Deb is describing probably would be the best. But it's tough to see what that world looks with the role clinical trials.gov plays. And it's difficult to see that happening when clinical trials.gov couldn't even become a primary registry in the system we have now even though I know that might be a distinction without a ton of meaning.

Deborah Zarin: It's not that it couldn't. It didn't. It’s because it didn't believe that the WHO approach, actually made sense.

Nicholas DeVito: So when you look at some of the countries that did decide to create their own registry infrastructure it's places like China and Iran, that might have distinct reasons of not wanting to rely on US infrastructure which you mentioned Deb you know there's a whole there's a lot of reasons for this. I think we also have to recognize that there's obviously the collection of all the registries, that form the primary registries on the steering committee of that level which you know Mark is involved in. But the day to day operations of the ICT is from my understanding, pretty much one guy, like in Geneva. There was this big ground swell from the 80s through to the 2000s and all this advocacy and all this work in this collective action that went into creating the system that we have now. And then once we got there and its sort of like we hit the ICMJE statement. And that sort of formed the peak, maybe that and the FDA amendments act formed a mini peak, and I feel like it's been a lot of people went and said Oh great, we solved the whole clinical trials transparency thing. What's the next thing, when they were far from it. So I think only global concerted passionate effort from the people who are involved like the people who got us here could lead to that again, could fix this situation in the end. And that involves a massive redirection of resources and thinking of, how we do things in one way or another.

Leeza Osipenko: Mark your comments and you also have a question from till.

Marc Taylor: First of all I'd like joint billing with Deborah as being responsible for the chaos, because I think we just seriously have to recognize the political obstacles. There was a point when we wondered whether it would make sense, more or less to merge or put the ISRCTN registry under the same management as the US registry but in the end, mostly for what reasons of political nervousness we didn't. And when you come to the European Union where one would have thought that we might be able to act together I don't think that the damage to the registry system was the decisive point in the UK’s decision to leave the European Union and withdraw from the European mechanisms for the regulation of medicines and devices. Somehow, we have to find a way of prioritizing and for the collaboration around the improvements to hold the registry system together and make it work as a worldwide enterprise. Recognizing that probably, it would be a wonderful thing if we say we had a pan-African registry that other people could collaborate with, because there’s a serious difficulty if we have some registries which were relatively well funded and others which are trying to operate on almost nothing. Whatever you say to those registries about sharing technical measures and solutions with them, they hardly likely to be able to accept. So, we need a way of working, which takes some of the thinking away. 

Leeza Osipenko: These are very interesting suggestions, and it looks like there is no easy solution to this, but we are talking very much about the future. But we currently have a mess of decades of registered trials which are very hard to make sense of for research purposes for example. Even In 20 years from now, trials which are right now duplicates and they're in a system poorly filled with poor quality data. They will remain the problem, even if we find an ideal solution in five years. What are your thoughts in terms of forward thinking solutions to perhaps dealing with what we already have: hundreds of thousands of records, we'll hit the million pretty soon. At least on clinical trials. gov and then we're probably over a million across all the registers in any case, so any comments, any thoughts on the retrospective fixing?

Deborah Zarin: I have thoughts on real time fixing. So, this is what at least clinical trials. gov had been trying to do and I don't know the status now, is that if a trial comes in, and again if you see an Indian connection, or Canadian connection, or a UK connection to make sure to ask. So we asked people to list other trial identifiers, but there's no way to enforce that because of course you don't know for sure that they have another trial identifier, but you can guess that given that you know the policies in India, that if it's a trial that's centred in India, that they must have registered in the Indian registry. So you could then insist, if each trial registry would try harder to get what are likely identifiers from other registries for any trial. So if you got a trial from the US, you should ask for the NCT number, because once you have those read those numbers it's an informatics problem. That's very solvable right, you can link those up so that that's one thing to do, and it does take staff to do it because just having an automated message saying “please give us any other registry numbers” leads to the situation we have now, so you have to go the next step of saying “No I know you must have an Indian registry number, can you please go find it” 

Leeza Osipenko: Any other comments on potential retrospective solutions or current time reflections to Deborah's point.

Thomas Wicks: When I start working with new companies, what I'll often do is look at some of the registrations they've done in the past and just run simple reports. Just as an example it's one I just did recently. You can find a slew of studies that were registered 10-15 years ago, and you can see how long the sponsor thought the study would run so it will tell you one year and three months. And then what I do is I take that, I double it, because nothing ever runs to plan. And then I say, it says that you thought you would end this study in 2009 and it's still showing as active and just maybe it's time to clean that up unless something really happened but at least you get a candidate list of likely nonsense. I think that's a fairly simple analysis that any registry should be able to do is look at when was the study registered and is it likely that it's still ongoing.

Nicholas DeVito: Yeah, I've done the exact same thing. I was just wrapping up a study where we did basically almost that exact thing using the EUCTR data. Thinking retrospectively in fixing, some of that stuff is lost forever right the pi is dead, there's no records or whatever there are people or someone who doesn't even work in clinical research anymore who has no interest, and you can't force them to care. All these all these reasons, the login was last forever. But what you can do is start caring a lot moving forward right and that's what you can do that, that's where you get to things like journal editors, not simply looking and this helps with some but not all trials but journal editors: they require that you register, that checks a box, why not the natural next step which is to say, does what's in this trial report you're sending me right now match what's on the registry and if it doesn't it needs to before we publish this? It seems like a natural next step, but we've never taken that next step and it doesn't seem like anyone's very interested at the moment taking that next step. Or a university recognizing that they are under a legal obligation to report. If it wasn't for Till, half the universities in Europe wouldn't even know that they had to report trials to the EUCTR still and it's only by you know him, adapting the work we've done, and a Herculean effort to raise that awareness that we've started to see real progress on that. So why did it have to be that way, and what were the failures, and what can we learn. So it's a lot about learning from what happened retrospectively, and probably acknowledging that a lot of what's wrong retrospectively is probably irredeemably messed up. We can do some, but we can't do all and making sure we get it right moving forward, really should be the priority.

Leeza Osipenko: Do you think the biggest problem is that we left this registration of clinical trials to an individual, to a person, the PI, the representative of the company. So actually, hundreds of thousands or 10s of thousands of people are doing this and this one of the reasons that we have a mess. Would that be better if there was a designated person or group or team, entering the data, saying we cannot register your trial because you haven't supplied this, this, and this, and this team would be following up everyone. Of course you can say this is hugely expensive, but we tried to go down the cheap route, but it didn't work like eBay, it didn't work like Amazon. It created, unfortunately this very difficult situation. Any comments on that and can we reverse this at this point?

Nicholas DeVito: Others are moving, and I mean, I've researched this, Till has written up like this. Some people are moving to that method, but it's very ad hoc and it's very, “did we get a naughty letter from the government”, that we didn't do this right that's causing us to make these changes, and that's only in the places where people are campaigning on this effort to listen to like in Europe and in somewhat the US. You get to some of these other places, and I'd love to hear perspectives from India on what this looks like. There was just a preprint a few days ago, about how all these doctor readies, I believe, published all these peer reviewed studies for covid in the Indian registry, and they can't find any information on what happened to these studies and recovering thousands of patients on an active treatment that is being given to hundreds of people, thousands of people across the globe for covid right now preliminarily. So yeah, I really be interested to hear Gayatri’s perspective on that 

Gayatri Saberwal: So a couple of points. One is you know what Anthony Keith has done at John Hopkins. If you could have an older person like that in other institutions or even for groups of institutions if the trial density is not that high, that would be great. In India, I suspect that such a person may not be treated very well by the medical community. We tend to be extremely hierarchical especially amongst doctors so I don't know how well that would work. Now I just wanted to mention the idea that journals should sort of catch this and the reviewers should catch this. So I think there was a study, I'm sure you know this person I'm forgetting who it is, where false, Maybe NCT IDs or something were put into the papers. Those weren't really genuine. So it's almost like I have to put in a number, I put in a number, I may not even put in the correct number, you know. But then, I was talking to a medical doctor quite overwhelmed with his volume of work, and he was so fed up because the journals were asking him to go back and check the registry. He said you know “they pay me an honorarium, let them not pay me an honorarium, let them check everything there and just not ask me to do it.” And he wasn't trying to belittle, the fact that this data has to be accurate. He was just overwhelmed with work. So, I think, in different ways. So the Anthony Keith kind of solution seems to me, at least reading what he's done, (that you've got a dedicated team, they're building up sort of SOPs, they're sharing that with others) at least from this distance, it would work for some. I don't know if it would work in every situation.

Deborah Zarin: You know in registration when we were first developing the data elements and working with them, the assumption was that this would be an administrative task, because the information would be clearly laid out in a trial protocol. And so if the trial protocol, really had the information clearly, then say lower level, even non-scientific person could probably extract it and put it in the registry. And that's how it was kind of marketed to begin with. What we all found out during this whole process is that trial protocols are quite deficient. And the registration record is supposed to be a summary of key facts from the protocol. But frequently you can't find those key facts and the protocol, or they're contradictory in various parts, you know, you try to find a primary outcome measure in a protocol, you could find evidence for six different primary outcome measures, depending on where you look. And so, that again is a research quality research milieu issue where I talked about this as a sausage factory. It turns out that trials are being run based on really poor protocols, or poorly described protocols for sure. And probably, honestly, you have to go back to that source, you can hire all the people you want, but I've had guys who are NIH funded who couldn't tell me how many arms were in their study, because they just don't even conceptualize it that way. So all of the user help you're going to give them isn't going to get to a reliable registry record, because it turns out that their study was really poorly conceptualized and/or poorly documented. So unfortunately, I don't think again, this can be solved easily at the registry level, it starts before that when the protocol is being written and approved.

Thomas Wicks: Yeah, when it came to it, Nick and Till will agree, there's been a lot of work done on the industry side. That's not to say especially small industry sponsors couldn't do a better job because there's a direct correlation between company size and compliance with disclosure requirements. But on the investigator initiated side, what I think often causes the mess is that it'll be “Doctor DeVito” who was running a trawl, but doesn't necessarily indicate within what institution, because ultimately, especially in the US the institution is on the hook, but there may be no mention of it, so there's a lot of professors and doctors running trials and no sense of what institution they are affiliated with, what hospital they're affiliated with so when they leave, there's no one there to clean up these trials. So back to what Nick was saying earlier and maybe also at the beginning, about reminding people. That only works as long as the person is still in that institution and then things go away, there needs to be some kind of follow up mechanism and I think  one of the simple things would be to require: that you indicate what institution do you work for, or what company, to get away from those mistakes.

Leeza Osipenko: I have a high level question for you. We mentioned this briefly but this comes from what you've been saying in the beginning, who should be in charge, can there be one regulator to whom people respect whom people be afraid off, whom can fix and clean things up because once again this distributed system, on one hand, there was some logic behind it, but it led to a mess so local regulation does not work, and decentralized regulations seems to fail. So, anything we can do at that level. What roles do WHO play

Nicholas DeVito: So, for WHO to play a role as I said before, they would need to rethink their participation in this entire system and if they want to take a leadership role, they need to fund it like they care about taking that leadership role and not simply host the ICTRP which is one guy, who runs a registry system that's built, as far as I can tell, like technological gum and glue, which is basically is the ICTRP meta-registry. Like, you know you could invest in real IT infrastructure, you could invest in like Till was saying in the chat earlier, where would it be technically feasible to have a centralized back end, yeah right we could potentially have like a minimum registration data standard that could work in every registry, so therefore if you did have to register at the EU registry, in the US registry for regulatory reasons, you could just fill out one thing and send it to both registries. But once again that requires the registries to be in accordance with what they want to collect, how they want to collect it. And then subsequently they all basically have the same information at a base level, they have the ICTRP minimum data set standard, or they should at least. Even that's not fully implemented as well as it should be everywhere, but like, there's no reason it couldn't be. There's no reason we couldn't have a technical data entry standard that works cross-registries, but the thing is it's even that requires a whole thing because let's say we did that, let's say we got that together, you know who at the Indian clinical trials registry is going to adapt the Indian clinical trial registry to actually be able to accept the standard registration format. Maybe the thing is if you don't do that you shouldn't be a registry anymore, but I don't know if that's an acceptable political or logistical answer. With every single answer, you run into these hurdles that's created by the system we have, and just really rethinking it fundamentally, all these cool things that we could do with modern digital standards and things like that, would need to probably start from some measure of scratch based on what we have now I think right. It would be very difficult to adapt what we have now into this digital utopian sort of view and even that's not even what I want to say because we don't need a utopia to do better than we have now, but just thinking about it in sort of idea idealistic and ideal terms.

Thomas Wicks: Just one suggestion, rather than a controlling global body, what seems to have worked the best recently is peer pressure. So there's the really valuable work that Till and Nick and everyone is doing around the EU trials track reminding academic institutions of their obligations, forming a scorecard in the States, those are all commendable but it's also industry associations and like Pharma EFPIA I have done a lot a ABPI in the UK has done a lot. The academic working group in the States, I think the members of that, by in large are doing much better than they used to. And maybe that's kind of how a lot of this is done,  that it becomes normative behaviour, maybe for in academia, you need to show evidence that you've done things properly when you're applying for new positions or if you're trying to get funding from wherever it is you have to show that your past research was properly disclosed. I think that's probably going to be more effective because it gets away from politics and it gets away from that.

Marc Taylor: I just wanted to bring something up from the UK experience which is partly encouraging and partly not. There is a combined review process which works between our health research authority and the medicines regulator nowadays. So there's a harmonization of what they require to know in order to sign off studies, clinical trials, of that kind. I mean Deborah and I can remember that the WHO data set was cooked up by a small bunch of people who agreed violently over a long period before they managed to turn the dataset into words. And it's been relatively stable, but so far as I know there isn't, for example, a working group of the WHO list of favoured regulatory bodies, which has decided that the registration data set is the same as what they need to know for their regulatory purposes. I mean, that's one example. Also I don't think that any consensus about needing precisely that information is present among the funders of this kind of research in any country. And I mean I don't know if we've got any French colleagues with us but they kind of walked out of it by saying go and register with clinical trials.gov, but I'm not sure whether there again is any link between what you're required to tell clinical trials.gov and what you're required to tell a French equivalent of a human research ethics committee. It’s that kind of detail that makes the whole process tiresome, and in some cases, defeating for poorly resourced research teams, and especially if they don't have the money to keep somebody on the payroll, who's going to develop any expertise in this kind of thing, so they just have the most junior person of the team who knows nothing about it dipping in and out of this kind of stuff every so often. I mean, that's where you really lose quality. 

Leeza Osipenko: My final comment is that I think we just started and there's so much that needs to be discussed and pretty much every sentence you pronounced can be depicted in one hour discussion so Till, looking at you, I hope we'll be able to continue this theme. Lots of unresolved issues, lots of very interesting questions and comments, so let's treat this session as opening the can of worms. Hopefully we can keep not just talking about that but perhaps coming up with proactive solutions and making a difference at the policy level through people who can make the change and who are currently working on this, many of whom are you.