PRERECORDED TAPE TRANSCRIPTION
SHARED PATHOLOGY INFORMATICS NETWORK
RFA PRE-PROPOSAL MEETING
May 22, 2000
EBERLIN REPORTING SERVICE
14208 Piccadilly Road
Silver Spring, Maryland 20906
[This prerecorded tape transcription contains inaudible portions and speakers were not always identifiable as herein indicated.]
DR. [BERMAN, ed.]: Okay. Well, it is 12:30 and I think most of the people are probably here. The rest can catch up once they have come in.
The purpose of this meeting is to really clarify for the sake of the applicants the request for applications for the Shared Pathology Informatics Network. And, you know, we have found in the past or at least others have found in the past -- this is the first one I have conducted myself -- that it is useful to have some of the issues of a complex RFA, and this is a complex RFA, discussed in a public forum so that by the question and answer period people end up with a better idea of what is expected from the RFA.
Now before I get into any particulars of the RFA, I just want to give you seem [some, ed.]background and an overview for this project. It is really based on one of the key interests of the NCI.
NCI is interested in getting human tissues and data to cancer researchers so that progress can be made in the cancer field. There is a perception that this is something that is hard to do and if it cannot be done then that is a major, major impediment in any progress in cancer research.
In the past that has usually been done by creating, at really enormous expense, tissue repositories, essentially de novo repositories. And NCI now sees that there is an opportunity to do things a little differently.
Suppose that every cancer researcher had access to the hundreds of thousands of tissues and tissue related data that are currently being stored and archived in pathology departments. The question is, is it feasible to construct a model system that allows researchers to query the accumulated electronic pathology records from multiple large institutions and instantly determine the availability of tissue? That is basically the key question for this -- that the RFA is really trying to address.
I have to constantly speak into a microphone. By the way, this is -- this meeting is being transcribed and in the e-mail that probably most of you received or the letter that you received announcing this thing, there is an address for the -- if you want to read the transcription. But the transcription does place the burden that every word that is said from now on should be spoken into a microphone or else there will be no way to have it transcribed.
And basically the way this would work would be that there would be some sort of internet set up so that a cancer researcher can put a request into the internet and that request would -- you know, as far as request, what we are generally thinking of is sort of tissue availability.
Let's say you are interested in an unusual tumor, you would like to make a tissue microarray, you want to find out, you know, how -- what the availability is for that tissue. Maybe it is so rare that it is not even thinkable to try to build a microarray for rhabdoid tumors or somethign ilke that. You want to be able to determine quickly whether it is available, how many tissues are available in large institutions, the frequency that they occur in the institution. Or if you are doing, you know, really any kind of study that requires human tissues, you would want to start off by knowing something about the availability of it in the network data base.
So the query would go into the internet and here it has it directly going to the firewalls of the institutions but presumably there would be some sort of server that would take the query and maybe fix it up so that it would fit into an SQL or some sort of format that best suits the institution information systems that it is going to go to.
And then it would be distributed through the firewalls of the institutions and there can be any number of them.
And it would -- the institution would then accept that as a query as though it had been formulated with -- by a legitimate user within the institution and that query would then search through the database of the institution.
When we think of the database of the institution, we are thinking basically -- sort of the very minimal thing that we are expecting is the surgical pathology report dataset.
Other things such as the linkage to cancer registries, linkage to molecular biology data, CLINPATH data, treatment data, outcome data, anything else that a hospital might have that is related to the data in the surgical pathology report would be icing on the cake.
But the basic thing that is needed for this RFA would be the SURGPATH data and it would go through the data and produce a query output for that institution.
That query output would then have to be put into a format that, number one, cleanses it of confidential information related to the patient and, number two, syncs with the format used by the other institutions in the network consortium.
Then it would go back out to some -- into a network to some server and be collated with the other query -- institutional query replies and then sent back to the person who originated the request.
And the sort of ideas we have as sort of the main work of the RFA is that the institutions would have to accomplish -- is that they would have to agree on the common data elements that would be the things which would appear in the final report that would go out, and they would have to agree on the rules for converting the free text from the pathology reports into some sort of nomenclature or data element. Something that can be queried in a database.
So, for instance, the free text of the reports might be translated into a succession of SNOMED numbers and the SNOMED numbers might have some sort of relationship so that the report, even though it might be a little ambiguous, might be able to search on sort of different hierarchies of concepts.
So not -- I do not want to get into the technicalities but, you know, squamous cell carcinoma might be a type of neoplasm and then there might be a site for the squamous cell carcinoma so that the sites and types of tumors and disease can all be hooked up.
They will have to select and implement their search software so that if they -- if their own laboratory information system cannot do this kind of search then they would have to have -- to work on something so that the query could be handled so that their datasets could be searched.
They would have to adapt and interface software to accept the query from the internet and to authenticate it in case we only allow queries from certain people.
They would have to develop the procedures for protecting patient confidentiality and also for working out a schema that their IRBs can approve. And then they would have to make the system work and test the system. And that would be the main work of the RFA.
I want to give an -- having said that, I just want to go back now and talk a little bit about what we are going to do for the rest of the afternoon.
Hopefully, most of the work of this meeting is going to be done in the first couple of hours. I know that there are already people who have told me that they have plane flights that leave early so that they cannot stay for the entire time. I hope during the question and answer period that the questions for the first couple of hours will be general questions about the RFA and not particular questions about your own institutional sort of difficulties or the questions you have about how to respond to the RFA.
We will have a break at around 3:00 o'clock. We will come back and then for the remaining time, for those who are going to stay, then we will have time to ask more specific questions.
We have a number of people here besides myself who will be fielding questions and talking and one of them is Barbara Fisher. She is from Grants Administration and she will be answering questions -- she is already giving me a look -- related to budget and to the things that the RFA must have in order to prepare it in order to meet award criteria from Grants Administration Service.
Mike Kerwin has not come yet but there should be a member from Review to talk about how once the applications come in, what the review process will be and, hopefully, he will show up pretty soon but I can also answer some general questions.
We have Roger Amid (phonetic)[AAMODT, ed.], who is head of the Resources Development Branch, and this RFA is work of the Resources Development Branch in Cancer Diagnosis Program. He has a copy of the RFA and probably he will be the judge if a question comes up as to exactly what the interpretation is of the RFA.
And Sheila Taube, who is the Associate Director of the Cancer Diagnosis Program, is here as well and she has -- probably has the best concept of the overview of where this RFA fits into the general objectives that NCI has.
Okay. I would like to just speak a few minutes before we -- oh, and one other thing. Tom Puglisi from the Office of Protection from Research Risks is going to show up at around a quarter of 2:00 to talk about generally the issues of confidentiality that come up in projects that involve sort of data mining and retrospective studies using existing medical data sources so he will talk about that and also will be available for a while after that for questions.
I just want to say a few words based on the problems that sort of come up over and over again when people respond to complex RFA's or when people respond for the first time to these RFA's as opposed to just submitting their own investor initiated R01's.
This thing, you know, is not done just by a PA. It requires people with a lot of different talents and expertise to work together and not only do they have different talents but they also have very different perspectives into what -- sort of how these -- what these projects are really like and what their role might be.
This RFA will require at the minimum, I mean, pathologists, informaticians, network specialists, confidentiality experts. It might require people who are experts in security and cryptography, and it will also require nomenclature experts. It will require some -- the pathologists will probably have to be very savvy about nomenclature issues.
So when the RFA is written it has gotten to be written in a way that convinces the review panel, which will also be a collection of people with all these different areas of expertise, that the whole thing can work in a concerted way and I think that is a very difficult thing to do and people responding to the RFA should put a lot of effort into making the whole thing make sense to any of the people with those expertise, not just to -- it cannot make sense to just one type of person.
The other thing is that this RFA as opposed to an R01 has a lot of requirements in it and I am not going to -- I assume that the requirements will come up in the question and answer period but the -- because it has a lot of requirements and because the people who are reviewing this thing are only human, they are probably going to read these RFAs mostly to see whether or not the requirements specified in the RFA are met. In other words, to see how closely the RFA responds to the -- the application responds to the requirements of the RFA. So if they get a very, very -- even if it is a very well done and responsive RFA but it does not make it easy for them to judge, then you are sort of working -- doing a disservice to yourself so you want to write the RFA in such a way that it is very clear, you know, what the requirement is and what you have done to meet the requirement.
And it would not be a bad idea to include a table of contents that says, you know, exactly, you know, what all the sections are of your RFA and how -- and then in each section to somehow make it clear, either by bolding or by having a header or something, that this is the paragraph that you are going to say how you are going to respond to such and such a requirement in the RFA.
Again this is done mostly to make it humanly possible for the review -- for the reviewers to read your RFA and understand that you have done what you are supposed to do without their spending a zillion hours going through and looking back and forth to see what you have done.
The other thing is that the -- every service grant or assistance mechanism has a section in it that requires innovation and especially people who have been writing R01s. They will read an RFA and sort of think about all the innovative things that they can do and I think that when you write your applications for an RFA like this, you should devote your innovation to innovatively conforming to the requirements of the RFA.
In other words, how smart can you be at convincing the people in the review panel that you have a really great innovative way of doing exactly what the RFA says you should be doing. Not how great are you at taking the RFA and using that as a starting point for something related and maybe even more worthwhile than the RFA. Who knows? But not really responsive to the RFA project. So devote your innovation to responding to those terms of the RFA.
The other thing is that the -- you are required to come in with your application with a preformed consortium of three to five institutions. That makes the RFA doubly difficult to respond to in your application because you have got to somehow convince the review panel not only do you know what you are doing but all your consortium members know what they are doing and it is all being done together in a way that is going to work out.
You might not be able to convince the review panel that all your consortium members can really do that and you might want to consider when you write your application to make it very clear what each consortium member is bringing to the application and even writing a separate budget for each consortium member with the idea in mind that it is possible that the review panel might love your application and hate one of your consortium members, and might want to sort of cut out a consortium member that they think is very weak and they cannot do that unless the budgets and the sort of work that you say is going to be done delineates who is going to do what.
So it might be to your benefit to have well defined consortium members and what their work is.
Okay. The other thing is consultants I wanted to mention. I have already had a few discussions and most people -- a common question is can we call in consultants, especially consultants who might be in the company of the laboratory information system because you would have to do modifications in your own search engine software. The answer is yes, of course, you want to call in consultants.
You might want to consider that when you bring in a consultant, in addition to just identifying them, you might want to consider having a letter from the consultant that the review panel can read and the letter probably should have something in addition to just yes, I work for such and such a company and this project is very interesting and I will be working with such and such a group if they get the award.
Probably it might be good to have the consultants indicate in the letter that they know what the project is, that they understand what their role in the project will be, that they -- if they have done related work, to include that related work in that cover letter, to include words to sort of allay any suspicions that the review panel might have that the consultants might not be serious about this or might not really, you know, be ready to put in the effort that is required for this.
You want to make sure that that -- at least you have -- I do not want to tell you what to do but you should consider, you know, having a consultant letter that really convinces the review panel that this is going to be a working consultant that understands the deal and everything is going to be okay if this consultant is working on the project.
Okay. And that is really all that I wanted to say. Again, Barbara Fisher is here for questions that involve the budget and other grants administration issues and we have other people.
And this is being transcribed. This is a public arena. In theory, you do not have to identify yourselves if you want to -- if you feel that there is some advantage to being an unknown entity, and even if I know you and I call on you for questions, I am not going to mention your name. If you want to mention your name, you can.
We can start questions.
Pull the microphone, first.
DR. __________: Just a quick question. I was reading over the RFP this morning. I am assuming that when you come in, you will need an IRB approval. It seemed like there might be a two stage process to that. Sort of one that accompanies the early -- you know, the application, and then some more detailed issues that might get addressed when somebody actually tried to access tissue. I do not know. Maybe they are one and the same or -- anybody care to comment on that?
DR. __________: [DR TAUBE, ed.], you look like you want to answer that?
DR. __________: I mean, I am really asking a procedural question.
DR. __________: Yes.
DR. __________: Not a philosophical --
DR. [TAUBE, ed.]__________: Yes, it is actually a very important question. Certainly you are correct in that the IRB approval will be in two stages. Initially you are going to get approval basically for the schema of what you are planning to do but not for the specifics because the specifics are going to be dealt with later in the process. However, I want to make very clear that there is no intention in this five-year project to ever access any tissue. All we want to know is that you could access the tissue and so we are asking for a statement from you and from your institution that the tissues related to the data that you are going to be collecting in this project would be available or could be available under some circumstance.
DR. __________: Can I build on that question and ask does that mean every member of the consortia has to have an IRB for this grant?
DR. __________: Yes.
DR. __________: Okay. The answer is yes?[yes., not yes?, ed.]
DR. __________: Yes.
DR. __________: Could I ask another question then? In terms of the grant resources and budget, does that mean every institution in the consortia has to sign off on each individual budget for their particular part of the project?
DR. __________: That is correct. In order to provide the individual -- that ia correct. In order to provide the individual IRB certification that is on the face page that we will need in order to issue the grant, there should be a separate face page for each of the third party participants as long -- as well as a budget and that would provide us all the information up front.
DR. __________: You call them third party but they are actually first party?
DR. __________: Members of the consortium are third party.
DR. __________: Yes. Members of the consortium are third party participants.
DR. __________: Okay.
DR.[BERMAN, ed.] __________: Barbara, maybe -- the due date for this thing is like July 24th. Okay. So when they come in with their application, they have to come in with a project assurance. Before they get the award they have to have the IRB approval but they do not need to come in on July 24th with the thing being already approved by IRB; is that right?
DR.[FISHER, ed.] __________: That is correct. According to the new guidelines that NIH has, it states that applications can come in with pending as far as the review but prior to award we will need the IRB date.
DR. MILLER: Bob Miller from Johns Hopkins. Jules, you alluded to many organizational technical challenges here. Could you talk a little bit about the interaction between consortia, presuming that multiple consortia will be funded, and the role of the NCI and the project personnel in that interaction?
DR. [BERMAN, ed.]__________: Okay. The -- let me tell you a little bit about the history. Originally we were just going to have individuals come in, not preformed consortia, and the general conception was that it would be just too tough to get people coming in who had completely different ideas who had never met before to gather around at a table and do all these things together that require a lot of cooperation. So we figured, well, let's cut down the problem a little bit by making lead PIs form a consortia that they have proven they can work with by responding to the RFA and writing an application that would do a lot of the work of the cooperation in order to even respond to this RFA. Then we will pick probably two of those consortia and then those two then would form a working group, the PIs would form a coordinating committee that they would work with program director, presumably myself, and also we would get help from the Office of Informatics. And that -- those two consortia would then be able to come to agreements as to how to go forward.
We did not want to just have one consortium sort of get the award because then it would be too easy in a sense. We -- since this thing is -- you can think of it sort of as a feasibility study, although it is more than that because we want at the end of the five years to have a working network, we also are interested in the general feasibility of having disparate institutions with different types of data and different types of information systems being able to network together.
So if we -- so we wanted to make it a little tough by at least having two consortia getting together and having to come to grips with the issue of making things work across different institutions so there will be a coordinating committee. There will be meetings. At the meetings it will be decided how to proceed as far as what nomenclature to use, what rules will be used for translating free text of the nomenclature, what the, you know, report format will be. Sort of what the schema will be of the security for this whole thing.
Now your -- you might say, well, if the coordinating committee is going to decide all these things, why do we even have to write any of that in the RFA?
Well, we need to look to see that the consortia coming in have a clear concept of what the RFA is about, what they can bring to the -- what they can bring to this network project, that they can have a solution in mind for all these issues, including the confidentiality issues.
So your solution might not be the one that is finally chosen but your solution will be one that the review panel will look at and they will say, ah, these people know what they are doing. They can work with another consortium and make something happen.
DR. [AAMODT, ed.]____: Yes. Let me elaborate a little bit on that. The reason that we ask for two preformed consortia is because we wanted to have very clear lines of responsibility and central control for each consortia group.
In terms of the coordinating committee, it is very clear in the RFA that the coordinating committee has to have representatives from each of the participant groups so it is not just the PIs getting together but it is the PIs and the lead individual from each of the consortia groups coming together to form a committee that will make all of the kinds of decisions that Jules just mentioned.
DR. [BERMAN, ed.]__________: It is a little confusing because I think in the RFA each institution -- we have sort of different level PIs so there is the lead PI and then there is going to be a designated person who is the PI from each of the consortium -- from each place is also called a PI. So there is a lead PI and there is consortium PIs.
The sentence that Sheila wants me to read is "The consortium principal investigators for the parent institution --" this is for the -- what the coordinating committee will consist of -- "The consortium principal investigators for the parent institutions, the principal investigators for each funded consortia site, and the program coordinator."
DR. __________: Ownership of software solutions?
DR.[BERMAN, ed.] __________: That is an interesting question. And the RFA does not go much into that. The RFA says that your data is your's and the -- I really do not know exactly how to answer that as far as what the ownership will be of the software that comes out.
I mean, I can give you an opinion. Roger, if you have an opinion on this you can -- it is a complex issue and maybe you might have something you want to say.
DR. [AAMODT, ed.]__________: Yes. I think I can clarify it a little bit. Because these are done as grants, basically any products that come out of these belong to the institution. On the other hand, since it is being done collaboratively and it is going to be difficult to pick a single institution, I think there is going to be a need at some point for the coordinating committee and probably the institution business officials at all of the participating institutions to come to some conclusion about how to deal with that.
The other point is that to the extent that you are dealing with commercial software, and many of these things can be done with commercial software, the issue will not come up at all because it will be perhaps some minor adaptations rather than starting fresh.
DR. [BERMAN, ed.]__________: Do you want to say something?
DR. [TAUBE, ed.]__________: Yeah. No, I just -- I just wanted to add a little bit more to that and that -- when we discussed this, and it was discussed at the Board of Scientific Advisors as well, if you are modifying software that you are using in your institution, the company that is providing that software obviously will own those modifications and in addition they would really only be useful in conjunction with your software program there.
So there is probably little that will come out of this that will be sort of generalizable. The only thing is if at some point it is decided that the actual translation in the end to form a report, a standardized report, becomes something that is uniform, then I guess we will have to deal with that at the time and I do not know, you know -- I mean, that will be a discussion item.
DR. [BERMAN, ed.]__________: Yes. I mean, I think that the two things most likely to be of interest would be the mechanism to go from free text to a nomenclature and the software that is agreed upon by the consortium.
And the other one that might conceivably be of some interest would be the -- assuming it is a software approach to the de-identification of the data. And even the very concept of de-identification of data is something which has not -- people have not been giving a lot of thought to. Although in the -- I do not want to go too much into the technicalities. In the HIPAA regulation there is essentially a -- the proposed rule, there is a request for people to start thinking about what de-identification of data is and how it can be done.
But those areas might have some -- assuming that this -- the people in the network can come up with a very good solution to these issues, might have some value but again I -- it would be something that the coordinating committee would have to face as to how it would be dealt with.
DR. __________: To what extent is the software at the NIH/NCI coordinating site something that applicants can propose or is that something that is independent of whatever the applicants do?
DR. [BERMAN, ed.]__________: You mean the software that would be made available to the consortium by NCI? Is that what you are talking about?
DR. __________: No, the other way around. Specifically presumably NCI will eventually have a resource that it provides to the public or some restricted subset thereof that allows them to query for the tissue samples and related data eventually. Is that software that does the querying something that NCI will directly develop on its own independent of applications or is that something that applicants can address?
DR. [BERMAN, ed.]__________: The software that does the querying would be something that either you would develop or take from some -- or modify that someone else has developed. I mean --
DR. __________: Well, there is local level querying and then there is central level querying. The question is whether the central level querying at NCI is also part of the thing.
DR. [BERMAN, ed.]__________: The central level querying at NCI?
DR. __________: The local sites.
DR. [BERMAN, ed.]__________: Can we try to clarify exactly what the question is? Are you talking about supposing in the long-term, and this is very long-term now, not within this time frame basically, but if the NCI were to incorporate this approach, and actually make specimens available, we would have some central entry point where people would query? Is that the point that you are talking about?
DR. __________: Right. The question is the front door for the public to get access to the system.
DR. __________: Right. I --
DR. __________: The model could be distributed that you query everywhere that wants to listen to a query or that there is a front door.
DR. __________: I -- right. Okay. Go ahead. Finish.
DR. __________: And the question I had was whether or not developing that front door could be part of what a consortium applicant proposes?
DR. [BERMAN, ed.]__________: No. I -- well, I mean, you could propose that but that is further down the line. The main issue that you need to address at this time are the issues of figuring out how you are actually going to -- if you have a query in hand, not how you get the query into your hands, but if you have a query in hand how are you going to distribute that to existing databases and pull back the information.
Ultimately, if we set up some sort of front door for investigators to query, that probably would be an NCI contract, I would guess, in the long run where we would set up a web site and query tools and so on. But it is from that front door on that we are addressing at this point.
Did that answer your question?
DR. __________: Yes.
DR. __________: Where our focus is for this first five years is basically what wa s shown on the cartoon that Jules showed earlier. So there would be some sort of an internet request server that will be a private site, not a public site, and from there sending the query to and getting an answer back from the institutions will be the main job of the consortiums that are funded under this RFA.
DR. __________: Before you turn the slide off, I wanted to ask, you have shown a -- in some ways -- quite specific architecture. Can you talk about the extent to which that is a requirement in this RFA?
DR. [BERMAN, ed.]__________: We have a concept but it is certainly not a requirement of the RFA. If you can come up with a clever idea that is not the same as our clever idea, that is perfectly acceptable.
DR. __________: Okay.
DR. __________: Could I just follow-up real quickly on that? I mean, in preliminary discussion -- you do not envision querying the in-house database of a hospital each time, you know -- again this is down the road but it is still relevant. That is sort of the way the arrows look there. I mean, I was envisioning another box, you know.
DR. [BERMAN, ed.]__________: A warehouse box you mean?
DR. __________: Yes.
DR. [BERMAN, ed.]__________: Yes.
DR. __________: I mean, that would also be behind the firewall but --
DR. __________: Yes.
DR. [BERMAN, ed.]__________: I mean, my concept is if you can actually use the search software that the hospital has to query its own data without making a warehouse and do it in an efficient way, then great. All the more power to you. If you -- but I also envision that in practical -- for practical reasons, it might be more feasible to have something where the hospital database is constantly feeding into a warehouse and then the query is done on the warehouse data, and then it goes back to the network. So but -- you know, whatever you --
(End Tape 1, Side A.)
DR. [BERMAN, ed.]__________: -- want to do.
But if you have a warehouse, it cannot be a static warehouse. It cannot be one that is built once a year and then next year you refresh the data in it. It has got to be something that is constantly being piped in with data from the hospital database.
DR. __________: Right. But would not necessarily have to be real time either.
DR. [BERMAN, ed.]__________: Would not have to be real time but it would have to convince the review panel that it is close enough to real time to sort of meet the general concept of what this RFA is about. So, you know, like every day, I guess. I do not know what the review panel will be thinking but I guess I am thinking in terms of every day or every few days. Not necessarily every day.
DR. __________: I think somewhere in the range of every day to once a week. You certainly probably do not want the data to get much more out of date than that but it also depends on the rate at which the hospital is accruing new cases.
DR. __________: You mentioned briefly the vision of the NCI view in five years. Can you talk a little bit about that vision and the implications of scaling to this current RFA.
DR.[BERMAN, ed.] __________: That is a good question. I mean, we -- the idea is that this thing should be like infinitely scale-able, that not only can it add on additional pathology departments but it should be able to add on like outpatient clinics and cancer registries and, you know, any source of data and would be able to take the query and generalize it so that it can go over to genome databases and microarray databases, and all that sort of thing. But --
DR. __________: We would call that extensible.
DR. __________: Right, extensible.
DR. __________: So it extends the --
DR. [BERMAN, ed.]__________: Yes. And that is sort of, you know, our hope for the distant future but for this RFA, just for, you know, writing an application, you want to be careful to -- not to write something that is suitable only to the members of your consortium. You want to make something that, you know, will be able to segue into other projects but you are not responsible for having lots of different projects in your RFA.
Then the other thing that, of course, we are interested in, and again it has already been mentioned but I will just sort of mention it again, is that the primary -- I do not know if I want to use the word "primary" but one of the key goals that NCI has in mind is that -- is to be able to retrieve tissue for -- and related data for researchers.
So you want to be able to always think in terms of adding on or working with data that links somehow to a piece of tissue at the end of it. If it -- and that -- so that at the end of the five year period that the network could presumably be in a very good position to -- the members of the consortium would be in a position to compete for something that the NCI comes out with where this network, possibly with different added on consortia because there should be a way of adding on different consortia, could actually start providing tissues.
DR.[AAMODT, ed.] __________: Okay. I just wanted to make one minor comment in relation to that. The end goal of this project really is to develop a model system that would allow each of the consortium participants systems to interact with the central system. So that is a fairly constrained goal.
Ultimately we would envision that this model system could be expanded and cloned in various places so that any group that wanted to set up a network of activities related to tissue or related to data or anything else could make use of the techniques that are developed in this process.
DR. [BERMAN, ed.]__________: Also, the reason that we are asking for consortia with a number of members relates to this scale-ability issue and that is we want the consortium members to actually be using somewhat different informatics systems in their individual institutions and part of what we are doing within this time frame, within the first five years of this, is to make sure that institutions with different systems can communicate. And the reason again for two consortia is that we will hopefully include major laboratory information systems that are in use around the country in those two consortia and try to link those together so the building of the various hand shaking software tools will help ultimately make it possible, we hope, to expand this beyond the initial institutions that are involved in this initiative.
DR. __________: Are you planning to end up with two different networks or one network?
DR.[BERMAN, ed.] __________: One network.
DR. __________: One network. Okay. I just wanted to make that clear. That is what I thought.
DR. __________: Yes.
DR. __________: If one or more members of the consortium has fewer than ten years of archived material, you mentioned ten years as some sort of guideline or --
DR. [BERMAN, ed.]__________: It was not mentioned as a requirement.
DR. __________: Not as a requirement. But you say most institutions have more than ten years. Is that going to impact on the prioritizing of that application?
Or just to use another number, are the number of specimens per year going to be significant? Is there a minimum that you are looking for or a total number of specimens, years times specimens per year that you would consider to be a minimum acceptable number?
DR. [BERMAN, ed.]__________: No. But you just have to realize that the review panel is going to have all the different applications in front of them and if somebody comes in with an application where they have got everything all figured out for a million specimens, and someone comes -- has everything all figured out for ten specimens, it is going to count against them.
So as far as, you know, what we are thinking of is where -- you know, what we -- what program would like, and that really has no significance especially to the review panel because we are not going to tell the review panel how many is the right number.
But I was thinking in terms of consortium members coming in with -- I do not know. In the range of at least 10,000 specimens per year and maybe several years of -- that is each institution. At least several years of data so that the review panel is going to look at how -- you know, how much -- how many cases you bring in, in your application.
They will also -- you know, there is also wording in there that you have got to demonstrate that the cases that you have in your institutions are good cases. There is some wording in there that -- let's say you have -- for a surgical pathology report, is there a place for clinical history? Is there a place for growth? Is there a place for diagnosis? Is there a place for the date the specimen was taken? The date of birth of the patient and all that sort of thing.
And then I think there is wording in there that you have got to provide some sort of information relating to -- for those sort of add elements in your surgical pathology reports, what percentage of them are actually filled in on your report, so is there a clinical history for all your SURGPATH cases. And of those filled in, how many of them are filled in, in a sensible way.
So if date of birth is one of them, and I assume that it will be one of them, is it -- can you be sure that the date of birth is always a sensible number? Is it going to be -- you know, is the date of birth, you know, October 15th, 1942, or is it going -- is your date of birth five, you know, something that does not make any sense.
So you have got to show that your reports have data fields that are relevant and that the data -- whether or not the data fields are filled in and whether or not the filled in data fields make any sense.
DR.[AAMODT, ed.] __________: Yes. Let me restate Jules' answer in sort of more general terms.
There are basically three areas in the RFA that you should really focus on. One is the responsibilities under the terms and conditions of award of the institutions and of the NCI. The second is under application procedures where there is some very specific requirements in this RFA that you should focus on and make sure you have met. And the final one is in the back under review considerations, the review criteria.
And to some extent, you should really write your application to those review criteria because that is what the reviewers are going to be using when they evaluate the application.
So regardless of numbers, as long as you have met all of those requirements, basically you are going to have a viable application and the degree to which you really focus in on the issues that the reviewers are going to focus in on will improve your standing in the ranking of the reviewed applications.
DR. [BERMAN, ed.]__________: It might be worthwhile for us to just stop and just take a moment to look at -- you do not all have the -- anyone not have the RFA? We will give you a copy. Page 7. It lists things that you really have to do. And just take a look over those and maybe some of them might just strike you as you glance over them as things that you might have questions about.
One that -- I have gotten a number of calls -- telephone calls. I think I have gotten a telephone call that probably asks questions about every single one of these things.
One that -- let me just go over number three, which is related to the last question. Applicants will provide a measure of the completeness and validity of their datasets. Completeness refers to whether a data element in a given data record contains data. Validity refers to whether the data element contains sensible information. You might want to have a look at that paragraph.
I am looking for the paragraph that describes the letter -- the statement. Number five is -- some people find to be a toughie.
Applicants -- I am going down to the second sentence. Applicants will include a statement from each consortium institution regarding its willingness to provide archived specimens and related data to researchers within and outside the consortium institutions. Include, if applicable, past history of the institution in providing specimens for research.
The statement of willingness is requested to allow the NCI to evaluate whether network institutions would participate in a subsequent virtual resource to make archived tissue and data accessible to researchers. The NCI understands that these arrangements could be effectively negated by circumstances beyond the control of the participating institutions.
This means that you have got to have a statement or an attached letter signed by --
DR. __________: Just the institutional --
DR.[BERMAN, ed.] __________: -- the institutional official. Presumably the same institutional official who is signing the front page of the PHS398 application.
And again it does not commit you to anything but it tells us that the institution knows and is willing at this point to say that it would like to be part of a network that does do some work in distributing tissues at some point in the future.
DR. [TAUBE, ed.]__________: Yes. This came up because when the Board of Scientific Advisors reviewed this concept, a number of members of the board are very well aware of the problems that they and colleagues have encountered within their own institutions or when engaging in collaborative research with people from other institutions that often people do not want to give up specimens.
So although no specimens will be requested during this five year period, and I want to re-emphasize that -- the board members said that they wanted at least to have institutions committing themselves to the idea that they would be willing to participate in research and to allow specimens from their institutions to be used for research some time in the future, and that is where that comes from.
DR. __________: Within that same context, we can extrapolate out when you do want tissues exchanged. The question of intellectual property in a sense is who publishes what. If I am an investigator, I got to the network, I query, I gather from five institutions cases, the individuals at those institutions currently might certainly want to have done that themselves or be a part of the process.
Do you envision some way that you can assuage those fears basically?
DR.[AAMODT, ed.] __________: Well, I -- the only way that I can actually address that is that we currently support resources like the Cooperative Breast Tissue Cancer resource where the -- we have over 9,000 cases where the principal investigators of the institutions have put -- have pulled the hard way all of the information from charts and registry information into a centralized database. They do not get involved in research efforts that are pursued by applicants to the resource unless special information or testing of the specimens is required. In which case there is an intellectual input to these -- to the actual research project.
However, we have not really decided how we would set the large virtual resource in the sky here up and so there could be many different ways of approaching it.
However, we think that unless there is information beyond the basic dataset that already resides in your electronic databases, you know, and -- so that the individuals who collected those cases are not really intellectually involved in the research project that probably we would -- I think that it would be unreasonable for those people to believe that they can control all of the cases.
This is an ongoing battle and, you know, obviously there are many differences of opinion on this.
DR. [TAUBE, ed.]__________: Yes. I think your question, though, raises another question, which is a really critical one not in terms of this project but in terms of the follow-on to it that we have been thinking about, and that is that there is probably some need for incentives to get people to play in this area. So part of what I would hope this group would do in the five years that you are going to be working on the project would be to think about what kinds of compensation, what kinds of incentives there should be for giving away data, giving away specimens.
I could envision, for instance, a system where there was sort of a royalty. Every time somebody accessed your system and took data away from it and accessed your specimens, the institution would get a certain amount of money that would come from charges that were levied on those people that were requesting the specimens.
But it is an area that needs a lot of work and a lot of thinking and certainly five years from now or ten years from now or whenever we get to the real project of making tissues available, we are going to have to have some ideas about how we are going to deal with those issues.
DR. __________: Just to follow-up, I mean the reason I bring it up now is because there -- I mean, I perceive a certain level of resistance. I am not a pathologist. I am in the informatics medicine side of it as to why should I participate in this. I used my brian [brain, ed.] to make this diagnosis, therefore I have some, you know, control in this.
DR. __________: But you got paid for making that diagnosis already.
DR. __________: Wait, wait a second. But they are going to have to change the way they do some of their things, you know, the way they label blocks, the way they do some other logistical things, and you need -- you are keeping track of stuff internally in order to track back.
So there has to be a little bit of discussion of this anyway up front even though you are not going to distribute tissues.
DR. [TAUBE, ed.]__________: I can assure you there has been and will continue to be considerable discussion of those questions all the way through the project. And as Roger just said, the issue of the types of incentives that will be required are issues that will be part of the discussion as we go forward.
DR. __________: Right. But in terms of your own specific needs basically what you are going to have to do is find ways to build into your project -- into the project budget or however you choose to do it, the incentives that you need locally to make this happen.
DR. __________: I just --
DR. __________: Okay.
DR. __________: Well, wait a minute. [name deleted, ed.], you had a question?
DR. __________: No, I was just pointing out he has been very patient.
DR. __________: All right. Sorry.
DR. __________: Too patient.
DR. __________: That is all right.
DR. __________: A question about the quality data requirements that you have listed in some ways go against HIPAA and a lot of other things in terms of confidentiality and I guess a question -- if you have the date of birth of an individual, the date that they had a biopsy for some rare tumor, basically that is enough to identify that person pretty well.
DR. __________: Sure.
DR. __________: And so the fact that we are going to be providing information as applicant institutions on that, probably the IRBs will get a little bit uppity if that is looking like a promise to provide that information to some central resource.
DR. [BERMAN, ed.]__________: The only thing you will have to provide is proof that you have that information. When the actual query reply goes out, it is going to be de-identified. You are going to take that stuff and, you know, date of birth might be converted to year and year of birth and, you know, everything is going to either be modified or encrypted so that when you do send something out you do not have to worry about those things.
DR. [AAMODT, ed.]__________: That is part of what you have to deal with when you write the application, how you will address those issues and that is what the coordinating committee will deal with as you set up the whole database or a system, I should say. It is not a database but the system and what information will go out. How you will represent each of the fields in such a way so that the patients will not be identifiable.
DR. __________: One other unrelated question. If a consortium somehow came up with a scheme that could meet the objectives in three or four years instead of five, is it appropriate to go on and propose other things or would it be better to cut the budget off at that point?
DR. [BERMAN, ed.]__________: You are -- I would not recommend it. See, the -- we might as well talk a little bit about the time line. We put in there a time line for doing this thing in five years and some sort of general scheme for how it could be done. I forgot. At an organization phase for years one and two, a component selection phase where you either sort of make your own software or pick it out from some place, figuring out all this handshaking work, then a testing and validation phase for years 3.5 to five.
You will have to come in with a time line. If you come in with a time line -- let's say you think you can do it all in two years. The problem will be what other consortium are we going to -- assuming that your application is a good application that the review panel likes in every way and you have this two year or three year time line, how -- what are you -- how are you going to work with the other consortium who comes in with a five year time line and cannot get their data together in the time that you get your data together and all that.
I think that it would -- you are not required to come in with a five year time line but I just do not see how that would help you to come in with a short time line.
DR. [AAMODT, ed.]__________: One of the important things about this kind of a mechanism where we are putting groups together to accomplish a project is in a lot of ways it becomes a do it yourself project. Once the group forms and the coordinating committee starts to meet, you will get down to a lot of planning about how you are going to work together and what the end products are going to be.
So whatever you have in mind when you start may not be what ultimately results. I think that is a real strong reason why you should think more in terms of the time line that is proposed here rather than something that you think you can accomplish more quickly.
If it did turn out that the group got together and they were really rapid and got everything up and running in two or three years, there is nothing to stop us from advancing the time table and just moving this project along in the remaining two years so that we could accomplish more than we are envisioning right now.
DR. [BERMAN, ed.]__________: I will get you in just a second. You guys are so lucky. Not only do you get an answer to your questions, you get three or four answers to your questions.
DR. [BERMAN, ed.]__________: But the fudging around with the time line for your application would probably be -- let's say you are a place that has all these security issues and all the handshaking issues, and all the -- you are very, very, very strong on the networking issues. Well, then you would probably want a condensed time line for the networking stuff but you may not be that good at getting all your data together and, you know, putting it in the -- you know, checking out what you have and maybe the testing and validation thing, you have very extensive plans for validation and that would take a long time.
I mean, you have to -- the -- I would probably -- again I do not want to advise you what to do but it might be reasonable to take what we have here and then to contract or expand things based on your weaknesses and strengths and how much time you have to put into things.
Probably the worst thing you could do is to -- again I do not want to tell you what to do but if you have in your mind certain weaknesses, the worst thing to do would be to send in an application that sort of gives very little time to that weakness thing and sort of makes it like not an important part of your application. And then devote all your time sort of writing the application to your strengths.
My -- certainly we do not want that from program because we want an application that really does reflect the reality of the situation and I really -- my guess is it would not help you much because the review panel is going to realize that every place has strengths and weaknesses, and you have got to address, you know, what you can do and maybe take more time to do things if you are a little weak in an area.
DR. [AAMODT, ed.]__________: Yes. The other thing to think about as you are putting this together is that you really should look for the areas where there are going to be pitfalls along the way. The places where if something is not accomplished you are going to have to find some other way of doing it and to the extent that you can recognize those and tell the reviewers, yes, we know this might happen, and when we get there if this approach does not work, we have another approach that will work, and we will try that. Or maybe we have three approaches that might work and we will go from one to another. Your application is much likely -- much more likely to be reviewed favorably.
DR. __________: No, you were next, Bob. I told you --
DR. __________: I guess I was going to say another way that the question from the other end of the table could be viewed is from the point of view of applicant initiated enlargement of the work scope. I think that is what I heard.
DR. __________: Applicant initiated enlargement?
DR. __________: Enlargement of the work scope. If we can get done in a shorter period of time. You commented on deviations from the requirements of the application.
DR. [BERMAN, ed.]__________: Right. You can do that but be careful when you do that. You know, there is already -- I have already gotten some questions. I mean, one of the things that -- I have sort of gotten two types of questions as far as the -- for enlargement of the work scope. One is really this kind of thing would be best done if there was an image archive that goes along with every single specimen. We would like to budget, you know, a million dollars for the image archive.
The other thing that sort of comes in is really this kind of thing is something that does not mean anything unless you have a whole bunch of treatment data and we have all this treatment data that we have been collecting and, you know, we want to mostly budget the sort of treatment aspect of it and maybe not deal that much with having large numbers of surgical pathology cases and all that.
My advice, and it is hard to give advice on something like that, is that if you are not -- like for the imaging thing, which by the way -- I mean, I think is a terrific idea. We just could not get it into this application. But that is the kind of thing that probably might be best included if you get an award to then try to get supplemental funds for something like that because it would probably not be considered by the review panel in a complex RFA to be directly related to what you should be responding to. And I might be wrong about that.
The thing about, you know -- let's say you are a cancer registry so, you know, the consortium member is a large cancer registry that has all this, you know, treatment information and outcome information, pathology information, maybe not as many pathology reports and maybe, you know, excluding a large variety of the types of reports in a pathology department because they will have cancer cases only and then only cancer -- they will not have skin maybe, they will not have precancerous lesions.
Well, then that is sort of another issue because that is what they are. They cannot decide, well, we are not going to go forward with this as an expansion of the RFA because that is what they are. They cannot -- they cannot -- if they did not go with that expansion of the RFA then they would not have any response to the RFA. They would not have an application.
So then their job would be to convince the review panel that what they have is, in fact, something that contains the information that the RFA is asking for and which they can fit into the requirements so that it is a proper response but then they -- because they are doing something else from what the typical probably RFA response would be, which is a bunch of pathology departments with a whole lot of pathology data, then they have a lot of extra work to do to show that their expansion is absolutely justified.
DR. [TAUBE, ed.]__________: Let me just make one thing clear. This whole project is really to support cancer research. Therefore, we are definitely interested. There will be an emphasis on cancer tissues, premalignant tissues and any normal tissue that is associated from those cases. And, also, we are very interested in treatment information if it exists because we want to relate -- I mean, many researchers will want to relate various kinds of markers or pathways or mutations or whatever to outcomes that could be related to treatment.
What we are concerned about is that treatment related -- that treatment information is not easy to get and so we are not counting on the fact that this will be available in all cases but I do not want you to get the impression that we are not interested in treatment information and only interested in what is on the original surgical PATH report.
DR. __________: Yes.
DR. __________: We are interested in as much clinical information associated with a case as we can get.
DR.[BERMAN, ed.] __________: I did not want to give the impression that we were not. Just that there is language in the RFA that talks about things like laboratory information system, describe your LIS, describe your hospital information system if you are going to be working off a hospital information system. So someone coming in as a registry has got to write the response in such a way that it makes it very clear that there -- they have an information system for the registry. Maybe it is not a laboratory information system. And they have all these good things to offer treatment or whatever so they have to make their expansion of the -- of some of the points of the RFA into something which is -- works to their benefit in the review panel.
DR. __________: Yes, what would be even better is if you were a registry to make part of your consortium people who actually had pathology information systems in their pathology laboratories who could be more direct participants in this but link it to the registry data in a way that you make it a richer dataset.
The key element here, as we have said several times, is to be sure that everything is linked to a piece of tissue somewhere.
DR. __________: We keep talking about tissue. Are we only talking about surgical pathology paraffin sections or are we talking about something else?
DR. __________: Frozen tissues.
DR. __________: Frozen tissue, pleural fluids, you know, the gamut, cytology, cytopathology.
DR. [TAUBE, ed.]__________: All of those specimens I consider tissue. I mean, blood is tissue. Anything that has cells in it is tissue. I mean, from our perspective they relate to the case and give information about the course of disease and maybe the diagnostic material that you use.
You said something at the -- the frozen, yes. If you have frozen tissue and that is all you have, that is fine. I mean, there is no problem with that. The issue is just that since the major -- the overriding rationale for this relates to making something physical available for researchers that relates to a case, not just data, it is not simply for data mining, it can be used for data mining but it is not only for data mining. It is for actual research on specimens, that there should be, in theory, at least a specimen that is related to every case.
DR. __________: Just to explain it to pathology departments that are a consortium, it makes it -- it is different if you say to a pathologist I want to be able to search to get paraffin sections from your cases as opposed to I want paraffin blocks or I want this or that.
DR. [TAUBE, ed.]__________: Oh, no, we would never ask. Just as with our currently supported resources, we do not give out the blocks. We only give out sections and we have -- and every group has put in place rules that they do not give -- they do not exhaust the clinical diagnostic material that could be requested for ongoing medical care for the patient. So that is definitely -- you do not have to worry about that.
DR. __________: So I was wondering what your optimal consortium would be. What would make up a good group of institutions? I was kind of thinking along these two lines: One was what distance would be appropriate? So,for instance, you know, is there -- is it seen as good or bad if they were close or far away from each other? And, two, what about a business relationship between some of the institutions? Is it frowned upon if there is like a preexisting business relationship or is that a good thing because they are more used to working together? That kind of thing.
DR. __________: Yes, I mean, to -- well, first -- Okay. Close -- when you are talking about closeness, you are not talking about geographic closeness at all?
DR. __________: No, actually I am.
DR. [BERMAN, ed.]__________: Okay. Geographic closeness. We do not care. I mean, as far as, I think, we are concerned, it would be nicer to have it geographically distributed so that you can show that it is not just you are sending out a query to, you know, one place. I mean, it is -- if you can work it in a convincing way to the review panel that you can manage this thing over a distance, I think that would be a plus but then you have the added job of being able to convince them that you can really do it.
As far as preexisting business relationships with these places that might be spread over a distance, you know, this thing does not even restrict itself to academic institutions. It can be for profit or not for profit and you want to think about writing your application in such a way that you turn that into a strong point by showing that, you know, you have a working relationship with these people. You have made things happen. You have made things happen which have, you know, worked out well for you.
And I do not think you should probably get real into the money aspect of it but things have been real nice working with these guys and we have made things click and I think that that would probably work in your favor. But again you want to respond to the RFA in a way that makes everything click.
DR. [AAMODT, ed.]__________: The trade off on geography is always that if your groups are widely separated, it becomes harder to coordinate them. If they are close together, you have got to deal with the issue of are they really separate?
I mean, for instance, if you had multiple pathology departments in one institution, theoretically you could come in with a consortium that all represented one institution but I suspect the reviewers would be less interested in that.
So basically both of your questions, I think, end up being sort of neutral issues as far as the RFA goes but you have got to think about the ramifications of anything that you choose.
DR. __________: Jules, the more I look at what is -- you know, you are saying needs to be compressed in this application and the fact that it is supposed to fit under -- I assume, the usual 25 page format of a PHS398, which is kind of an R01, it is kind of difficult to conceive how this is all going to get crammed into one space.
So what are the flexibilities between the construction of the grant and pushing out appendices that describe the individual institution's LIS systems and their components, and validation and all of these requirements? Or do you want to really just keep it to the 25 page limit and pick your words very carefully and eliminate all consonants?
DR. [BERMAN, ed.]__________: Roger, when you finish this --
DR. [AAMODT, ed.]__________: Yes, okay. Yes, I think what you want to do to the extent possible is use broad brush strokes, try to be conceptual rather than detailed. You have got to say how you are going to be working within your own consortium and you have got to provide enough detail that the reviewers really know what your approach is and how you are going to approach it.
The specific details, the very highly technical details, should be in the appendix but you have got to be very careful with appendix material. You cannot depend on the reviewers reading it. So anything that you think is critical for the review has to be in those 25 pages and if you have to leave something out that you think is less important or not go into as much detail, make a statement in the application saying page limitations did not allow me to go into as much detail as I wanted for this particular point and a further explanation of it is given in the appendix material.
DR. __________: But just looking at the responsibilities portions of the grant, take three institutions and just take the reporting requirements that you are going to have to demonstrate that each of the partners are there, and that -- there is a significant amount of verbiage just in describing, you know, if you have three partners or five partners, how that all goes in. Those are important things you have highlighted have to be in the format of the grant and, you know, I just see a very -- it being very difficult, you know, in terms of page limits to kind of get just the requirements of what is in the RFA into the application.
DR. __________: You just have to do the best you can.
DR. __________: We cannot lift the PHS --
DR. __________: Okay. So it is 25?
DR. [BERMAN, ed.]__________: Yes. Again, thoughtful use of appendices. Like for the surgical, at one point you have to talk about the surgical pathology reports and their completeness. So probably you might want to include in an appendix or think about including the data dictionary and then maybe a report. And when you write in your 25 pages, somewhere in that paragraph you just want to give some serious statistical, you know, breakdown in a very terse a way as you can that gets the information across.
You know, if we could just say you have 100 page limit instead of a 25, maybe we would not but -- but you have to do your best.
Anyway, I want to break just for a while because Tom Puglisi is here from OPRR and there are a number of , as you all know, confidentiality issues and human subject issues related to this thing. And he is going to speak a few minutes and then be available for a while for questions afterwards.
DR. PUGLISI: What I would like to do is just spend a little bit of time talking about the issue related to human subject protections.
I have got a real nice Power Point presentation but I turned on our projector this morning and immediately blew the bulb out and I could not find a replacement so we are going to go low tech with handouts.
DR. __________: How do you spell your last name?
DR. PUGLISI: I spell my last name P-u-g-l-i-s-i.
And somehow my e-mail address got left off. My e-mail address is TP10Y@nih.gov. TP10Y@nih.gov.
OPRR, as you know, I think, is the office that has responsibility for implementing the Health and Human Services regulations for the protection of human subjects.
We are scheduled to undergo a name change some time within the next several weeks so you may start to hear about an office called the Office for Human Research Protection or OHRP. That is just good old OPRR with a new name. Do not let that throw you.
For the time being we will be at our same web site, which you see listed there in the second slide. I encourage you to check out our web site. We have lots of information there and on the web site you can find the main phone number and e-mail address of every person who works in OPRR.
I mention that because we really do spend a lot of time dealing with investigators and IRBs one on one just discussing issues that you may need to discuss. So do not hesitate to call us. We get about 200 phone calls a day in our office and we try to get them all returned as quickly as we can.
So if you have specific questions feel free to call or e-mail me or anybody else in OPRR. We would be happy to talk with you.
I want to just remind you --
(End Tape 1, Side B.)
DR. PUGLISI: -- about whom an investigator obtains data through interaction or intervention with the individual, or a living individual about whom an investigator obtains identifiable private information.
And it is this second definition, the obtaining of identifiable private information about a person that throws many investigators.
The thing to do is to remember that if you are accessing identifiable private information about a person, you are engaged in human subjects research, whether or not you ever interact with that person physically.
Identifiable under the regulations means something very broad. It means the subject's identity can be readily ascertainable by the investigator or can be readily associated with the information. That means if there is any code linking a specimen or a piece of data to an individual, that specimen or piece of data is considered identifiable and you are engaged in human subjects research. So if there is any code, any linker, any ID record number, anything by which anybody can get back to the identity of the person from whom the material or information came, then you have human subjects research and you have to worry about the requirements of the regulations.
Private information basically means any sort of information that a person could reasonably expect would not be made public. Specifically, information from a medical record. So if you are dealing with private information, information from a medical record or any other kind of information that a person could reasonably expect would not be made public, and that information is identifiable, you have got human subjects research. That means that the activity has to undergo review by an institutional review board.
And the context in which all of you are operating, the review by the institutional review board is primarily going to look at how the specimens were originally obtained, whether or not the specimens or information are going to be used in a way that projects the privacy of subjects and whether or not there are adequate mechanisms in place to maintain the confidentiality of the data.
In terms of privacy, the IRB is going to want to know under what circumstances were the tissue or data obtained, was there any sort of informed consent, was there informed consent for clinical purposes, or for research purposes? Is the material going to be accessed only by persons who would normally have access to it? In other words, are other folks who would not normally have access to the person's medical record or the person's sample going to be given access to that information or sample in an identifiable way? The IRB is going to want to make sure that the privacy of the individual subject has not been violated.
Then secondly the IRB is going to look at the confidentiality protections that are built into the research. Is the research -- is the information coded in some way so that one has to access a key in order to obtain the identity of the individual? Will it be encrypted somehow? A more sophisticated encoding mechanism. Who will have access to that encoded information?
If the information is going to be anonymized -- in other words, if all identifiers are going to be stripped from it, who is going to do the stripping of the codes? How is that going to be handled?
Is the information stored in a secure way? Either in computers that are protected against invasion or physically in some limited access storage facility?
Is it possible that the information is sensitive in some way that might make it attractive for a law enforcement agency to subpoena it? If so, you might want to think about getting a certificate of confidentiality. If you are going to be collecting information that relates to criminal behavior, for instance, or some other highly sensitive information, you might want to make sure that you are protecting against disclosing that information to a law enforcement agency or in a court. So you might think about getting a certificate of confidentiality.
The IRB is going to look at all of those things and then the IRB is going to have to make a determination about whether or not informed consent, additional informed consent is necessary for you to proceed with the research the way you want to undertake it.
IRBs basically will have three criteria that they will look at when they are judging the type of activity that you are engaged in, in order to justify waiving the usual requirements for informed consent.
First of all, the IRB is going to have to be convinced that the level of risk to individual subjects is no greater than minimal. That means having very, very good confidentiality protections. Either have the information and data completely anonymized or have a coding system with protections that makes it very, very unlikely that there will be a breach of confidentiality.
So work very hard on making confidentiality protections as air tight as you can make them.
Secondly, the IRB is going to want to make sure that subjects' rights and welfare are not adversely affected in some way. Usually this means the IRB wants to be convinced that you are not unduly invading a person's privacy.
And, thirdly, you have to make the case that it would not be possible, would not be practicable, to do this research activity without waiving the informed consent requirement.
Typically you have a situation where you may have thousands of samples, thousands of subjects or patients from whom data are obtained, and it is not practicable to go back and find each one of those persons and get informed consent or for some other reason it is simply not practicable to do that.
In presenting your activity to your institutional review board you are going to want to address these three criteria very specifically and make it very easy for the IRB to determine that you do not need to get any additional informed consent from your subjects.
Okay. Depending upon how you set up your activity there are a variety of different scenarios under which IRB review might be conducted. In kind of a traditional repository center you usually have a setting -- you usually have a number of sites where information and tissues are collected, and those information and tissues are moved to a central location. The repository and the repository sets up some criteria for sharing the specimens and information with investigators. Typically the repository has protection in place to make sure that those investigators cannot get the identities of the persons from whom the information or material is derived.
As I understand the RFA, what you are doing is a little bit more complicated in that there will be certain elements to this repository that are virtual, if you will, and that there will be a lot of requesting and sharing information and specimens through a computerized system rather physically storing everything all in one place.
At any rate, you are going to have to face IRB review at the site where the materials and the information are accessed. Depending upon how you set up your activity, you will probably have to have IRB review at the site of whoever is responsible for coordinating the whole thing, for making the whole thing work, for deciding how and who gets to access the tissues that are available and information that is available.
If you set up your system in such a way that the recipient investigators cannot get the identities of the individuals from whom the information and the tissue is derived then you do not likely need IRB review at the site of the investigator who is actually using the tissues. At least you certainly would not need that IRB from our perspective.
Some investigators receiving tissue even if it is anonymized will be bound by local institutional policies that require IRB review but it is not a regulatory requirement.
Complicating the whole process is the fact that if this activity is supported by the Department of Health and Human Services, the activity involving obtaining the tissue and specimens and disbursing the tissue and specimens is going to have to be conducted under what is called an assurance of compliance. Every institution that receives HHS money for a research activity has to supply OPRR with a document called an "assurance." It simply lays out how the institution is going to enforce the regulations.
If you are from a large research institution chances are your institution already has an assurance. Probably a multiple project assurance that covers all of your federally supported research.
Some of your collaborators may not have assurances. If you are dealing with a community hospital or some other facility that does not do a lot of research, they may not have an assurance. And so before the award is made they will have to file an assurance with OPRR.
At the point of application you do not have to worry about that so much. If you are from an institution that already has a multiple project assurance, you are all set. If your performance sites or collaborating sites do not have assurances, do not worry about it. Those can be obtained before the award is funded.
Okay. The last page of the handout is just a schematic of what we have already talked about. You can think of a data or tissue resource as having three components, a tissue collector, a repository or data management center that either stores the information or controls access to it, and a recipient investigator.
Some IRB review is going to be needed at the tissue collection site, some IRB review is probably going to be needed at the site that controls access to the information, and no IRB review is probably going to be needed at the recipient site as long as the recipient does not have any way to obtain the identities of the individuals from whom the tissue or information derived.
I am giving you two additional web sites on our web for specific information about repository research and genetic research if that is of interest to you.
And I have already gone over time so let me just stop there. If you have any questions, I will be happy to answer them as well as I can.
Does anybody have anything?
DR. ___________: I asked the question earlier about IRB approval and it seemed to be a two step process. Are you familiar enough with the RFA to -- the words "approval of the schema" was used earlier. I was just curious how you envisioned that IRB approval. You know, in some ways it is not a typical hypothesis, you know, driven --
DR. PUGLISI: Yes. Okay. The question was because of the nature of the RFA you are not going to have every detail of your application -- of your activity worked out beforehand. What is it appropriate for the IRB to review?
The first question to the NCI staff, when are these due -- are they going to be subject to the new just in time rules or to the old rules?
DR. [BERMAN, ed.]___________: It is due July 24th.
DR. PUGLISI: Okay. So just in time starts July 1st, right?
DR. ___________: Yes.
DR. PUGLISI: Okay. So many of you may have heard that NIH is going to a new system relative to the timing of IRB review. Previously if you came from an institution that has a multiple project assurance, you had to have certification of IRB review at the time of application or prior to peer review.
Effective July 1st that will no longer be the case. Certification of IRB review must come before funding is awarded but it does not have to come at the time of application.
Now your local institution may or may not decide to take advantage of that change in policy so you need to check with your institution to see whether or not they are still going to require you to have IRB review at the time of submittal of the application or not.
Secondly, even if you have put off IRB review until the point of -- until just prior to the award, it is still the case that your application is going to be incomplete as to the details. What your IRB will be asked to do is to review the application and the activity as completely as you can describe it. And if that -- if you do that, one of two things will happen. The IRB will either say, "Okay, fine, we will give you an approval with the understanding that you cannot go forward until we have received a more complete application," or the IRB will say, "We do not have enough information to review this. We are not going to issue any approval."
If that happens, that is not a problem for you because there is a mechanism within the NIH grants management system to allow for an award to take place with a restriction to the award that essentially says you cannot go forward until IRB review and approval has occurred. It is an exception that is called an exception for delayed onset of activities where the IRB simply does not feel that it has -- that the application is flushed out well enough in order to review the details of the proposal.
So if the IRB goes ahead and reviews it, gives you an approval, that is fine. You just submit that to NIH.
If the IRB says they do not have enough information, contact your program person, tell the program person that the IRB says it does not have enough information, ask the program person to have the grants management person request permission for a restricted award because of delayed onset.
And I would say that you can do that by contacting OPRR but by the time this is awarded that will be a function of NIH.
It will be the Office of Extramural Programs that will be doing that by the time this is awarded.
So just the magic words are "restricted award because of delayed onset." And the grants management people will know what that means and you should not have any trouble.
Anything else? Okay. This looks like a really exciting project and I wish you all luck with it. If you have questions between now and the time that you actually start doing what you are doing, feel free to give me or anybody else in OPRR a call. I will be happy to talk things through. Okay. Thanks a lot.
DR. [BERMAN, ed.]___________: Thanks for coming on such short notice.
DR. PUGLISI: Sure.
DR. [BERMAN, ed.]___________: Thanks for that offer to let people call you. I know OPRR is very busy these days.
DR. PUGLISI: If you leave me a voice mail message or send me an e-mail, I will get back to you within a day. Take care. Good luck to all of you.
DR. [BERMAN, ed.]___________: Okay. So back to questions. General questions about --
DR. ___________: Jules, we have heard that the primary thrust of this RFA is towards the NCI's mission of making tissue available to investigators but at the same time we have heard about the desirability of large numbers of cases. We have heard the term "data mining" mentioned a couple of times. What is the vision of the drafters of this RFA vis-a-vis the use of this facility or this resource for epidemiologic purposes?
DR. [BERMAN, ed.]___________: Well, what we expect is that it will be used for things that we have not envisioned. I mean, that is one of the things that we wanted, was to have something so that there is sort of so many -- there is so much data, there are so many ways of moving -- linking with the data that people will come up with things that we have not thought of.
The -- we do not want to say that this thing has an epidemiologic intent, you know, for use. The epidemiologist -- my own feeling, and I am not speaking for NCI now at all, is that the -- some of the epidemiologist have problems with datasets that are not population based, that are hospital based instead. This will be a hospital based one. So it might be considered sort of inflammatory to people in that field to say that this is something to be used for epidemiology although there are certain questions that I think most people would consider to be epidemiological that might be addressed in a system like this once it is up and running.
There are other things which the system might be used for in addition to just procuring paraffin tissue, is that if you can do a query and find out what is out there and how much comes into institutions in a given period of time, you can identify the likelihood of accrual of tissues from an institution for prospective studies.
So presumably you can do a query, find out that, yes, you know, these people get a larynx, you know, every week and we can probably accrue enough for a prospective study in a year. Maybe I ought to contact this network institution and see what I can do with arranging something, if they are interested, for a prospective study instead of just doing retrospective studies.
There are just an enormous number of things that can be done but we have sort of engineered it so that it will meet the requirements that we have been discussing, you know, sort of over and over again today, which is to identify existing blocks of tissue and the data related to those tissues.
DR. ___________: Jules, I am not sure that anybody will be able to know where the tissues reside. I am not sure that this is going to be set up. I mean, that gets down to a confidentiality issue.
DR. ___________: Right.
DR. ___________: So I am not sure of the idea that somebody -- that some institution collects so many larynxes a year is going to be know-able in this system.
DR. ___________: Yes. I mean it is --
DR. [BERMAN, ed.]___________: But also in answer, I think Jules was correct in the -- raising the issue of what your definition of epidemiology or epidemiologic studies is because I think in addition to the caveats that Jules raised about population based versus hospital based and so on. The other issue is that we do not anticipate having the kind of family information and exposure information and all of that kind of data that would normally be incorporated into the more traditional epidemiologic studies. So I do not think that this system will serve that function.
On the other hand, the issue of data mining, which you mentioned -- I think what Jules has raised my sensitivity to -- over the past year or so -- is that the idea that you could do a lot of research on prevalence of cases of -- particular types of cases, ages that -- you know, of onset and things like that. That kind of -- it is not truly a population based study but depending on how broad this network is. Ultimately those kinds of studies could potentially be done and this certainly would enrich the potential for those studies.
DR. ___________: I would take exception with you. As a matter of fact, everything you have said is epidemiology. Epidemiology means that who pertains to group, whether it is population study or anything.
DR. ___________: Well, that is -- right. I mean, that is --
DR. ___________: In fact, this is a very valuable epidemiologic --
DR. [TAUBE, ed.]___________: Right. No, I said it depends on your definition of epidemiology and in that sense, yes. But if it is exposure or, you know, etiologically based research, questions, hypotheses that you want to ask, I am not sure that this database will have -- this data system will have the necessary information. But anything that depends on group analysis of the data that exists will be possible.
DR. ___________: Let me just --
DR. ___________: Okay.
DR. ___________: Yes, as a matter of fact, what this could do is you could generate hypothesis, which is very, very important.
DR. [BERMAN, ed.]___________: You know, it would be lovely if that is -- if it came to be that way.
I just want to say one little thing about what Sheila had mentioned about the identification of the institutions, she is right that presumably the data that goes back, that is collected and assembled, and goes back to the person who put in the query will have information that does not include the name of the institution it came from.
However -- I mean presumably not in this RFA but in something else that might come out when this shows the feasibility of this model, there might be a system whereby once you have identified the -- what is available through the network that there might be a -- the researchers who have done the -- that first pass would then be permitted to send in an application, essentially a research application to the network saying I would like to do this perspective study and I understand that the network has, you know, this many total specimens of a type coming in and could I do that.
And then there would be some sort of evaluation panel to determine whether or not this is the kind of thing that the network would be interested in, in which case some -- the next step might involve essentially collaborative work.
Who is -- we will get you and then you, Bob.
DR. ___________: A question related to what was just discussed. In theory, the design of applicant architecture is to be very different based on whether it is -- the sole purpose is focused on providing a tissue sample hypothetically or for real versus providing a epidemiological resource. It would also get into the issue of who was allowed to access the front door so to speak at the government level. You can have the equivalent of denial of service attacks theoretically on the machines if anybody can ask epidemiological questions that are poorly formed and load down whatever machines at participating institutions.
Similarly, you could have -- if in the next phase anybody that wanted a tissue sample could have it -- as you said, somewhere you have to draw the line but how would you do that and would the government limit who could query the system, the people with a legitimate reason or some sort of application that they filled out or whatever?
DR. [BERMAN, ed.]___________: Well, I mean, essentially as far as the distribution of tissues, I mean that is -- you -- that is not for this RFA. I mean, something else would have -- you know, since you will not be distributing tissues. But presumably there would be an evaluation panel to determine whether or not the request for the tissues is valid and not only valid but whether or not it is of sufficient merit -- if it is a request for many, many tissues to merit all the work involved. But that is something that is sort of done.
There are -- you know, NCI has experience in having repositories that deliver tissue that have to make the determination of what tissues go out to whom.
The issue of the denial of service problem if -- not as an attack but simply because there might be so many people with queries of a type that are so computationally intensive that it would use up all the server time so that other people could not get on. I mean, again that is a kind of problem that you love to have because it means that the system is going to be used and that is the presumption of the question.
But, yes, I mean there would have to be -- the coordinating committee would have to meet and try to determine -- and, again, this -- it would not be a problem with this RFA because this RFA only has to go to the stage where you have validated that the process could work. But if a network were accepting queries from the universe then presumably it might -- the coordinating committee would meet and make a decision as to whether or not they think they are going to be overloaded and who should be allowed to put in queries and should it be only people who have NCI grants or should it be people, you know, whatever. And then how those people will get passwords or something to authenticate them to the system.
I mean, those -- all those are issues that somebody would have to address at some point in the future but presumably they are all solve-able.
DR. ___________: Could you comment on why this RFA was not framed in terms of -- a little more narrowly in terms of some of the other organ specific programs that you have underway in which there are tissue banking and tissue resource activities? Why was this framed broadly as opposed to narrowly, I guess, is my question?
DR.[BERMAN, ed.] ___________: Broadly in the sense as not picking a tissue of interest?
DR. ___________: Or not coordinating with the -- seem [theme?, ed.]of your existing programs?
DR.[BERMAN, ed.] ___________: This is a whole new idea because what we are attempting to do is to determine the feasibility of trying to access information from existing databases. So that is the aim of this RFA, of this initiative. Whereas, the other programs were specifically addressing the need for tissues of a particular type and what we are trying to do is get away from the need to do it that way because it is extremely expensive to collect the data, centralize the data, go back and re-access the data. You know, I mean, most of the data for the cooperative human -- I mean, for the cooperative breast cancer tissue resource came from chart review and there are issues regarding consent to go into the charts and all of that. Some of which apply to going in through electronically as well except that you can encrypt.
So this is really a different effort. It is a different direction. It is an attempt to establish a system that will obviate the need to set up specific organ site repositories in the future. Does that answer your question?
DR. ___________: Yes.
DR. ___________: I just wanted to follow-up on your comment, Sheila. It is Sheila, right?
DR. ___________: Yes.
DR. ___________: About why is the site not knowable. It almost seems to me that the way I am imagining this is that it has to be knowable. So I was a little surprised at your comment and I would like to know more about that.
DR. [TAUBE, ed.]___________: It will be knowable within the network. It will not be knowable to the querier and the reason for that has to do with the confidentiality issues because that allows a track back for the investigator to potentially go in through the system and get to the electronic information that resides at the home site.
So, I mean, that is something that I think remains for discussion. I think that eventually if we are giving out tissues it may be impossible to blind the recipient to where the tissues actually come from but for the initial querying to find out what types of cases might be available from that perspective, we think that is much more protective of the patient information to not reveal the sites.
DR. ___________: There are a whole host of issues that relate to identifying tissues available for research in the broadest sense that relate to, you know, how much tissue, how many blocks, volume of material, that essentially existing systems, the existing lab information systems, medical registries, existing built and developed systems that I know of and as many flavors that are out there for sites to use, that essentially just ignore this. There just is no system that records this data effectively.
There are efforts obviously to go and try to solve that by creating ancillary systems that interact with other systems. It is a huge problem because really to know that the resource is there, there has to be some kind of inventory mechanism.
The lab information system industry has just ignored this. Yet you are assuming it is there in some perspective for this proposal.
DR. [BERMAN, ed.]___________: Well, what I am assuming is that when the network starts its work that it will find out exactly what you said, that there is all sorts of weaknesses in the general sort of pathology infrastructure and data related to the specimens, and that one of the tasks and also one of the strengths of this thing is to identify those things and start adding them.
So let's say this thing is a five-year -- it is one of the reasons why it is a five year program because we expect that the first year a large part of it will be determining what things you would have to start adding in maybe year -- the beginning of year two in order to have a robust system that really does all the kinds of things that you would like done.
So the coordinating committee presumably would say, well, look, you know, none of these institutions has been collecting -- I do not know -- data on the consent status of the case or data on the actual number of blocks that are available or anything that you would like to have that you can conceive having in a query.
And then you probably start -- the consortium institutes of the network would probably start adding that thing and at the end of the five years they would be sitting pretty because they would have a system that is worked out to have those things that no one else had the forethought of including.
But, you know, what we want for the start is to -- at least in that respect in a more advantageous position. And then we would work from there to make it broader and to make it more robust.
DR. ___________: Can I follow up on that just briefly? I mean, there is a big difference in my mind between, you know, inventorying as was described, and I think it was a very good question but that is not necessarily what you are looking for in the budget.
I mean, for example, if that were anticipated that you would know how many blocks, for example, were available on cases, somebody is going to have to go, you know, do that. So I am going to have to budget for that but I want to be clear as to whether or not you are really looking for that.
DR.[BERMAN, ed.] ___________: We are not -- I think Roger wants to say something. My concept is essentially when you -- the coordinating committee will meet. It will determine what it has got and what it sort of would like to have. And the like to have things are not going to be things that it is going to go back and get. I mean, those things -- because like you said, they are not budgeted. There is no, you know, gazillion dollars for going back and doing a lot of retrospective add ons. But these will be things that from that moment on a pathology department would probably want to start including when it collects the data on a case and when it is -- so its report -- maybe the report presently has, you know, maybe nothing at all on the -- the reports do have numbers of blocks or they should have numbers of blocks. So maybe --
DR. ___________: In the narrative.
DR. ___________: -- in the narrative, maybe they will make a field that has number of blocks and maybe additional information about how, you know -- I do not know --
DR. ___________: Normal tissue or --
DR. ___________: Yes, normal tissue. Whether or not there is a whole bunch of necrosis, who knows what. But, I mean, those things would be things that the coordinating committee would say, well, let's start adding a field for this in all of our laboratory systems. Let's start, you know, collecting this so that we can have it available.
DR. ___________: But it is not assumed. I mean, you are not looking for that in the budget per se. In the budgets for what is in the RFA. I mean, I do not see an inventory in there.
DR.[AAMODT, ed.] ___________: Yes, you are absolutely correct and what I was going to say is that the only thing the RFA really specifies is that you know that a tissue exists that relates to that pathology report.
The other thing, though, related to the budget, is in our experience with this kind of group, we find that most of the time we do not anticipate all the budgetary items that are going to be done or needed for the actual project. So you put in a budget that reflects your best estimates of how you think you would do it and then as we go along it is likely that there will be some rebudgeting. There could under extreme circumstances be a need for the NCI to add more money to the budgets to cover some unexpected thing that came up in the course of development.
But you really need to stick to what is stated in the RFA in terms of thinking about what you want to propose.
DR. ___________: Okay. I will get you and then you are after him.
DR. ___________: Thanks. So again kind of thinking of who would be good in this consortium, there is -- if you have people that have well developed information systems, they are more likely to start with systems that have a significant population of data elements and inventories.
On the other hand, we might end up developing a system that is only going to work for those kinds of institutions and then the community hospitals and so forth would be kind of left out because they would not -- none of them would have this kind of thing. So -- and for them to get up to speed would be impossible so we kind of leave them out.
And I was wondering -- I mean, so there is kind of a give and take. I mean, clearly, you know, we would not be starting at as good a place if we had some community hospitals in the consortia because we have to spend a lot of effort, it seems, to get them up to speed.
On the other hand, from kind of a global thing, we learn what it would take to get those kind of people up to speed so it is a long winded way of asking is it good to have the community hospitals in it now or should we say, well, for now, let's just go with the well developed institutions and -- I am just not clear.
DR. ___________: Do you want to answer this?
DR. [AAMODT, ed.]___________: Well, I think that part of the review process at NIH is the investigators making those difficult choices and judgment calls but -- so that I am throwing it back to you. You have to figure out what makes a strong application that is responsive to the RFA.
On the other hand, it was our thought as we developed this that initially we would probably be looking at institutions that are better developed but we also think that probably as this model system gets developed vendors will come in wanting to develop easy to use software that makes it easy -- that makes it possible for pathology departments every place to enter data in a simple way.
I know that the College of American Pathologists is working on a set of standardized pathology forms for -- with drop down menus and so on that make it easy and they are talking to vendors and so on.
So, I mean, I -- I think what Jules' idea was, which I agree with, is that this will push the field along so that eventually data will be more accessible and richer but in terms of your own application at this time you have to look to being as responsive as you can be and put together the strongest program that addresses the issues of being able to query broadly into existing databases. Does that answer it?
DR. ___________: Thank you. Yes, that is very good. Thank you.
DR. ___________: You said before that the ultimate goal of the use of this tissue is to predict, you know, response to therapy or look for -- well, among the goals is certainly treatment related matters, you know, treating cancer more effectively, ultimately --
DR. ___________: It is to facilitate cancer research on all levels.
DR. ___________: Well, ultimately it is to cure cancer as best as one can.
DR. ___________: Right. But biology -- right.
DR. ___________: And the problem here is that, you know, as I see it, is you are proposing a five year program that two institutions will participate in, let's say, two -- you know --
DR. ___________: Consortia.
DR. ___________: Consortia.
DR. ___________: -- that represent --
DR. ___________: Two consortia.
DR. ___________: Two consortia. Okay. And the other consortia wannabe's will not participate in and know nothing about for five years, presumably, right? I mean, they are not going to --
DR. ___________: Possibly.
DR. ___________: -- participating in these --
DR. ___________: Directly, right.
DR. ___________: -- progress reports and so on. And it seems to me that funds would be well spent to include updates for those of us that are interested in networking and sharing resources to know what is going on with those two consortia.
DR. ___________: Well, hopefully -- I mean, we are in the --
DR. ___________: And that should be part of the allocated funds, it seems to me.
DR. ___________: Well, I mean, first of all, the NCI can do that at any time it so desires. But in addition we hope that --
DR. ___________: But it can be integrated into this --
DR. ___________: -- that this is a research project and we hope that people will publish the results of the research and the development of systems and so on. And we certainly will encourage that.
DR. ___________: Why can't you have the two coordinated?
DR. ___________: They may be.
DR. ___________: Rather than putting it into a paper.
DR. ___________: I mean, it is the kind of thing that -- I mean, my vision -- I do not know exactly what is going to happen but let's say there is significant progress in two years.
DR. ___________: Right.
DR. [AAMODT, ed.]___________: Then I as a program director would probably say, gee, why can't we have a workshop where all the people who were in this thing get to say what they have been doing and then we can invite some other people who are not in the consortium and they can all get together and they can talk about the tools that are available and the problems they have been having and that sort of thing.
But that would not be described in the RFA but it would be the kind of things that -- but that is what NCI does all the time.
DR. ___________: That is what I was going to say.
DR. ___________: I just wanted to add one thing to what I was saying first. The other -- I do not know if it is a contradiction but there is sort of a backward looking aspect to this and a forward looking aspect.
The backward looking aspect is the archival nature of what you want to access. And most of the archival data that goes with the various PATH specimens is kind of weak in the category of what happened subsequently to the patient. There are probably exceptions at some institutions.
And it may be worth somehow including the prospective over the five years, you know, incorporating changing the accessioning (sic) process and so on to include that. I am not sure if I missed something in the discussion. Is that part of the -- it is not just evaluating what you have had over the past X years but what you want to enter prospectively and change the entry system. Is that not part of it?
DR. ___________: No, that is definitely part of it.
DR. ___________: Okay.
DR. [AAMODT, ed.]___________: It is also part of a much bigger thing. Jules was brought into the NCI precisely so we could develop a strong informatics program related to pathology and so there will be a number of activities going on simultaneous --
(End Tape 2, Side A.)
DR. [AAMODT, ed.]___________: -- which will be informing the field. I think Jules is absolutely correct. Nothing precludes us from having workshops and it makes sense as we move along and we start running into difficulties or finding major breakthroughs that we get the community together and say, hey, look what we have done or, gosh, we have got a problem, do you guys want to talk about it?
DR. [TAUBE, ed.]___________: And Jules held a workshop last May, a year ago, that addressed some of these issue and in the breakout groups -- I mean, it became clear that the follow-up issues are a tremendous problem, and my view because I know nothing about it, so it is easy to think big, is that eventually we would get professional organizations involved in some way so that data from physicians' offices or from third party payers or whatever, that other sources of information that provide follow-up information would somehow become more accessible.
Not in this first five years because first we have to find out if it is even feasible to think the way we are thinking, which when we first started thinking about it, seemed awfully pie in the sky as we talked about it.
So this other part is yet a bigger umbrella that we would like to bring people under but certainly it is in our thoughts.
DR. ___________: But five years from now you do not want to just have developed a system that --
DR. ___________: Sure.
DR. ___________: -- recapitulates what is on a standard pathology data entry set of data. You want to incorporate prospective treatment response so that that pathology specimen will be able to provide the researchers with valuable information about what is in that genome and so on that made that patient respond --
DR. ___________: Right, of course.
DR. ___________: -- whereas the other patient did not.
DR. ___________: Of course.
DR. ___________: So that is a little different from what this five years is --
DR. ___________: Well, but you cannot --
DR. ___________: -- going to be --
DR. ___________: -- but you first have to think about connectivity and that is what we are attempting originally as a first step. To just determine the feasibility. All these other issues relate to the research that you want to do and clearly we have to think about how to do that and periodic workshops, including the people who are involved in the -- who do get funded and those who do not get funded but who are interested in this field, will continue to develop these other ideas. You cannot do everything all at once.
DR. ___________: I am just saying you do not want to use five years to learn how to do what you did before that was not relevant to treat.
DR. [BERMAN, ed.]___________: Let me just say we are going to break in ten minutes. After the break, you can ask questions that are more specific to your particular situation.
Barbara Fisher is going to leave at 3:00 o'clock so we will not have anyone from Grants Administration after that so if you have any questions that deal with those sort of issues you probably ought to get them out now but otherwise we will just have questions until 3:00 and then take a break.
DR. ___________: I have a question. I just wanted to clarify for sure that the amounts given for total indirect and direct are combined? I mean, it says total --
DR. [FISHER, ed.]___________: That is correct.
DR. ___________: Yes. Okay.
DR. [FISHER, ed.]___________: The total cost.
DR. ___________: Okay.
DR. [FISHER, ed.]___________: Third party indirects, direct cost to the grantee institution as well as the direct cost.
DR. [BERMAN, ed.]___________: Does someone back there have a question before the -- okay. Then we will -- no, they did not so you can ask your question.
DR. ___________: Will the awards be cost recovery or fixed price?
DR. [TAUBE, ed.]___________: This is not a contract. It is a grant.
DR. ___________: Can I ask a question? In terms of when we send the budget pages in there would be, let's say, three or four budget pages for each institution in the consortia.
DR. [FISHER, ed.]___________: That is correct.
DR. ___________: And each one of those would have their indirect costs.
DR. [FISHER, ed.]___________: That is correct and that is part of the total cost to the grant.
DR. [BERMAN, ed.]___________: I just want to say because Sheila answered without a microphone, let me describe, the question was it going to be cost recovery or fixed price and the answer is this is a grant mechanism so it does not apply.
DR. [FISHER, ed.]___________: That is right.
DR. ___________: Subcontracts -- I mean, I would assume then that the subconsortium are just subcontracts to the institution and somebody still needs to be a prime and run the whole thing through its institution?
DR. [FISHER, ed.]___________: That is correct.
DR. ___________: And a sub is allowed so if you have a site --
DR. [FISHER, ed.]___________: That is correct.
DR. ___________: -- and they want to sub to, let's say, X,Y,Z corporation or somebody at another institution but are part of let's say an umbrella member of a consortium then that would be just a sub to the sub?
DR. [FISHER, ed.]___________: That is correct.
DR. ___________: All requiring their own budget sheets?
DR. [FISHER, ed.]___________: Correct.
DR. ___________: Well, I just wanted to -- so the way it works is that the consortium members are subcontractors, if you will.
DR. [FISHER, ed.]___________: That is correct.
DR. ___________: Okay.
DR. ___________: If they are to receive money.
DR. ___________: Yes. Okay. All right.
DR. ___________: I have another general nonbudget question if you are done with the budget questions.
DR. ___________: Okay.
DR. ___________: Jules, there are many dimensions to this but on your slide you listed as the last bullet validating and testing your "system."
On page five of the copy of the RFA that I have is a comment about the parcers (sic) that preserve the conceptual content of the free text diagnosis with high accuracy. How important is it in the minds of those who drafted the RFA to evaluate the performance characteristics of the encoding and retrieval systems that are proposed and developed?
DR. [BERMAN, ed.]___________: The -- it is important. I mean, the testing and validation -- I guess when I was writing that I was mostly just thinking more in terms of if a query comes will the network be able to respond and produce a good query reply in a standard format. I was not thinking particularly in terms of the testing phase being the phase in which you test the translation of the free text to a nomenclature.
I was thinking of that mostly in terms of the -- I think it was the first phase or the first -- I think in the first -- a combination of the first and the second phase. When you come up with the -- or, you know, when the coordinating committee comes up with the way it finally is going to choose as the way you are going to move -- get your free text translated into something which is more query-able, SNOMED or whatever, that decision is going to be made on testing that the coordinating committee does in that early phase.
In other words, you are not going to come up with a decision that is going to constrain you to an activity that you are going to work on for three years and not find out until five years whether you made a big mistake.
So the testing for that translation is going to be essentially during the decision. I mean, you are going to have to say, well, yes, we decided to do it this way but, you know, we just tried it and it does not work for us over here in X hospital and maybe you ought to come up with something else. So that is an early phase kind of thing.
DR. ___________: Just sort of a follow-up to the same concept and that is will hospitals and coordinating centers have to interchange pathology reports to see if these tools work more generally? I mean, you have a great parcer (sic) for you.
DR. ___________: Right.
DR. ___________: But other people may encode things very differently.
DR. [BERMAN, ed.]___________: Again it is going to be an issue for the coordinating committee to decide whether or not it is going to have a universal parcer and translator or whether or not it is just not feasible to do it that way and you have to finetune things for institutions. Presumably there will be some sort of finetuning need because of the -- every institution has idiosyncratic ways of presenting data. I mean, so you are going to have to have something to deal with the idiosyncracies of reports.
There might be general algorithms which can apply to every place but, you know, there is going to be some differences. So I do not expect one institution to be able to -- to have to prove that it can take any report from any of its consortia members, run it through its software, and come out with the optimal translation, although it would be nice if it could.
DR. ___________: I think on page nine it describes the review criteria. As I saw that title I thought, oh, good, we will see the weights now. Can you describe why you have not identified a priori the weights for these sections or particularly what it means that they will be weighted as appropriate for each application?
DR. ___________: Is that what it says? Weighted as appropriate?
DR. ___________: Yes.
DR. ___________: There is a soda machine around the corner to the right.
(Whereupon, a break was taken.)
DR. ___________: Okay. I think we already extended over the allotted time for the break. So we can get back to asking questions and now you can ask any kind of question that you want. There is no --
DR. ___________: What is up, Jules?
DR. ___________: Nothing much.
DR. ___________: I have one. In here you talk about the NCI informatics resources available. Are those --
DR. ___________: Right.
DR. ___________: -- currently existing or to be developed during this whole process as hand in hand type?
DR. [BERMAN, ed.]___________: They are mostly under development and the reason that is in there and just for sake of other people, there is something in the -- I think it is in the coordinating committee description. There is words to the effect that there will be a member who is not voting from the Office of Informatics to sort of act as a consultant.
That is done mostly in the interest of NCI but I think it would also be in the general public interest and the interest of the people of the consortium to make sure that the sort of -- especially the standardization efforts, the common data element efforts that are in the confidentiality sort of mechanisms that we use for this thing will have applicability or at least not be conflicting with other projects that are going on that the Office of Informatics at NCI knows about.
So we do not want to shoot ourselves in the foot by producing some sort of list of common data elements that has no chance whatsoever of being re-used or merged with other databases that are being prepared at NCI either in the Genome Project or any sort of things that are going on.
We want to maximize our ability to work with other systems and at the same time the Office of Informatics is interested and is sort of dealing now with some of the sort of network security issues and is contracting out -- I do not want to go into the specifics but they are -- you know, they are interested in the general area of the authentications of queries and things like that.
And if they have contracted and produced through a contract a software tool that is available to anybody in NCI, we would probably want to know about that so that we could use it. So essentially it is just to have somebody there who has expertise about everything that is going on at NCI so we can work a little better.
So to answer your question, do they have tools or are they developing tools? Mostly they are developing things and we will not be bound to use any of those things. It is just that it will provide us the opportunity not to do something stupid and make a system that cannot work with any other system you do not want to do.
DR. ___________: Does each consortia have to have a bioethics committee?
DR. ___________: Each -- no. In fact, what -- you know, Roger, you might want to answer that. But each consortia has to fall under the review of an institutional review board.
DR. ___________: Okay.
DR. ___________: And that is -- okay. I knew you would want to say something.
DR. ___________: You are right on target.
DR. ___________: Okay.
DR. [AAMODT, ed.]___________: I think the expansion on what Jules has said is that one of the major activities of the coordinating committee is going to be to deal with some of the bioethics issues that emerge in the course of this project and there are going to be a lot of them. So to the extent that you want to develop some bioethics expertise in your consortium, it would make sense to think about, for instance, bringing somebody on as a consultant who had some of that background.
But basically all you need to do is be sure that you have got IRB approval for all the activities you are goign to do and the real stiff bioethics issues we may have to do entirely outside of the range of this group and we have other ways of doing that. We could convene a meeting or whatever and really bring in some heavy duty bioethicists if we need to.
DR. ___________: Just -- I am trying to find it in the -- there is also language which you should be aware of in the RFA. I cannot find it right off. I am sure someone will point it out to me. But there is -- in addition to the sort of milestones that have to be met, we also have language in there that indicates that if, you know, there is some law passed or there is some eventuality where the kinds of things that we want to do are just considered illegal, that essentially there is no way of something working like this where you go out and do this and that for a query, and provide tissues through it. So that the whole thing from that point on is just legally nonfeasible. The funding for the network can be withdrawn by NCI.
It is just something that you should keep in mind but we do -- you know, the way things are turning out and we do not anticipate that, and certainly for this particular RFA for what you will be doing, which is more or less a feasibility study and will not be distributing a whole lot, we will not come to the point in this RFA where an IRB should be saying that you cannot go ahead and do this.
The only -- it is a very unlikely possibility that there might be something passed because there are a lot of -- there is lots of legislation though that would say from this point on this -- this, that and the other is illegal and this network might fall under this, that or the other.
DR. ___________: I am not sure if it is page 8 of 12, item 5, that you are talking about but there may be another one but that says -- the last sentence says that the NCI understands that these arrangements could be effectively negated by circumstances beyond their control.
DR. ___________: That is -- that refers to the letter that comes from your institution that says it is willing to work with these objectives of the RFA, including getting tissues.
So that if your -- that protects your institution from writing something -- I do not know if fraudulent is the right word but writing something that events -- after sending in that letter -- make it impossible for them to --
DR. ___________: Participate.
DR. ___________: -- to participate in.
DR. ___________: So your institution --
DR. ___________: For example, like being bought by another hospital?
DR. ___________: Like being bought by another hospital.
DR. ___________: Now that is --
DR. ___________: Any and all circumstances under which you find your cannot provide the tissues.
DR. ___________: Okay.
DR. ___________: I thought that would be an advantage because it increases the complexity.
DR. ___________: That is right.
DR. ___________: At least you listen to what I say.
DR. ___________: I have a question about the uniformity of the two different consortia over time. It sounds at one level like in a way what the coordinating committee is goign to do is establish standards for how you submit queries and what comes back as a result of a query. But what happens in between may be totally different between the two consortia. Is that true or false?
DR. [BERMAN, ed.]___________: Yes, I mean we -- we would -- we expect the way the query is going to be handled -- you know, there will be completely different software handling the query at all the institutions.
But we expect that if you -- if an institution has, you know, 30 liver tumors in its electronic archive, and another institution has 30 liver tumors, we would hope to get back a total of 60 liver tumors. I mean, it is -- they have got to come up with some way of making sure that the responsiveness is similar to queries.
DR. [AAMODT, ed.]___________: Yes. You can divide it basically into local issues and network issues or system issues. And the system issues will be dealt with by the coordinating committee as one group so you would be joined at the hip once these two groups are funded for most of those kinds of issues.
The local issues you can deal with within your own consortium.
DR. ___________: Where do the network issues fall since you brought it up?
DR. [AAMODT, ed.]___________: Okay. The network issues have to be decided by the coordinating committee so there would be a -- presumably a unified response that all the groups would participate in, in dealing with the network issues, and everything basically beyond the firewall at the local institutions.
DR. ___________: Okay. Now what we could do -- I mean, it seems to me there is a little lull in the questions. Do you have one?
DR. ___________: I --
DR. [BERMAN, ed.]___________: Let me just say this. We have a couple of things that we could do. We could end early so that the official meeting is over and just have -- I would stay around to 5:00 o'clock regardless. And we can just sort of stand around and ask questions without the microphone or we can continue as a public forum.
Does anybody have any -- Sheila, do you have any opinions?
DR. [TAUBE, ed.]___________: I think the latter is fine. It is what we have done in the past, that at a certain point the large grand size questions have been addressed, and the others would not appear in the transcript anyway because they are really not relevant.
DR. ___________: Okay. So --
DR. ___________: So you may adjourn if that is the wish of the group.
DR. [BERMAN, ed.]___________: Okay. There is at least one more question. How about we will take your question, answer that and then we will adjourn, and then we will -- after adjourning, though, we can just continue but without the official public forum.
DR. ___________: The thing is this might be a better question for the second -- or after we adjourn.
DR. ___________: Because it is kind of a general question in that I was wondering why it was that there was kind of -- it has been -- it is decided that we should not have a central repository and that institutions should not just feed their stuff into a central repository but rather we should have a shared query mechanism? I mean, it just --
(End Tape 2, Side B.)
DR. __________: -- for, you know, the data fee since it is such a repository, which would take all the same work in terms of coming up with it but -- or we could say that -- and different things get trendy at different times and I am just -- I was just wondering about that but --
DR. [AAMODT, ed.]___________: You actually just picked the two strings out of our bow because one of the things we are considering is what we are calling a tissue resource of last resort. That is we are getting more and more information that hospitals are under pressure to discard particularly their archival blocks and it makes sense that there be a research base of archival blocks with some design and we have not thought yet about exactly what that design would be but it certainly would over represent rare tissues over common ones, and would have some significant representation of common ones.
And we think that this system that we are talking about for this group, in fact, may well serve as a way of getting the data for that resource as well.
The reason we do not focus or trying to move our focus away from setting up individual organ system based resources is that, frankly, they are inefficient.
They require a lot of effort to go back and keep the data up-to-date. You have got to -- normally you have a central database of some sort which needs to be updated regularly as you get more and more information coming in about outcomes and other issues. You have got quality control issues. You have got auditing issues. Some of which will come into this one as well but I think it is a different magnitude of the question when you start dealing with electronic data. It becomes how do you validate the data going into the electronic systems.
DR. ___________: Yes. I mean, it is just -- I mean, sometimes I have noticed that with -- when it is in a distributed system that you hide some of these questions and that -- you know, so the answer is, well, so each little place gets to manage their own but then that does not turn out -- they are just doing as bad a job as would be done, you know -- it just --
DR. ___________: Yes. And you -- you know, that is one of the big risks to this project, is that we could find out that, yes, we can do it. But, no, the data is not worth actually having because the quality is so bad and there is nothing we could do that could fix it.
But part of our hope here is that, in fact, this would induce people to fix the quality of the data and make the materials available.
DR. [BERMAN, ed.]___________: There is also -- we had a workshop in the summer where we discussed some of these issues and it was my perception from listening to most of the people that they felt uncomfortable with the idea of pathology departments, in particular, of taking all their data and moving it to a central database.
They wanted their data to reside in their institutions and that -- just that moving -- the whole thing is not -- just the response to a particular query but their whole data set over. They really did not like.
The other thing is we wanted to take advantage of emerging internet technology and we think that the distributed network query is a very interesting model. We do not want to think in terms of especially this network being the end all but being something that can work with other distributed, you know, networks as well. And we wanted to get away from just this is my database, that is your database, that sort of thing.
So, you know, we just sort of -- it was -- partly it was a perception from the workshop that this was what people would prefer.
Okay. Well, then unless there are any objections, I'm adjourning and anyone who wants to leave can, and otherwise the only difference between now and post adjournment is I am going to turn off the tape recorder.
(Whereupon, the proceedings were adjourned.)
* * * * *
Link to text of RFA
[The transcript is a public document]