Biostatistics + Ethics Meetings | VICTR Operations Workgroup Meeting Report

17Nov09

Attending: Lane, Harrell, Scott, Blume, Wang, Phillips, Heitman

• Jeffrey updated us on the interim monitoring project
  • Group met to discuss standard analyses/reporting
  • Frustrated about getting data from investigators; there may be issues for future discussion related to scientific quality control
  • Waiting to enter data into a database may cause quality/completeness problems
  • Ethics may be impacted if safety data are not being entered in a timely fashion (this would not pertain to serious adverse events which are immediately reported); trends should be monitored
• Motion to add to the VICTR StarBrite application a small section on research data management plan; passed unan.
• RedCAP clinic 2nd and 4th Tuesday 2-4 in Biostat small conference room; perhaps publicize more
• Potential BERD project to create and fill a RedCap database with characteristics of BERD members
• Potential BERD Online Resources ctspedia.org project with FDA CDER
• Liz reported that FDA may have funds for ethics education; focus on training of investigators to do good studies
  • Of interest would be an ethical conduct of research for statisticians in drug development
• Discussed recent studio application by an MSCI student that has ethics issues
• Terri discussed new certificate program being designed by Michael Stein and Gail Mayo that includes an ethics component
• Liz discussed planning for biostat/ethics seminar on Crohn's disease (probably for January); may want to involve the investigator
• Sharon mentioned a surgical study in which inclusion/exclusion criteria were not required by the IRB

15Sep09

Attending: Wang, Yu, Heitman, Churchill, Harrell, Scott, Blakemore, Byrne, Lane

• Liz spoke about her discussion on the national CTSA ethics KFC Ethics teleconference
  • Case Conference in Ethics & Biostatistics - educational, logic of the combination, getting biostatisticians to recognize some of the ethical issues
  • Questions about content of sessions
• General issues regarding ethics of scale construction, e.g., depression scales, use of surrogate endpoints
  • arbitrariness, applicability, number of items vs. quality of items, initial foundation
  • possibly do something for SCTS meeting April 2010?
• How to enhance the impact of translational research?
  • How do you tell something is a bad idea before it has failed?
  • Need to tie to public health needs

18Aug08

Attending: Jeffrey Blume, Ben Saville, Kirk Lane, Shraddha Nigavekar, Terri Scott, Tan Ding, Frank Harrell, Larry Churchill, Dana Blakemore, Sharon Phillips, Dan Byrne, Leena Choi, Janey Wang, Lei Xu

Special Session on Interim Monitoring

• New CTSA supplement mandating interim monitoring of certain VICTR-funded studies
• Funds some non-VICTR statisticians
• Focus on optimum use of funds, early stopping for futility, adequacy of sample size calculations, occasional early stopping for efficacy
• Reports will be reviewed by the SRC both individually and as portfolio of previously funded studies
• Some of the criteria for selection of studies for interim monitoring: single-center study, randomized or observational; larger use of resources, long accrual phase; early outcome variable determination
  • FInd studies on the basis of design characteristics then look at resource use
  • Go back 6m-12m to see if studies underway can be selected. But focus is on new applications.
• Three approaches: blinded sequential confidence intervals, likelihood, Bayesian; preface with accrual monitoring
• Stat pre-reviewers will note study characteristics in a new RedCap database. We need to make a coded list of study designs.
  • Pre-reviewer's understanding of study design will be fed back to investigator for checking
• Ethics issues
  • Is an IRB exemption needed?
  • What are ramifications of judging study yield on effect estimates and their uncertainty intervals vs. rejecting H0?
• Statistical issues:
  • Interim analysis may not have been specified in the proposal, or our interim analysis methods may be at odds with pre-specified statistical analysis plan
  • Alpha spending
  • Adequacy of normal approximation to Bayesian posterior distribution (will speed up calculations if approximation is adequate)
  • Emphasis on estimation vs. signal detection
• Reporting issues
  • Decide which approach(es) should be presented to the SRC
  • Partially automate reports once we learn more; find out how uniform the reports can be
  • Automatically produce portfolios with snapshots of multiple studies
  • Feedback to investigators that will not unblind final analysis
• Initial implementation
  • While recruiting a nearly full-time MS biostatistician, each interim monitoring PhD statistician will assume responsibility for one study from start to finish
  • Start making study design dictionary
  • Need to make a list of benefits to investigators
    • Availability of future resources in the future
    • Educational opportunities for investigators to learn more about study design
    • Increasing awareness of advantages of sequential experimentation and better designing their next study
  • Need new text in approval letters
  • Need to finish enumerating new RedCap auxiliary database fields and find ways to pre-populate some of it from StarBrite and other sources

21July09

Attending: Vincent Agboto, Kirk Lane, Frank Harrell, Chang Yu, Li Wang, Dana Blakemore, Liz Heitman, Sharon Phillips, Dan Byrne

• Discussed 2 Meharry submissions to VICTR and need for more detail in proposals
• Kirk discussed the confusion that many investigators have - the belief that IRB review covers every aspect of ethics
  • Example in which protocol specified that subjects refusing to be enrolled in a study could be gotten around by going back to the IRB to ask for a single exemption to the need for consent
• Should we raise awareness of the use of non-science-based therapies used and increasingly advertised at VU?
  • Consider offering a training session for the marketing department? This could be delivered by ethics + biostat.
    • Need to cover the issue of marketing clinical trials as benefiting patients as long as a specific trial is not mentioned
    • Also point out that many surveys require IRB approval
• Liz reported on the national CTSA ethics key function committee, involved in drafting a core competencies document including biostat/study design
• Liz discussed the climate survey to test the waters at 3 CTSA sites; covers research environment etc.; will use RedCap and will not be limited to CTSA sites
• Frank reported on the approval of the interim monitoring VICTR supplemental grant application. This is for 2 years. We need to finalize the criteria for selection of VICTR-funded studies for mandated interim monitoring. Consider making a presentation at an upcoming SRC meeting. There are opportunities for research on research.
• Discussed the possibility of unitizing the cost for ethics reviews
• Checkbox for ethics was added to studio requests

16June09

Attending: Lane, Blakemore, Xu, Wang, Phillips, Yu, Harrell

• Discussed Nature Neuroscience article
• Discussed Frank's visit to new Cincinnati CTSA
  • Like us, no methodologists are on their IRB, but they give more input to IRB
• Clarified coercion vs. incentives in clinical trials, especially in pediatrics
  • Paying for time and inconvenience, not for the deed; are not paying people to take risks

19May09

Attending: Churchill, Harrell, Xu, Wang, Phillips

• Need to check that Ethics has a checkbox in studio requests
• Need for statisticians to more often insist on a statistical analysis plans
• Looked at paper in Nature Neuroscience 1May09

21Apr09

Attending: Phillips, Heitman, Harrell, Scott, Byrne

• Discussed Crohn's disease proposals
  • Could lead to a paper linking ethics to stat
  • IRBs are not aware of some statistical issues. It would be good to address the question of whether waste is harm.
  • This may be suitable for a Ethics/Biostat case presentation in our seminar series
• Brief update on CTSA supplement application

28Nov07

Attending: Heitman, Churchill, Donahue, Lane, Yu, Wang, Byrne, Harrell

• Talked about integration of research ethics and biostatistics, while they are separate with regard to national CTSA consortium
• Liz clarified that there is much of ethics that is not clinical per se, e.g., responsible conduct of research
• Liz suggested all of us take a close look at the http://www.amstat.org/profession/index.cfm?fuseaction=ethicalstatistics ASA code of ethics, which is not as strong as it could be
  • A joint session would be valuable
  • Kirk suggested that an end result could be a paper circulated to other CTSAs
• What are points at which research ethics pre-reviews can take place?
  • CTSA research ethicists can come to random biostat clinics, then we can learn to better target participation
  • Studios haven't by default included research ethicists
    • Perhaps triggers for ethicist involvement should be described and given to the studio director
    • Studios officially start 2008-01-07
  • Kirk is involved in GCRC reviews
• We discussed the strong interface of ethics and biostatistics in the area of study design and study futility/waste
  • Implementation of research ethics and its combination with biostatistics in the context of daily practice is worthy of a paper (a very practical one)
    • Include a section on the ethical implication of design choices
• We briefly discussed the Palmer paper
  • Larry feels that since a trial intends to affect future collective good, the within-trial ethics may be thought of as side constraints, not as all encompassing.
  • Liz brought up the example of how do you know whether a drug is safe to use on children until you actually test it on children
  • Some informed consent documents say that the trial is not intended to benefit you, the subject
  • To the point that Palmer states it is unethical to make a sample size large enough to estimate an effect with precision (as opposed to being just large enough to detect a nonzero signal), using minimal sample sizes may be unethical to the larger population
    • DMCs have prolonged studies to result in data that can change practice
  • Ethics of use of placebo is worthy of future discussion

Action Items

1. Read ASA ethics document and think of points to discuss/rework/add
2. Think about impacting the IRB directly
3. Consider working on the practical paper described above

15Jan08

Attending: Churchill, Donahue, Yu, Wang, Byrne, Harrell

• Discussed placebo effects in clinical trials - when is it ethical to have a placebo group?
• Risk perception - an issue for future discussion
• Do patients have access to trial protocols?
• Larry's vision for how we can work together * Determine ways in which it makes sense * Biostatisticians catalog issues that come up with investigators, keep in mind places where ethics makes sense to address * Research ethicists could be involved in biostatistics teaching, e.g. MSCI
• Liz is designing a document on responsible conduct of research for CTSA
• Are there issues in need of discussion related to financial incentives to investigators?
• Will it work for IRB to not have biostatisticians but to rely on the scientific review committee?
  • Not all studies reviewed by IRB go to SRC
• Review tracking for research ethicists - Larry said we should do this

Action Items

1. Continue to read ASA ethics document; be ready to discuss
2. Watch for teaching opportunities
3. Watch for research ethics-related issues in our biostat collaborations; think of ethics questions that come up or should come up; how does improper interpretation of evidence fit in?
4. Insure that a research ethicist is always on the SRC
5. Larry look at what items need to be tracked in a review tracking web page; Li will think about location for information

19Feb08

Attending: Heitman, Churchill, Donahue, Lane, Wang, Byrne, Harrell

• Discussion of ethics reviews
  • Kirk discussed an example in how conveyance of highly sensitive information to genotyped persons was to be done was not covered in the protocol; originated from VICTR SRC
  • We need a mechanism to trigger ethics reviews more routinely
  • Is there a need for rotating ethicist review of every protocol submitted to VICTR?
  • Suggested a retrospective review of all protocols submitted in Feb 08
    • Count the number of protocols that warranted a fuller ethics review
    • Catalog the reasons
    • For those warranting further review, review minutes or IRB to find out how many were already flagged for a potential ethics concern
  • Reviews could allow ethicists to map criteria for triggering reviews, once a large variety of protocols are seen
  • May be good to coordinate with Jan Zolkower, clinical research advocate
  • Liz will talk to Denise Roe to coordinate with the IRB and find out about confidentiality of minutes
  • Split Feb08 protocols by four ethicists; Kirk has already reviewed 2; will be 4-5 protocols each
  • Need to anonymize with regard to investigators; only need narrative
  • What do we do if we identify an ethical issue not identified by the IRB? Liz will talk to Denise about this.
  • Data would be great for upcoming progress report and for a paper
  • Kirk gave another example in which data were collected with no plan for analysis of that data component
  • In the future we might propose that protocols with CTSA ethicist concern don't go to SRC until the problems are resolved
  • Liz developed an initial outline of study components needing ethical review
    • confidentiality
    • consent, minimal harm
    • selection criteria (inclusion/exclusion) / discriminatory labeling
    • duel use potential
    • deception and disclosure
    • unclear goal of collection of random data / fishing / lack of analysis plan
  • Need to think about how/when studios can help
• Tabled ASA ethics guidelines discussion for next meeting

25Mar08

Attending: Churchill, Lane, Yu, Byrne, Harrell

• Debriefed on yesterday's special core meeting to assist Sports Medicine
  • Need to follow-up on assent documents for patients < 18 years old
  • Related to percentage criteria for site inclusion the most important consideration is watching for undue pressure on patients
  • Sometimes a large number of refusals may reflect a good consent form
  • Next time make the context more clear as well as the data presentation
  • We can usually differentiate IRB vs. non-IRB issues
• Ethics review pilot study for Feb 2008 protocol submissions
  • Liz and Kirk met with Denise Roe of the IRB; IRB was happy for this pilot study to proceed and recognized there are ethical considerations outside the standard regulatory review mode; Ms Roe thought this would help them
  • There are 7 deidentified protocols to review; all have been distributed for review
  • It would be useful to extend this over a few more months
  • Can develop a list of concerns from the reviews, and also look out for known concerns from the literature
  • The bioethicists might educate the biostatisticians about points to look out for in our reviews of all the protocols; this would trigger ethics reviews
    • Cannot do this by looking at specific protocol sections
• Keep looking out for opportunities for ethics reviews (studios and elsewhere) and ethics-biostat reviews of current studies or those in the planning stage
• Kirk brought up the issue of subject notification of results of genetic testing in an observational study; the investigators were concerned because of a one-year followup of behaviors linked to the gene

15Apr08

Attending: Churchill, Heitman, Wang, Byrne, Harrell

• Discussed an ethics issue from Kirk: 2 April 2008, Ongoing study involving children, weight and diabetes. Design was to have participants that were enrolled come to the CRC to have height and weight recorded. These visits were scheduled to coincide with clinic visits so as not to require duplicate trips to the VUMC area. The consent form allows the researchers to contact participants. The physician who sees these patients in the clinic is a co-investigator in the study. The issue is that some “small” number of patients did not come to the CRC to have their data recorded. None of the patients with missing data had officially withdrawn from the study. That is none of these individuals or their parents or guardians (consent provider) had informed the investigators orally or in writing that they wished to withdraw. Since these are pediatric patients it is usual practice to record height and weight at clinic visits. The co-investigator (clinic physician) stated he could “get the missing data from the medical record.” Was this appropriate? The short answer is “no”. However, since there is consent to re-contact the participants, no withdrawal notification has overtly been expressed, the missing data exists in a “private” file, the co-investigator has legitimate access to this information, there seems to be a reasonable way to proceed. The investigators should contact the participants (consent provider) and inform them that there is missing data. That the data exist in the medical record and that their physician is also an investigator in this study. Does the study have permission to obtain the height and weight information from the clinical data (medical record)? An affirmative response is required before proceeding.
  • Feeling of the group is to find out what the protocol says about re-contacting subjects. At any rate a call from the investigator will be confusing to the subject. An up-front solution would have been to mention in the informed consent that the investigator has access to the medical record. Recommended to call IRB and ask the question about the use of the medical record and about re-contacting subjects.
• Liz is going to a meeting at Washington University sponsored by the Office of Research Integrity: http://epi.wustl.edu/epi/rcr2008.htm; concerns responsible conduct of research; several interesting topics
• Discussed how existing courses (e.g., clinical trials methodology) could have a segment on rcr added
• Ethics reviews for Feburary 2008 have been completed; some wait to be recorded; still intend to add another 2 months' worth of reviews
  • Biostat reviews need to compile a list of trouble areas they think might trigger ethics reviews
• Next time consider going through the WHO checklist for what informed consent and protocol documents should contain
• Frank relayed the discussion from today's BERD online resources task force about the potential for having a discussion board for biostat (and possibly ethics?) questions from CTSA researchers nationwide

20May08

Attending: Heitman, Wang, Byrne, Harrell, Lane, Xu, Agboto

• Discussed Liz's biostat seminar presentation
• Discussed current problem of having an analysis file containing non-consented patients even if a flag can be used to remove them from analysis; non-consented patients were used in very indirect analysis decisions
  • Could bias sample from post-facto consent; subjects unhappy with their outcome may not give consent; need to monitor re-consent acceptance fraction
  • What is the basis for re-contacting patients to re-consent them?
  • Have a file with study IDs of patients who are not known to have consented; however for one site we don't have the list
  • Need to remove non-consented patients from the primary dataset and re-create the analysis file
  • Must exclude all subjects from the site having no list unless a list of definitely consented patients can be obtained; those patients may be included from that site
    • Note: The list of non-consented patients from the site in question may be available soon so this may not be an issue
  • IRB should be notified each time a patient is discovered not to have consented
  • How should this be written in a paper? Need to reflect that non-consented patients were used in preliminary analyses
• Frank relayed an ethics story from a statistician at another university
• Ethics review update: files are being saved; need to be distributed

17Jun08

Attending: Churchill, Wang, Byrne, Harrell, Lane, Yu, Heitman

• Discussed an issue with a surgical study with only 5 controls
• Update on multi-center orthopedics study
• Liz summarized the CTSA Clinical Research Ethics Workgroup teleconference from yesterday; she is on the educational curricumum task force of CREW
  • A task force is trying to write a white paper on clinical research ethics consultation; a survey of the 38 CTSAs will be done
  • Goal is to document "best practices"; this is a good time for our input
• Retrospective ethics review of CTSA protocols is at a standstill
• Need for research ethicists to give biostatisticians a 'checklist' to alert them of potential ethical issues to note while doing biostat reviews
• Might be good to bring a grad student in to screen protocols for ethics concerns; could ask for CTSA funding for this as a research project; goal is to produce a validated template that may be useful for biostatisticians; might be done as a voucher
• Kirk reviews all proposals that go to the SRC

19Aug08

Attending: Lane, Heitman, Xu, Byrne, Harrell, Churchill, Saville, Wang, Phillips

• Welcomed Sharon Phillips (new member, collaborates with Surgical Sciences) and Ben Saville (new faculty member and visitor to CTSA) to the meeting
• Reviewed status of extra 2-month retrospective review for ethical questions, and creation of a checklist and template of issues to address
  • Format of voucher funds application does not fit this type of research; asking for $2k
  • Will deal with development and rewview phases
  • Need to deal with clearance of grad student for access to original reviews and protocols
• Larry reported on multi-center orthopedics study; would be useful to have another roundtable discussion
• Larry related an issue from Emergency Medicine - questions raised on pink sheets; might suggest clinic
• A paper should emphasize what is in it for the investigators to have an early ethics consult/review
• Kirk discussed an example where the coercive nature of monetary/goods payment to a child for participating may be a problem; issue is also related to the source of the goods
  • Compensation vs. inducement

23Sep08

Attending: Lane, Xu, Harrell, Saville, Wang, Phillips, Heitman

• Griped about the economy
• Talked about biostat voucher submissions, especially for surgical research
• Discussed rules about direct participation in grant proposals
• Problems have arisen for biostat voucher awards that are insufficient for the work required (e.g., for large complete databases)

28Oct08

Lane, Phillips, Xu, Churchill, Harrell, Wang, Yu, Heitman

• Discussed upcoming External Advisory Committee meeting and new initiatives
• Discussed application for graduate student funding for retrospective ethics reviews; Diane is still working on scheduling
  • New qualitative research core may help
• Chang raised an issue of a pilot study for which the investigator wanted an interim analysis
• We might suggest to CTSA that it more closely monitor subject accrual, in addition to doing sequential analysis of results
  • Perhaps operationalize with Bayesian monitoring
  • Rather than SRC approving only a small (futile) sample size, approve the maximum sample size and demand sequential monitoring
• Liz reported on the Am Soc Bioethics & Humanities' Clinical Research Ethics Work Group
  • Consultation task force - research ethics consultation relationship with IRB (e.g., service advertised to IRB; legal discoverability; treat as QI or research with PI consented or oversight); distributing templates and best practices in clinical research ethics consultations
  • Need for a statement on financial COI; issue on institutional vs. consortium rules on COI
  • Need to clarify whether CTSA percent efforts cover any of our consortium work
  • New CTSA strategic plan issued by the consortium steering committee creates key function committees which do not seem to include ethics; ethics appears only once in the strategic plan whereas compliance appears often
• Next meeting: spend most of the time on brainstorming template content; Liz went over some aspects of the WHO ethics committee document that married ethics & biostat

18Nov08

Attending: Byrne, Harrell, Wang, Phillips

• Discussed progress related to the grant proposal's original aims
• Regarding putting innovative designs into play, Dan suggested doing a retrospective review of a few completed analyses to assess whether the answer would have been known sooner
• Ways of assessing impact to MSCI students are being considered
• Discussed adding a bank of covariables to RedCap
• Need to check that in proposals listing a statistician the investigator has actually worked with the statistician
• We need to be vigilant in not approving industry-sponsored proposals that are not good science or have inappropriate analysis plans
• Proposals using Cohen's effect size need to be improved

16Dec08

Lane, Xu, Harrell, Heitman, Byrne, Yu

• Need to revive the meeting to discuss CTSA coverage of ethics review
• Upcoming biostatistics ethics case conference will be on authorship; Kirk discussed another possible example
  • Liz emphasized the importance of determining authorship as early as possible
  • Good subjects for a writing course
  • Need to clarify power of senior and first authors
  • Another good area to cover is delve into how deep a co-author needs to be able to vouch for sections of a manuscript
  • Compare to department grant proposal policies for persons named as key personnel
  • Tried to find authorship criteria for biostat journals, finding only guidelines for Biometrics
• Can adherence to ethical conduct of research be quantified?
  • A survey of perceived ethical conduct at VU could be done, including surveying the community's perception of the ethics of VU research
• Kirk discussed a recent studio where patients having medical devices surgically implanted were being studied using data generated by the device
  • At issue is whether patients having computerized devices implanted should be informed at implantation time that they will likely be contacted to enroll in a research study in the future
    • Is it fair to exclude populations from using more expensive devices that can provide research data?
• We need to find out how many ethics consultations there have been to date.
  • Liz will contact Jan Zolkower one more time to see if she could attend one of our meetings; Jan is reviewing protocols for the IRB with regard to human subjects protection.

20Jan09

Attending: Byrne, Wang, Lane, Scott, Clayton, Harrell, Heitman, Jan Zolkower, Churchill, Phillips, Xu

• We welcomed Jan, Patient Advocate, and Terri Scott, Biostatistician III, 20% VICTR who is joining the team
• Jan is a member of the Performance Improvement Team of the IRB; attends VICTR SRC meetings
  • Jan has been involved in handling complaints and educating researchers
  • The CRC was handled separately, now things are rolled into one with advent of VICTR
  • Outreach to VU community, Nashville community, Cancer Center
  • Health fairs for potential study participants; IRB web site has a participants' page
  • Video being made for patient TV channel, working with marketing
  • Last year 17 complaints for 3349 active studies, from participants, families, or study staff
    • Example: lack of compensation, lack of time to review study
  • Studies can be audited with respect to the monitoring safely level assigned to them
  • Beginning to meet with PIs
  • Checks for existing plans for example for what monitoring is done if a patient is taken off a drug
  • Jan has reviewed many VICTR applications
  • Research ethics not directly related to regulation or subject protection is not specifically addressed
  • Larry pointed out the usefulness of getting investigators to think about research ethics from the outset of the development of a new study
  • Need to watch out for small financial requests to VICTR that result in expedited reviews have enough opportunity for research ethics review
• Liz talked about research ethics consultation; she reviewed some of the issues that have arisen
  • Discussed pack of forethought about ethics in a proposed project for which a studio was recently held; investigators thought of ethics as a "just in time" component even though the intervention was dangerous
    • a good example of the inter-relatedness of ethics and study design
  • Research ethics may need its own space on the StarBrite review web page
  • It would be nice if investigators came to clinics in the initial stage and we could alert ethicists to attend

24Feb09

Attending: Heitman, Churchill, Scott, Harrell, Byrne, Phillips, Wang, Clayton

• Plan for the meeting about voucher-based funding for our kind of projects (specifically having someone review CRC proposals to identify specific ethical questions they raise).
  • No funding for ethics vouchers per se; no funding for qualitative core functions
  • Can a biostatistician do the screening reviews? Need high sensitivity.
• Look at the “institutional climate survey” designed by my RCR colleague to consider whether it might be useful as an instrument to measure changes in the ethical climate of VUMC research over the course of the CTSA grant.
  • Two other schools interested (both are members of CTSA)
  • Interested in predisposition to cheating/cutting corners
  • General issue of sensitization and other unrelated interventions in any simple pre-post design
  • A general current climate survey may be beneficial
  • Would have been nice to do this before VICTR began
  • Discussed identifiability of respondents in national and local surveys and whether the survey should be national instead of local, after piloting
  • Effect of nonresponse bias could be large
  • Number of questions is too large
  • Would be good to compare with previous survey instruments that dealt with more specific infractions
  • Another consideration is to classify the type of research (e.g., 'omics' research has more temptation)
• Progress report - need input very soon

31Mar09

Attending: Heitman, Churchill, Scott, Byrne, Yu, Harrell

• Industrial Pharma Clinical trial reviewed by Frank - What does it mean to be "dangerous", "unethical" for a statistically underpowered study?
• Study is powered to detect a difference in a binary endpoint; it is over-powered to detect an effect on the underlying scale
• Discussed problems with CDAI scale; was never a justification of a cutoff of 150 in the original 1976 paper (which also overfitted the predictors of remission). A score of 100 barely reaches a probability of 0.5 of remission.
• Ethics issues: patients not consenting to be put at risk for a trial that doesn't yield valid results (scientific integrity is the reason we can treat them as subjects in the research); there is an obligation to reduce hazards in the trial
• Some patients may be enrolled needlessly; trial is longer than necessary; if the treatment is effective, the trial might delay use of the treatment in the control patients who were accrued last
• Patients rarely see information about payments to investigators
• Dan proposed an interim analysis; so far there has been no response from the investigator
• There are some substantial ethical questions that ought to be addressed
• Investigators are obligated to reduce the number of subjects exposed to harm and the amount of harm to which each subject is exposed
• Future topic: ethics of using a less informative response variable in a larger number of subjects vs. a more invasively obtained response variable in a smaller number of subjects

---

• Report out from VICTR Operations Workgroup meeting: VICTROpWorkgroup
• Report out from VICTR Scientific Review Committee meeting: VICTRSrc

Biostatistics Meetings

20Nov09

Attending: Byrne, Harrell

• April CTSA (SCTS) meeting in DC - Harrell, Byrne, Wang, Yu will attend.
• Poster - Dana - Overview of how the CTSA Biostatistics Group Supports VICTR
• Poster - Li - How sample size and power consultations improves the success of projects
• Poster - Terri - VICTR training
• Poster - Chang - I can present a poster on themes like: 1) enhanced CTSA support results in publications; or 2) CTSA study-motivated statistical methods research. Probably go with the first one.
• May be best to make posters for VU use for now as it may be too late for submission to SCTS
• Chang is representing us at the CTSA consortum evaluation task force. Are there others we should be more active in?
  • Perhaps new BERD groups: imact and reproducible research/good stat practice
• Online web training with Mike Stein
• Expanding the grant prereview studios - list of NIH study section members.
• Supplement for interim monitoring.
• REDCap Survey training on Dec 3rd at 9 am.
• Contribution to the consortium and CTSpedia.
• External Advisory Board 30 minutes - need ideas for content?
• Annual report
• Renewal plan
• Prepare wish list for CRC equipment carry forward $251,001
• Research on Research - IAP, need more ideas.
• We need to create something of value that we can show others - papers, posters, software, etc.

11Sep09

Attending: Wang, Yu, Harrell, Scott, Blakemore, Phillips

• Industry protocols
• Discussed consult on BMI, triglycerides, HDL, diabetes, and metabolic syndrome in adolescents; need for more fucused questions, perhaps at another clinic
  • Problem with the use of percentiles of BMI; probably best to adjust for age in some fashion
  • BMI may be defective, i.e., a "normal" may be at idea weight at every age but have their BMI change over time
• Discussed the need to be on the lookout for opportunities to assist with research related to the H1N1 epidemic

21Aug09

Attending: Lane, Clayton, Shyr, Harrell, Maron, Yu, Heitman, Wang, Bernard, Phillips, Blume

Special biostat-ethics clinic for David Maron's cardiovascular treatment clinical trial

17Jul09

Attending: Yu, Wang, Phillips, Harrell, Scott, Byrne

• Brief discussion on pre-reviews
  • Check for clinic notes
  • Need to make sure that Meharry investigators are fully documenting study designs and sample size methods in CTSA proposals
  • Hope that Vincent can attend these meetings once/month
• Need to better advertise clinics, Friday workshops, etc.
• Authorship issue related to a voucher
  • Referred to ManuscriptPolicies
• Meeting coming up about voucher payments
• Studios continue to be successful
• Can search PubMed for RR000095 or RR-000095 or RR024975 to monitor VICTR grant citations
• Upcoming pre-JSM meeting of CTSA statisticians

19Jun09

Attending: Wang, Blakemore, Yu, Harrell, Scott, Phillips

• Welcomed Dana to the group
• Need to let Shraddha know about Dana
• Need to update review randomization list and its allocation ratios
• Dana will rotated in at 0.1 effort share (Li and Chang will run the program); Chang, Frank, and Dan will assist Dana with reviews initially
• Update on WISDOM trial
• Update on BERD esp. importance of 1-1 work with investigators (part of the impact group)
  • Also: online resources (http://ctspedia.org), minding the gap, good stat practice, eval task force
• Discussed protocols from Meharry Medical College

15May09

Attending: Phillips, Wang, Yu, Harrell, Scott, Xu, Byrne

• Supplement grant on interim monitoring
• Scope of voucher requests
  • NOT for funded projects (grants, contracts)
  • Priority to design
  • Priority to trans-department high-profile high-impact projects
  • Can request voucher request data from VICTR
  • Need to do excellent time or task tracking and measuring supply and demand
  • Need to note completion of vouchers; can keep a voucher open all of the time wasn't used
  • Should VICTR ask home departments to pay any matching funds, as a governor on the system?
• Crohn's disease reviews
• Internal Advisory Committee

14Apr09

Attending: Phillips, Wang, Yu, Harrell

• Crohn's disease applications and SRC
• Supplement grant brainstorming

23Jan08

Attending: Donahue, Harrell, Wang, Yu

• Report out from 2 VICTR meetings
• Discussed activity tracking
• Discussed Meharry Biostatistics coordination
• Discussed protocol assignment and completion tracking
  • Expand status of reviews to include the following; consider using EDITTABLE plug-in
    • assigned/completed/in process (?)
    • Design problems noted: none/minor/moderate/severe
    • Analysis plan problems noted: none/minor/moderate/severe
    • Recommendation: accept/studio/clinic/resubmission after revisions
• Contribution to VICTR brochure; incorporate some text from proposal aims

13Feb08

Attending: Donahue, Harrell, Yu, Wang

• Reviewed brochure; will add two access points - clinics and applying for support through VICTR
• Discussed the fact that we are not supposed to provide help unless the investigator applied through VICTR, was approved, and we get a notion of the scope of work
• Need to determine how time scope is estimated and is quoted to the applicant
• Any investigator who is not ready to apply to VICTR who does not already have biostatistics support through the collaboration plan only has clinics as a recourse
• Be prepared for providing grant proposal assistance
• Frank reported on yesterday's teleconference with NIH BERD committee on electronic resources
• Problem with expedited reviews getting serious biostat concerns and still being approved
  • Helped by getting reviews done by the Wednesday after Friday notification
• Bayesian Biostatistics Conference at MD Anderson: 7 Biostatistics members attended, learned a lot about adaptive design; met with Vincent Agboto, MMC director of biostatistics
• Frank reported on successful teleconference with outside statistical center after review that raised some concerns
• Possibility of requesting additional funds for IT development, linked to RedCap

27Feb08

Attending: Donahue, Harrell, Yu, Wang, Agboto

• Noted that Quick Links in StarBrite referred investigators directly to Biostatistics. Changed the twiki page to tell them to apply for resources through StarBrite.
• Ask Paul Harris to change link to Quick Links to list Biostatistics up front.
• Brochure needs to be consistent with this
• Vincent talked about CTSA activities at Meharry. There are active awareness/educational processes going on to make investigators aware of the resources. Investigators will go through StarBrite to request support through a voucher. For the present, a database at Meharry is used to note requests. Until Vincent recruits more biostatisticians, some help may be obtained (through StarBrite).
• Rafe asked whether an MS biostatistician who is 1/2 VU 1/2 Meharry would be a good idea.
• We need to be ready for grant proposal assistance requests. Frank has told Jill that we will be available for grant development requiring short- and medium-term collaboration. Longer development phases need to be covered by the Collaboration Plan.

12Mar08

Attending: Harrell, Yu, Wang, Byrne

• Discussed recruiting and how to handle gap when Rafe's departure
• Li will take over protocols assigned to Rafe for review, with the other three biostatisticians helping often
• Would be helpful to add a checklist of protocol problems to avoid
• We need to be diligent in recording major analysis or protocol issues in our review status table
• Format CTSA twiki activities to match the progress report template to make the next progress report easier
• Bring up to ethicists the possibility of doing prospective reviews in April or May.
• Updates from CTSA Operations Committee meeting 11Mar08
  • Think of biostat/ethics needs for funding from year one funds
    • Think about software that will help the infrastructure
    • Jeff Horner may be able to help, extended RecCap more for R
  • Think of more ways to reach out to MMC and other institutions
• Do we want one replacement for Rafe or two, splitting the effort?
• Lei might work with CTSA until 1Jul08
  • May be some applications for analysis of EEG and sleep data; Paul Harris and Dan have worked on this before
• Get CTSA grant number in papers we participate in if the participation was even somewhat related to CTSA
• We need to start making progress in research on research
  • Dan will look at outcomes of some past protocols especially with regard to publishing papers
    • StarBrite could track outcomes of research; would have a bigger impact if requests for future CTSA resources were denied if outcomes of previous investments were not reported
  • Frank offered to do a live extreme collaboration during a GCRC workshop or clinic
  • Continue to think about publications we would want to write, including common flaws in protocols; a valuable paper would be on our joint work with ethicists
• Training in responsible conduct of research would be beneficial to all statisticians (also refer to ASA code of ethics)
• Upcoming biostat/ethics clinic for sports medicine

26Mar08

Attending: Yu, Byrne, Harrell, Wang, Xu, Agboto

• Welcomed Lei Xu to the group
• Need to rotate Lei and Li into the protocol review assignment table
• Lei and Li might bring rough drafts of reviews to clinics for comments and assistance
• Need to clarify why and where the short review (or pre-review) are placed on VCAMP, on topic CTSAreview; investigator needs to see comments as early as possible, on the VCAMP web page
• Demonstration of RedCap protocol review database

09Apr08

Attending: Wang, Yu, Byrne, Harrell, Xu, Agboto

Report from 1Apr08 CTSA BERD Teleconference on Evaluation

* Dave DeMets put together a attached spreadsheet. We need to review.

• The plan is to have 6 major categories of evaluation:
  • Consultations and Collaborations
  • Grants
  • Protocols
  • Publications
  • New Methods
  • Education
• Within each of these categories the workgroup will define specific metrics.
• A few items that we need to discuss at our next CTSA Biostat meeting:
• We need to demonstrate how we have leveraged our CTSA money to accomplish more, improved efficiency and increased our impact.
• We need to document how we have expanded to reach out to other departments, schools, and parts of Vanderbilt.
• It will be very important to get collaborators to include the CTSA grant number on all publications that result from our CTSA-supported work. We can include the following as a footer in our emails or just send to collaborators.
• Publications that result from VICTR support should reference the grant. A final reprint should also be sent to the VICTR-CRC office so the publication may be incorporated into our files and bibliography as required by the NIH. The following manuscript citation is suggested: "Supported in part by Vanderbilt CTSA grant 1 UL1 RR024975 from NCRR/NIH"
• We need to create a list of essential variables that should be recorded in most studies and put this into the base RedCAP default system (e.g. weight).
• We need to review the RedCAP protocol review form and move it to next level.
• Update twiki into the progress report format and keep up-to-date.
  • To avoid duplication, some material might be INCLUDE{}'d from other pages
• Discuss the paper we will write regarding CTSA.
• We need to try the RedCAP survey.
• VUspace.
• Recommend biostatisticians for CTSA External Advisory Board.
• Plan ways to use the CTSA consortium to fix something that was a duplication of effort.
• We need to make sure 100% of protocols have a formal biostat review.

Other Business

• Discussed being ready to deal with problem investigators if there are any
• Discussed time expectation for reviewing a protocol and whether you can do an adequate review without delving into the background material (e.g., references)
• When reviewing a protocol/statistical analysis plan from a coordinating center of a multi-center study, we still need to do a full statistical review. This may result in better stat plans for future studies from that group, and hopefully, for the current study.
• We are starting to review pilot study proposals; we need to make sure we have reviewed all of them
• Remember that most proposals not asking for biostatistics support still need concrete hypotheses and sample size justifications

23Apr08

Attending: Dan Byrne, Li Wang, Jeff Horner, Chang Yu, Lei Xu.

• Internal Advisory Meeting
• Protocol review - need to ensure that 100% of protocols in SRC folder have a biostats review.
• RedCap covariates - discuss spreadsheet.
• Research on Research Project - discuss questions and variables.
• Need to encourage our collaborators to apply for pilot and other CTSA resources soon.
• Job satisfaction and workload.
• Need to line up CRC workshops and speakers.
• Review CTSA goals and aims to assess progress.
• CTSA wiki for the consortium - Spreadsheet from Hell.
• Studios
• Vouchers
• Discuss publications demonstrating we have accomplished one of the aims.
• Progress report format on twiki - need to keep updating.
• Prepare for Cody Hamilton, July 1, 2008.
• Wish list - equipment, books, etc.
• Jeff Horner is working with Paul Harris too add features to Redcap using Hmisc and describe.

14May08

Attending: Byrne, Harrell, Xu, Wang

• The CTSA biostatisticians’ workgroup has an evaluation task force with monthly teleconferences.
  • The attached spreadsheet was put together by Dave DeMets to summarize the outcome measures from the various CTSAs.
  • See the “Measures” tab. (We will give them an update about our metrics.) This spreadsheet will give you an idea about what other CTSAs are measuring.
  • The group decided to categorize these into 6 major areas. See top of the BERD Evaluation outline document.
  • The plan is to write definitions for these outcomes.
  • The idea is that we define the appropriate metrics rather than having NCRR impose metrics they think up on us.
  • Scott Zeger, chair of biostats at Hopkins, is the leader of this workgroup and is very effective.
  • There is talk about adding a 7th major area = Contribution to the consortium.
  • Scott’s suggestion for next steps is to develop ways that we can share ideas about how we leverage our time and effort to positively influence the greatest number of researchers with our existing resources. How can we maximize our impact? The idea is that each of us has some techniques that we use for maximizing our impact. We want to find a way to share these best practices with the other CTSAs. * We need to capture much if not most of these metrics through the normal workflow processes. How can we accomplish this?
• Frank and Dan talked about the last BERD online resources teleconference
  • Central online teaching resources vs. links to resources at all CTSA sites
  • Updating http://wikipedia.org with stat definitions
• Discussed an interesting pilot study analysis of blood sample deterioration, with multiple blood analyses by patient; discussed summary measure approach, serial data parametric analysis, and having an observer write down the time until deterioration for each patient then do a simple one-sample confidence interval

27May08

Attending: Byrne, Yu, Harrell, Xu, Wang

• Should we change how protocol reviews are scheduled? Current method is working pretty well with occasional very heavy weeks for one reviewer
• Need to define 2 objectives for next year for CTSA Biostats, e.g.
  • Raising the level of science with more comprehensive protocol reviews
  • Rejecting at least 20% of submissions, many of these outright rejections
  • VICTR Grand Rounds; constructive review of published papers, emphasizing that there are other alternatives, not just right and wrong approaches; discuss alternative designs also
    • Want to do this only on articles published by non-VU reseachers?
• Need to get the message out that multi-center protocols based outside VU still need to have their scientific act together
• Use NCRR mission statement in wording these objectives
• Equipment ordered
• Workshop speakers needed for Friday
• Graphics core? Have a puzzler-type contest?

25Jun08

Attending: Byrne, Yu, Harrell, Xu, Wang, Horner

• Jeffrey gave a demo of the new Rapache Redcap extension
• Frank discussed budget issues and supplemental application from online resources task force related to central BERD twiki
• Use the following in our goals and objectives: "The CTSA program enables researchers to provide new treatments more efficiently and quickly to patients."
• Discuss proposing new metric: 5 best stories/discoveries; we need a way to capture these in an ongoing way; perhaps a checklist might help
• Equipment: bought anatomical model + books, laptops, software; smartboards should be coming
• Future wish list
• Protocol review: we need to be firm about sloppy or incomplete work and be clear on rejecting inadequate plans
• Vacation coverage
• Li Wang - Mentoring plan
• Li Wang - Green card
• IAP evaluation plan
• Redcap standard variables; need version control or repository

For next meeting:

• Expand teaching
• Paper with Nancy Brown
• Next progress report
• Upcoming conferences
• Recruiting and Ben S.
• Poster contest
• CRC workshops
• Studios and vouchers
• Clinical research solicitation feedback meeting
• Vanderbilt HRA
• Research on research process

23July08

Attending:Xu, Wang, Agboto

• We discussed issues related to protocol review process: Per Vikki's suggestion: "We suggest an approval" is too strong as it sounds like we are approving the protocol when we are only approving the stat part. We would suggest we just make a statement that it ready for agenda or hold for agenda.
• We discussed statistical issues of our current projects.

27Aug08

Attending: Yu, Byrne, Saville, Xu, Harrell, Shraddha Nigaveker, Wang, Phillips

• Introduced Shraddha from VICTR Research Support Services, supporting SRC
  • Goal is to have new version of StarBrite in 3-6 months
  • Demonstrated the RedCap implementation that will bridge the new StarBrite implementation
• Discussed vouchers; will be discussed at faculty meeting
• Lei talked about an old voucher she was working on which is highly problematic
• Sharon updated us on Surgical Sciences
• Frank talked about possible sequential monitoring of selected studies

10Sept08

Attending: Yu, Byrne, Wang, Phillips, Xu

• Chang discussed an ED project of n=20 + 80 pilot with inadequate sample size justification.
• Terri will join 20%
• 99% power $40,000.
• Need to ID the biostatistician on the protocol page 1.
• Help Lei find PIs to get publications.
• Need to redo the protocol assignment Lei decreased.
• Update on evaluation task force teleconference 7/29/08.
• Define the 7th major evaluation area and submit?
• Review the attachment "BERD Evaluation Guidelines V3.doc"
• Use CTSA as seed money to leverage more money and support.
• Show the leveraging you have done with grants and department collaboration support.
• Function as catalysts.
• The CTSA cannot do all of the biostatistical work. Need to recruit new faculty and staff for the collaboration plan. The CTSA biostatisticians can help identify needs and sell other departments on the collaboration plan.
• The CTSA activity is bigger than the funded part. This is important so that we can spend time doing more than "billable hours" working on specific protocols.
• We need to keep track of our CTSA work but not billable hours.
• We need to try a CTSA collaborator satisfaction survey using Paul Harris' RedCap survey tool or StarBRITE. The survey needs to be short and the cover letter needs to be carefully written. See consortium twiki. Consider sending every 6 months.
• Build the infrastructure to improve efficiency.

22Oct08

Attending: Yu, Byrne, Wang, Phillips, Xu, Harrell

• Discussed testing of new version of Dupont-Plummer PS software; possible request from Bill for CTSA support for testing/proofreading
  • Frank will alert the Online Resources CTSA consortium about this
  • Also consider web applications: how much trouble would it be to convert PS to an R web app?
• Voucher procedures
  • Requests not involving already-collected data (favored; planning process with early biostat involvement): voucher can be approved quickly; for selected applications recommend a clinic for jump-starting the project and exposing other statisticians to interesting brainstorming sessions
  • Requests involving already collected data: scrutinize more closely, require a clinic before voucher is officially submitted
    • Clinic will determine if dataset is appropriate for the intended purpose, and estimate the statistician time required; one or more CTSA biostatisticians will have to be present and will require coordination
    • For surgical research proposals emphasis is on appropriateness of dataset (and readiness for analysis)
• Status of open biostat vouchers
  • 9 approved, $0 spent; 6 went to Sharon where no $0 really involved. One was stopped due to attempt to change the scope of the project. System needs to add a close date and a current status category (not started, in progress, completed, stopped due to investigator attempting to change scope from what was approved). Need to also add a field for total hours worked to date.
• Tracking of utilization of CTSA-paid personnel
  • Possible categories: protocol review, planning studies and analyses (including meeting with investigator), data analysis, CTSA operations (mainly attending meetings), manuscript review/writing, studio, clinic attendance
  • Captured time spent in each category to the nearest 10 min. chunks, e.g. one month every 6 months; alternative is to do retrospective calculations if activity log is complete or can be completed; do for Nov 2008
  • Be better at documenting activities on twiki page
• Issue of suggesting fixes to problems detected in protocol review; mainly fix simple problems as we go, otherwise emphasize clinics and studios or resubmission
• Should requests for study design and study analysis be separated? Depends on complexity; discuss further next time

12Nov08

Attending: Yu, Byrne, Wang, Phillips, Xu.

• Discussed:
• Packets at the SRC. Li will print copies for the biostatistics group.
• A role we can play in making sure investigators work with their collaboration biostatistician.
• Protocols without sample size justification.
• BERD meeting next week; Frank will attend.
• Research on the research process.

10Dec08

Attending: Harrell, Phillips, Byrne, Wang, Yu, Xu

• Terri Scott will join the group in January; discussed involving her in data analysis and the Friday workshop
• Should the CRC Research Skills Workshop be renamed VICTR Research Skills Workshop? Dan will talk to Paul, Jill, Gordon, David R. about this.
• Li and group are meeting to finalize the list of "!RedCap essential variables"
• The group doing this found a great web site explaining how to prepare data for National Death Index inquiries; it includes suggested coding for gender and race
• Should lab data be included in the project?
• Report on BERD
  • Frank attended 2nd day; 1st day was for newly added CTSA members
  • Frank presented what is in the main content of our twiki site
  • There was discussion about the lack of visibility of biostatistics in the new CTSA strategic aims document
  • We need to emphasize aspects of biostat that need to be nationally coordinated
    • Gordon is interested in good statistical practice as an organizing principle
    • Also discussed credentialing of statisticians & reproducible research
  • Web computing tools are one area to emphasize as well as CTSpedia
• The #1 priority voted was development of the next generation of biostatisticians
• RedCap data cleaning module unveiled by Rob Taylor

14Jan09

Attending: Harrell, Phillips, Byrne, Wang, Yu, Xu

• Terri will focus on stat analysis consultations, teaching, and coordinating the Friday skills workshop
• Others will help Terri find speakers and will volunteer to present
• Anyone knowing of a session that would be of special interest to biostatisticians should contact Frank to send an e-mail to everyone
• Voucher charge rate ($100/hr 20 hrs total biostatistician time for a $2000 voucher) and training people who use voucher
  • Will need to keep track of time for each project
  • Created template under CTSAactivities
• Covariate project update
• Make sure to include CTSA grant number in publication (RR024975)
• Development of public relations
  • Doing video like Department of medicine
• This Friday's CRC Research Skills Workshop: “Vanderbilt's Synthetic Derivative Project: De-Identified Medical Record Data For Research Utilization”

20Feb09

Attending:Byrne, Wang, Xu, Harrell, Scott, Phillips, Yu, Agboto

• Clinic sponsorship and attendance
  • Will add a sign-in sheet for Thursday clinics
  • CTSA biostatisticians will come on average once/week
• Publications and 2 numbers needed for CTSA progress report. PMID and _ ?
• Last year’s progress report and what we need to update
• Flowchart on StarBRITE of the proper channels for CTSA Biostat requests
• Potential capacity of CTSA biostatistics
• Ways that we can help with the CTSA goals and keep the PI satisfied with biostats
• Updating the task list
• Attendance at SRC meeting: in the future we may want a rotation schedule but it will be good to attend when a protocol you reviewed is discussed

Mar09

Attending: Wang, Phillips, Xu, Byrne, Harrell, Scott, Yu, Agboto

• Problem with inconsistent powers used in sample size calculations; biostat reviewers need to be consistent
  • Advantages to plotting the power curve
  • Showed short article by Andrew Vickers on http://medscape.com
  • Discussed effect sizes and standardized effect sizes (Cohen's)
• Rejections during pre-review for bad statistics/bad science
  • Make sure that pre-disapprovals are known to Chair of SRC
  • Let's talk to the research ethicists
• Dan discussed the need to build the next progress report as we go
  • Each person needs to collect a table of vouchers executed, or find a way for StarBrite to do this
  • Keep a list of major accomplishments under CTSAactivities by calendar year
    • look at CTSA goals, papers published, courses and workshops taught
• Use of VICTR funds for purchasing equipment
• We are getting many requests related to the NIH stimulus
• JSM will have a joint GCRC-CTSA statisticians meeting