You are here: Vanderbilt Biostatistics Wiki>Main Web>TWikiUsers>JoAnnAlvarez>CastelArthralgia>PraiValidation (14 Jun 2013, JoAnnAlvarez)EditAttach

-- JoAnnAlvarez - 04 Apr 2013
# PRAI Validation

## Meeting notes

### 2013 June 14

### 2013 May 9 JoAnn, Ben, Liana, Ken

### 2013 May 8: JoAnn:

I made a change to the analysis plan because I realized that the single item FACT-ES is not binary but ordinal. Here is the revised part:
d) Concurrent/known groups validity: Single item from FACT-ES “I have pain in my joints” relationship at baseline and over time?
This is an ordinal scale with five categories. We will compute an estimate of the Kruskal gamma statistic and give a p-value and confidence interval separately for each time point, combining all groups with data at that time point. We will display this graphically with boxplots of PRAI for each level of response to “I have pain in my joints” to see the difference in distributions.
The inference is based on the asymtotic normal distribution of gamma/SE. I think we could use
rcorrcens(painjoints ~ praimean, data = castel, outx = TRUE)
to compute the gamma and sd.
### 2013 May 6: JoAnn and Ben

### 2013 May 3: JoAnn

### 2013 April 26: Liana, Joann phone call

### 2013 April 25: Liana, Ben, Joann, Irene, David, Bradley

### 2013 April 11: Liana, Ben, Joann

### 2013 April 11: Bradley, Liana, Irene, Ken, Ben, Joann

### 2013 April 4: Joann, Ben, Liana, Ken , Bradley

### 2013 March 21: Ben, Joann, Liana, Ken, Bradley

### 2013 March 14

Attending: JoAnn, Ben, Liana, Bradley, Ken ### 2013 March 7

Attending: JoAnn, Ben, Liana, Bradley, Ken ### 2013 February 21

- Liana is concerned with the association test on ECOG. It has a low estimated Spearman correlation (.3), and there were few patients in the 2 or 3 levels. I told her I would run a Wilcoxon test on the collapsed variable.

- We will probably not report any of the factor analysis. A rationale for not running factor analysis is that the internal consistency is so high.
- Ken recommended some terminology changes about types of validity to the analysis plan.
- We will run the validation analysis on the pain items only.
- We will send out the intro and methods section separately for co-author edits, and then the results and discussion later.
- JoAnn will make changes to the analysis plan as discussed.
- JoAnn will run the analysis and start the results within the next 2 weeks

- Discussed several changes to analysis plan
- JoAnn will make changes and send to group for approval and comments, clarifying that this is the AP while earlier they looked at the research aims.
- Ben will email the group regarding whether to do the validation analyses on the 16 pain items only or on the 32-item pain and stiffness items combined.

- For a way to summarize which joints were affected for paper, we could plot symbols on a diagram of the human body. I went to R clinic today and learned how to import a picture. We could take medians or some other summary within each joint and plot circles at the location of each joint, with the size of the circles representing the magnitude of the summary score, and the location of the point showing the location of the pain. Here's a way to do it:

body <- readJPEG("body.jpg") str(body) library(graphics) rasterImage(body, 0, 0, 400, 300)

- Liana would like a way to summarize which joints were affected in the descriptives of the paper.
- Irene said we should not do any separate analysis on the pain only or the stiffness only subscales, but that we should use the total, 32-item PRAI.

- Discussed aims
- Use cronbach's alpha to show internal consistency reliability. Give alpha for each item removed.
- Order should be: PCA (dimensionality), chronbach alpha, and test-retest reliability
- the amount of aims is probably sufficient for a manuscript
- to demonstrate concurrent validity, we will add an assessment of the association between the PRAI and the FACT-ES single item "I have pain in my joints." We will check this longitudinally.
- We should form/document our hypotheses regarding the concurrent (convergent?) validity items

- Avoid talking about power scales in paper
- PCA results show scale is unidimensional
- JoAnn will add frequencies of FACT-ES single item for every week.
- JoAnn and Ben will draft an analysis plan. We aim to have this before the May 9th meeting

- Discussed draft of introduction, including best way to motivate the validation study
- We may exclude the objective of creating a scoring algorithm
- We are going to go with the mean of pain, stiffness, and overall, and do all the validation assessments on all three means.
- Discussed distinction between analysis strategy and validating a measure
- We may give recommendations for analysis of the PRAI in the validation paper.
- Drafted research questions for validation paper.
- In the discussion, we will address research questions/analyses that we did not include, such as factor analysis.
- We will write up the research questions and send to the group for comments before next meeting, making them aware of the goal of a quick submission.

- Discussed study objectives/aims. Liana has drafted these.
- Refined the aims, combining validity items into one group
- Pointed out that we need to establish reliability in the paper before discussing validity
- Eliminated addressing the scale's dimensionality because of its nature as a power scale
- Liana will elaborate on the aims with sub aims and research questions. Then next week, we will discuss appropriate methods to address each

- Discussed flow chart. Want to show breakdown by the three groups
- Discussed table 1. Established that main comparison will be between the three groups. We will also make one for joint comorbidity for now.
*Also discussed including one by presence of inflammation, but later decided to treat this as an ordinal value.* - Discussed duration of morning stiffness. Decided to treat as ordinal rather than dichotomizing. Discussed methods of imputation for those who checked "greater than 120 minutes." For those who said "yes" they have morning stiffness, but did not indicate duration, imputation would not be a good idea unless we are doing a multivariable analysis.
- We are using the morning stiffness duration as a measure of inflammation. One of Liana's questions is in what percent of arthralgia is there inflammation?
- Discussed our analysis of relationship between pain and stiffness. Decided to address in discussion section.
- Joann and Ben will write the statistical portion of the methods.
*Work in progress* - Discussed refinements to Bland Altman plots. We notice that the limits of agreement are narrow. This is because the standard deviation of the differences is small, which is because of a very high frequency of zeros (when pain and stiffness were equal). While it appears that many points lie outside the limits, the majority lie within (can now differentiate density because of the jittering).
- Joann will:
- Make sure the analysis data she's working with is consistent in composition to that described in the flow chart.
- Work on making table 1s.
- Report on the distribution and missingness of the duration of morning stiffness. Need to include the 120 minute check box in the creation of this variable.
- Check trajectory graphs to make sure they are separated by joint comorbidity at baseline rather than time-varying.
- Make plot of means/medians of scores over time (by groups, jointcomorb)
- Test for difference at baseline in scores between those with and without joint comorbidities.

- Briefly discussed appropriate analysis for a power scale as opposed to an attitude/belief scale. Factor analysis is not appropriate for power scales. (?) Instead, you should demonstrate that the scale has a theorized relationship with some other variables.
- We narrowed the scope of this paper, deciding to consider item reduction and cluster analysis (?) in a subsequent paper.
- We will focus on known groups and construct validity as well as relationship between pain and stiffness in this paper.
- Need to demonstrate test-retest reliability in the control group using two assessments which are close in time. We think there might be a problem with the baseline measurements. Irene suggested using 2 randomly selected measurements. Would the same two time points be used for all patients?
- We reviewed Joann's plots
- Discussed some problems with analyzing percent change
- Another way to establish validity would be to look at physical function over time as a function of PRAI over time. We agreed that we would not want to adjust for any covariates in this model which aims to estimate the total effect rather than only the direct effect.
- In order to show construct/known groups validity, we should treat group as the per protocol value rather than the intent to treat value. That is, we should use a (possibly) time-varying indicator of whether the patient took an AI in the last interval.
- From the trajectories over time graphs, we noted that there were similar shapes in the pain graphs and the stiffness graphs.
- From the boxplots of difference between pain and stiffness within joint, we noted that pain was usually higher than stiffness when there was a difference, but there were cases where stiffness was greater than pain.
- From the Bland-Altman plots, we see that there is good agreement between pain and stiffness within each joint. We did see that for the left elbow, there was more disagreement for lower values and little disagreement for higher values.
- Liana will:
- work on introduction and outline

- JoAnn will:
- Show the trajectories graphs by joint comorbidities as well as by trt group. Could have the joint comorbidities shown with a different color.
- Label the x and y axes in the Bland-Altman plots and add confidence bands. Jitter points to visualize the density of points.
- Look at summary measures of aggreement.
- Get Wilcoxon rank-sum p-values for the joint comorbidity association with PRAI.
- Make new variable indicating whether the patient has taken AI most of the days since last observation. This info should be in chart. At baseline it is at1. The first three levels are AIs, and 4, 5 or missing should be set to no AI. (The non-AI group was not asked the question, so they have missing values.)
- Make new boxplots of PRAI by groups (per protocol) and morning stiffness. These would be separate for each time point. Can include group C for baseline.

- Want to look at plots of joints by joint comorbidity yes-no. Can average left and right for each joint and put all the joints' pain scores on one graph and the stiffness on another. I'm thinking boxplots. This is to just see if the data make sense. It would establish validity.
- Plot of pain data over time by group. Have we already done this?
- Hypothesis is that there is just one construct, arthralgia, rather than separate constructs for pain and stiffness.
- Compare the magnitude of pain and stiffness within the same joint.
- Run two CFA models. One with one factor, and the other with 2 factors, with all the pain items loading on to one of the factors and all the stiffness items loading on the other factor.
- Get correlations between pain and stiffness total scores at baseline.
- Make Bland-Altman plot for agreement between pain and stiffness for each joint.
- Get the ICC for each joint by running a mixed effects model for each joint, with a random intercept for person. Each person has two measurements, the pain and stiffness, for that joint. The ICC is estimated as the variance of the random intercept (between subject variance) divided by the total variance. In this model (random intercept), this is the rho = correlation between two measurements on the same subject.

- Discussed results of principal component analysis. Scree plots of pain and stiffness
**separately**indicate that there is only one latent variable, while the Kaiser method (eigenvalues > 1) indicate that there are more than one (about 4) latent constructs. I think we conceptualize the result with one construct makes more sense. - Ken will also try doing pca with all 32 items, and see if the pain and stiffness items fall separately into two latent variables.
- JoAnn will try to replicate the PCA
- Discussed criteria for dropping items. One possibility would be to drop items with very low means. For example, the means for the elbow questions were about 0.24. However, this could also wash out the effect of AIs on arthralgia if the questions we drop are the ones AI patients are differentially endorsing.
- Discussed whether shortening the instrument is within the scope of this paper.
- Can do confirmatory factor analysis to establish whether the pain and stiffness are two separate constructs or the same. Do a nested model. Confirmatory factor analysis specifies the model a priori. Two competing models.
- May want to repeat some of the cross-sectional analyses at a later time point, instead of just at baseline.

- Wolfe wanted us to have collected the Brief Pain Inventory
- Ken recommends against using pain interference in demonstrating concurrent validity. Should belong more with construct validity.
- Things to look at:
- Correlation between stiffness score and pain score
- Correlation pain and stiffness within each joint
- Difference in test-retest reliability between groups
- Could look at a variogram to show decreasing serial correlation over time (didn't discuss)
- Correlation between score and SES measure
- Correlation of each item with total score.

- Most important part of reliability is test-retest. Test-retest reliability is traditionally demonstrated in psychology by administering the measure twice around one week apart.
- Ken thinks we do not absolutely need to show discriminant validity.
- Another goal of the analysis/paper is to evaluate whether the PRAI could be shortened. It would be more likely to be adopted clinically if much shorter.
- Criteria for dropping items:
- Items which are vary rarely endorsed.
- Chronbach's alpha (of total or of a particular scale) without each item

- For factor analysis, we would be doing exploratory factor analysis.
- Ken's hypothesis is that the mean will have more variance (better distribution) than the maximum or top four items, and thus the mean will be a more reliable and valid summary score.
- Will read to learn more about item response theory (and scale analysis?) and whether it is applicable here.
- A good overall chronbach's alpha would be between 0.7 and 0.9.
- We will look at the baseline data only first, and then later observations.
- JoAnn will look into R packages for these analyses. Ken will run some preliminary report, and JoAnn will try to replicate.

- Next project is PRAI validation paper
- Reviewed different types of validity and reliability and whether they are applicable and possible to determine for the PRAI with the data from the BCAT study.
- Discussed possible collaborators

I | Attachment | Action | Size | Date | Who | Comment |
---|---|---|---|---|---|---|

docx | analysisPlan.07MAR2013.docx | manage | 16.2 K | 04 Apr 2013 - 16:49 | JoAnnAlvarez | Analysis plan March 7 2013 |

Edit | Attach | Print version | History: r10 < r9 < r8 < r7 | Backlinks | View wiki text | Edit wiki text | More topic actions

Topic revision: r10 - 14 Jun 2013, JoAnnAlvarez

Copyright © 2013-2022 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.

Ideas, requests, problems regarding Vanderbilt Biostatistics Wiki? Send feedback

Ideas, requests, problems regarding Vanderbilt Biostatistics Wiki? Send feedback