Chapter 7 Post-test Questionnaire- Descriptive and exploratory analysis
7.1 Experience and participation domain
The first part of the post-test questionnaire is about how the patient considers that was his participation in the study and if he think that the genomic testing was a positive experience.
These two questions do not show a high variability with regards to the answers but both are important to have an overview of how patients perceive the study and genomic testing. Therefore, they will be maintained.
7.2 Expectations and concerns domain
In the post-test questionnaire, the same items for expectations were included. Regarding concerns, the idea of the four questions was conserved but in this setting, the focus is not on the concern itself but if that worry or fear has become a reality. Again, there are 11 questions, but one (the seventh) has 5 items. Thus, there are a total of 15 items in this domain, 11 for expectations (from 1 to 7.v.) and 4 for concerns (8 to 11). All of them with the same likert scale.
7.2.1 Visualizing items and responses
From the first visual analysis, all items except the number 4 have a wide majority of agreement answers, with a higher percentage of totally agree answers. Then, items with less variability are 7.iii followed by 1, 2 and 7.v. The item 4, with an inverted design, have a 39% of agreement and a 26% of disagreement. Those with more variability are 6, 5, 3, 7.i, followed by 7.iv, and 7.ii in decreasing order.
With regards to concerns, there are variability in the four answers. The number 8 could have the lower variability with the higher percentage of agreement answers.
As second exploratory approach analysis, the difficulty and discrimination capacity is evaluate for each of these items.
## Scale for fill is already present.
## Adding another scale for fill, which will replace the existing scale.
## Scale for colour is already present.
## Adding another scale for colour, which will replace the existing scale.
The highest scores for discrimination are shown by items 7.v, 2, 1, and 7.i. On the other hand, item 4 shows the lower discrimination score followed by 3, 7.ii, and 7.iii.
## Scale for fill is already present.
## Adding another scale for fill, which will replace the existing scale.
## Scale for colour is already present.
## Adding another scale for colour, which will replace the existing scale.
Regarding discrimination and the concerns domain, item 11 shows the highest discrimination score, followed by 9, 10, and 8.
Partial conclusion:
Expectations.
Variability- Highest variability: Q6, Q5, Q3, and Q7.i.
Lowest variability: Q7.iii followed by Q1, Q2 and Q7.v.
Discrimination- Highest discrimination: Q7.v, Q2, Q1, and Q7.i.
Lowest discrimination: item Q4, followed by Q3, Q7.ii, and Q7.iii.
summary-
The best elements are: Q7.i, and Q7.v .
The worst two: Q7.iii and Q4, followed by item Q3 and Q7.ii. If item Q2 is selected item Q1 could be excluded.
Concerns.
Variability- All the items have a wide range of variability.
Discrimination- Item Q11 shows the highest discrimination score, while item Q8 shows the lowest.
7.2.2 Evaluating the reliability of questions (Cronbach’s α and Omega)
The Cronbach’s α and the Guttman’s lambda_6 (G6) are calculated for expectations and later for concerns.
7.2.2.1 Expectations
Regarding expectations items, the number 4 will be inverted.
First, analyzing the reliability of the whole subset of items, i.e., the consistency of the expectations section several measurements are display: the Cronbach’s α, with its IC, and the omega (ω) coefficient.
##
## 95% confidence boundaries (Feldt)
## lower alpha upper
## 0.54 0.73 0.86##
## Information about this analysis:
##
## Dataframe: post_test_Q_Exp_values[, 2:12]
## Items: all
## Observations: 22
## Positive correlations: 43 out of 55 (78%)
##
## Estimates assuming interval level:
##
## Omega (total): 0.89
## Omega (hierarchical): 0.68
## Revelle's omega (total): 0.89
## Greatest Lower Bound (GLB): NA
## Coefficient H: 1
## Coefficient alpha: 0.73
##
## (Estimates assuming ordinal level not computed, as the polychoric correlation matrix has missing values.)
##
## Note: the normal point estimate and confidence interval for omega are based on the procedure suggested by Dunn, Baguley & Brunsden (2013) using the MBESS function ci.reliability, whereas the psych package point estimate was suggested in Revelle & Zinbarg (2008). See the help ('?scaleStructure') for more information.Then, in order to inspect the role of each item, the r.cor and the Guttman’s Lambda 6 (G6) are explored.
Figure 7.1: Dashed line indicates the alpha (Feldt alpha) value for the whole set of items in order to compare this value if the alpha value that results from dropping the corresponding item (yellow bar).
| Items | G6 | raw_alpha_itemDropped | r.cor |
|---|---|---|---|
| 3.Estoy interesado/a en aprender más | 0.9117 | 0.7644 | 0.0999 |
| 4.Inverted- Necesito visita formal | 0.9063 | 0.7689 | 0.2441 |
| 1.Tenía suficiente conocimiento de beneficios y riesgos para tomar decisión informada | 0.8996 | 0.7118 | 0.4359 |
| 7ii.Cumplido expectativas sobre- El formato en que se me explicaron los resultados/informe escrito | 0.8940 | 0.6962 | 0.5599 |
| 7iii.Cumplido expectativas sobre- Que mi Dr pueda cambiar el tto de acuerdo a resultados | 0.8914 | 0.6830 | 0.7024 |
| 5.El resultado ayudará al control de mi cáncer | 0.8904 | 0.7282 | 0.3983 |
| 7v.Cumplido expectativas sobre- Poder recibir tratamientos experimentales | 0.8904 | 0.6759 | 0.7017 |
| 2.Recibí suficiente información para comprender beneficios y riesgos del análisis genómico | 0.8890 | 0.7114 | 0.4752 |
| 6.El resultado ayudará a aumentar mi expectativa vida | 0.8853 | 0.7185 | 0.4653 |
| 7iv.Cumplido expectativas sobre- Opciones de tratamiento adicionales | 0.8837 | 0.6697 | 0.7375 |
| 7i.Cumplido expectativas sobre- La explicación que recibí de resultados e implicación para mi salud | 0.8604 | 0.6313 | 0.9236 |
G6 values are very close altogether having more variability the correlation coefficient as well as the alpha value whrn the item is dropped. Thus, the best combination between these scores can be seing for items 7.iii and 7.v, 7.iv and 7.i. The worst combinations are for items 3, 4, 5 and 1.
7.2.2.2 Concerns
Regarding concerns, the last 4 questions are focus on this topic.
First, analyzing the reliability of the whole subset of items, i.e., the consistency of the concerns section several measurements are display: the Cronbach’s α, with its IC, and the omega (ω) coefficient.
##
## 95% confidence boundaries (Feldt)
## lower alpha upper
## 0.46 0.71 0.86##
## Information about this analysis:
##
## Dataframe: post_test_Q_Concerns_values
## Items: all
## Observations: 23
## Positive correlations: 6 out of 6 (100%)
##
## Estimates assuming interval level:
##
## Omega (total): 0.13
## Omega (hierarchical): 0.04
## Revelle's omega (total): 0.13
## Greatest Lower Bound (GLB): NA
## Coefficient H: 0.81
## Coefficient alpha: 0.72
##
## (Estimates assuming ordinal level not computed, as the polychoric correlation matrix has missing values.)
##
## Note: the normal point estimate and confidence interval for omega are based on the procedure suggested by Dunn, Baguley & Brunsden (2013) using the MBESS function ci.reliability, whereas the psych package point estimate was suggested in Revelle & Zinbarg (2008). See the help ('?scaleStructure') for more information.While the alpha coefficient shows a value above 0.7, the omega coefficients, both total and hierarchical, show extremely lower values. Then, in order to inspect the role of each item, the r.cor and the Guttman’s Lambda 6 (G6) are explored.
Figure 7.2: Dashed line indicates the alpha (Feldt alpha) value for the whole set of items in order to compare this value if the alpha value that results from dropping the corresponding item (yellow bar).
## Items
## 1 8.Creo que los resultados del análisis genómico guiarán mi tto
## 2 10.Los resultados han aportado información acerca del riesgo de otras condiciones que prefiriría no haber sabido
## 3 11.Los resultados me han preocupado o generado ansiedad
## 4 9.Los resultados han sido confusos o difíciles de comprender
## G6 raw_alpha_itemDropped r.cor
## 1 0.7271 0.7966 0.1815
## 2 0.5144 0.5724 0.7093
## 3 0.5061 0.5638 0.7205
## 4 0.4915 0.5591 0.7341Similarly to what was found in the pre-test questionnaire, the G6 and the correlation are in opposite directions. If both measurements are considered with the same importance, one strategy could be to choose those two items in the middle; i.e., with the best combination of both scores (item 10 and 11).
7.2.3 Global conclusions about expectations domain
Considering the aim of the analysis of this pilot study the aim is to identify the final set of items that will be chosen for the next step. Considering the fact that the post test questionnaire has less number of items, it is not expected to eliminate additional items from this post test questionnaire. However, each item was analysed to explore their performance. Finally, the main goal is to determine whether the items selected at the pre-test are still useful in the post-test questionnaire and vice versa, if those chosen to be excluded at the pre-test are not clearly relevant in the post test tool.
The results for the post-genomic test are summarized below.
Summary variability and discrimination-
The best elements are: Q7.i, and Q7.v.
The worst two: Q7.iii and Q4, maybe considering also item Q3 and Q7.ii.
Summary G6, correlation, and alpha when the item is dropped-
The best items: Q7.iii and Q7.v, followed by Q7.iv and Q7.i.
The worst items Q3, Q4, Q5 and Q1.
Potential items to be excluded according these analysis: Q4, Q3, Q1, Q7.ii, and Q7.iii.
Regarding concerns, item Q11 seems to be relevant. Item Q8 showed the lowest discrimination score. In the factor analysis, item Q8 was observed overlapped with item Q6. On the other hand, item Q9 did not outperform any item in any approach.
7.3 Attitudes domain
This domain includes 5 items, all focused on the attitudes and reliability of the test itself. All the items oriented to the motivations are not included in this stage. Therefore, all these 5 are similar to the previous ones in the pre-test questionnaire with a Likert scale.
7.3.1 Visualizing items and responses
Analyzing the variability between answers, items 4 and 1 show the lowest variability. For the remaining items there are considerable answer´s variability.
Secondly, the difficulty and discrimination capacity is evaluated for each of these items.
## Scale for fill is already present.
## Adding another scale for fill, which will replace the existing scale.
## Scale for colour is already present.
## Adding another scale for colour, which will replace the existing scale.
The is negative discrimination score value for item 5. Then, the lowest values are shown by items 4 and 1.
Partial conclusion:
According to these results, Q5 showed the worst discrimination score. However, it has high variability. Moreover, item Q1 will be finally excluded from the questionnaire.
7.3.2 Evaluating the reliability of questions (Cronbach’s α and Omega)
The Cronbach’s α and the Guttman’s lambda_6 (G6) are calculated for the attitude domain.
##
## 95% confidence boundaries (Feldt)
## lower alpha upper
## 0.55 0.75 0.88##
## Information about this analysis:
##
## Dataframe: post_test_Q_Actitd_values[, 2:6]
## Items: all
## Observations: 22
## Positive correlations: 10 out of 10 (100%)
##
## Estimates assuming interval level:
##
## Omega (total): 0.9
## Omega (hierarchical): 0.56
## Revelle's omega (total): 0.9
## Greatest Lower Bound (GLB): NA
## Coefficient H: 1
## Coefficient alpha: 0.76
##
## (Estimates assuming ordinal level not computed, as the polychoric correlation matrix has missing values.)
##
## Note: the normal point estimate and confidence interval for omega are based on the procedure suggested by Dunn, Baguley & Brunsden (2013) using the MBESS function ci.reliability, whereas the psych package point estimate was suggested in Revelle & Zinbarg (2008). See the help ('?scaleStructure') for more information.Then, in order to inspect the role of each item, the r.cor and the Guttman’s Lambda 6 (G6) are explored.
Figure 7.3: Dashed line indicates the alpha (Feldt alpha) value for the whole set of items in order to compare this value if the alpha value that results from dropping the corresponding item (yellow bar).
| Items | G6 | raw_alpha_itemDropped | r.cor |
|---|---|---|---|
| 4.El análisis genómico parece algo beneficioso para mi. | 0.8543 | 0.7747 | 0.3099 |
| 5.Inverted- El análisis genómico parece ser una prueba imprecisa. | 0.8373 | 0.8059 | 0.3068 |
| 1.Dispuesto a que me realicen un procedimiento POCO INVASIVO. | 0.7605 | 0.6953 | 0.6611 |
| 3.Dispuesto a que me realicen un procedimiento médico MAYOR. | 0.6150 | 0.5595 | 0.9386 |
| 2.Dispuesto a que me realicen un procedimiento médico MENOR. | 0.5690 | 0.5794 | 0.9370 |
Partial conclusion: G6 and correlation- In this domain, results are different depending where the focus is posed. While Q3 and Q2 have the best correlation (highest) and alpha values (lowest), they show the lowest G6. Besides, items 4 and 5 show the opposite behavior.
7.4 Expectations’ fulfillment
In this final step, the accomplishment of expectations is described through two additional items. The first one, a multiple choice question; and the second one, a Visual Analogue Scale.
The first item asked about which of the following personal objectives the patient fulfilled after this process. For this item, patients can select more than one option.
Then, the results from the Visual Analogue Scale tool are depicted.
| n | missing | Minimun | Maximun | Mean | DS | Median | FirstQ | ThirdQ | IQR |
|---|---|---|---|---|---|---|---|---|---|
| 25 | 6 | 0 | 100 | 70.68 | 35.72 | 83 | 65 | 100 | 35 |
