Using ready4 modules

• 1: Modules for modelling people
• 1.1: Add metadata to datasets of individual human records
• 1.2: Validate variable total scores
• 1.3: Score health utility
• 1.4: Explore candidate utility mapping models
• 1.5: Implement a utility mapping study
• 1.6: Find and deploy utility mapping models
• 1.7: Use utility mapping algorithms to help implement cost-utility analyses
• 1.8: Develop choice models
• 2: Modules for modelling places
• 3: Modules for modelling platforms
• 4: Modules for modelling programs

1 - Modules for modelling people

1.1 - Add metadata to datasets of individual human records

Note: This vignette is illustrated with fake data. The dataset explored in this example should not be used to inform decision-making.

library(ready4)
library(youthvars)

Youthvars provides a two classes - YouthvarsProfile and YouthvarsSeries that are useful for describing features of datasets. The tools in youthvars build on the metadata included in a Ready4useDyad.

Ingest data

To start we ingest X, a Ready4useDyad (dataset and data dictionary pair) that we can download from a remote repository.

X <- ready4use::Ready4useRepos(dv_nm_1L_chr = "fakes",
                               dv_ds_nm_1L_chr = "https://doi.org/10.7910/DVN/W95KED",
                               dv_server_1L_chr = "dataverse.harvard.edu") %>%
  ingest(fls_to_ingest_chr = "ymh_clinical_dyad_r4",
         metadata_1L_lgl = F)

Add metadata

We could add metadata about X, such as the unique identifier variable name, by transforming it to a YouthvarsProfile instance.

## Not run
# X <- YouthvarsProfile(a_Ready4useDyad = X,
#                       id_var_nm_1L_chr = "fkClientID")

However, in this case the data we ingested includes a longitudinal dataset. It is therefore preferable to transform X into a YouthvarsSeries instance. YouthvarsSeries objects contain all of the fields of YouthvarsProfile objects, but also include additional fields that are specific for longitudinal datasets (e.g. timepoint_var_nm_1L_chr and timepoint_vals_chr that respectively specify the data-collection timepoint variable name and values and participation_var_1L_chr that specifies the desired name of a yet to be created variable that will summarise the data-collection timepoints for which each unit record supplied data).

X <- YouthvarsSeries(a_Ready4useDyad = X,
                     id_var_nm_1L_chr = "fkClientID",
                     participation_var_1L_chr = "participation",
                     timepoint_vals_chr = c("Baseline","Follow-up"),
                     timepoint_var_nm_1L_chr = "round")

YouthvarsSeries methods

Currently, only methods for YouthvarsSeries (and not yet YouthvarsProfile) have been included with the youthvars package. These methods are summarised in the following sections.

Validate data

We use the ratify method to ensure that X has been appropriately configured for methods examining datasets reporting measures at two timepoints.

X <- ratify(X,
            type_1L_chr = "two_timepoints")

Inspect data

We can now specify the variables that we would like to prepare descriptive statistics for using the renewSlot and renew methods. The variables to be profiled are specified in arguments beginning with “compare_”. Use compare_ptcpn_chr to compare variables based on whether cases reported data at one or both timepoints and compare_by_time_chr to compare the summary statistics of variables by timepoints, e.g at baseline and follow-up. If you wish these comparisons to report p values, then use the compare_ptcpn_with_test_chr and compare_by_time_with_test_chr arguments.

X <- renewSlot(X,
               "descriptives_ls",
               compare_by_time_chr = c("d_age","d_sexual_ori_s","d_studying_working"),
               compare_by_time_with_test_chr = c("k6_total", "phq9_total", "bads_total"),
               compare_ptcpn_with_test_chr = c("k6_total", "phq9_total", "bads_total")) %>%
  renew(type_1L_chr = "characterize")

The tables generated in the preceding step can be inspected using the exhibit method.

X %>%
  exhibit(profile_idx_int = 1L,
          scroll_box_args_ls = list(width = "100%"))

X %>%
  exhibit(profile_idx_int = 2L,
          scroll_box_args_ls = list(width = "100%"))

X %>%
  exhibit(profile_idx_int = 3L,
          scroll_box_args_ls = list(width = "100%"))

The depict method can create plots, comparing numeric variables by timepoint.

depict(X,
       type_1L_chr = "by_time",
       var_nms_chr = c("c_sofas"),
       label_fill_1L_chr = "Time",#
       labels_chr = c("SOFAS"),#
       y_label_1L_chr = "")
#> Warning: `stat(width * density)` was deprecated in ggplot2 3.4.0.
#> ℹ Please use `after_stat(width * density)` instead.
#> ℹ The deprecated feature was likely used in the youthvars package.
#>   Please report the issue to the authors.

SOFAS total scores by data collection round

Share data

If and only if the dataset you are working with is appropriate for public dissemination (e.g. is synthetic data), you can use the following workflow for sharing it. We can share the dataset we created for this example using the share method, specifying the repository to which we wish to publish the dataset (and for which we have write permissions) in a (Ready4useRepos object).

Y <- Ready4useRepos(gh_repo_1L_chr = "ready4-dev/youthvars", # Replace with your repository 
                          gh_tag_1L_chr = "Documentation_0.0"), # (need write permissions).
Y <- share(Y,
           obj_to_share_xx = X,
           fl_nm_1L_chr = "ymh_YouthvarsSeries")

X is now available for download as the file ymh_YouthvarsSeries.RDS from the “Documentation_0.0” release of the youthvars package.

1.2 - Validate variable total scores

library(ready4)
library(youthvars)

Variable classes and data integrity

The youthvars package defines a number of vector based classes that can be used to quality assure the data recorded for individual variables. youthvars variable classes are potentially useful for:

1. facilitating automated data integrity checks that verify no impermissible values (e.g. utility scores greater than one) are present in source data, transformed data or results; and
2. automating the selection of the appropriate method to apply to each data type.

Included classes

The initial set of classes included in the youthvars package are one class for Assessment of Quality of Life (Adolescent) health utility and one for each of the predictors used in the utility prediction algorithms included in the related youthu package.

Assessment of Quality of Life Six Dimension (Adolescent) Health Utility

The youthvars_aqol6d_adol class is defined for numeric vectors with a minimum value of 0.03 and maximum value of 1.0.

youthvars_aqol6d_adol(0.4)
#> [1] 0.4
#> attr(,"class")
#> [1] "youthvars_aqol6d_adol"
#> [2] "numeric"

youthvars_aqol6d_adol(c(0.03,0.2,1))
#> [1] 0.03 0.20 1.00
#> attr(,"class")
#> [1] "youthvars_aqol6d_adol"
#> [2] "numeric"

Non numeric objects and values outside these ranges will produce errors.

youthvars_aqol6d_adol("0.5")
#> Error in make_new_youthvars_aqol6d_adol(x): is.numeric(x) is not TRUE

youthvars_aqol6d_adol(-0.1)
#> Error: All non-missing values in valid youthvars_aqol6d_adol object must be greater than or equal to 0.03.

youthvars_aqol6d_adol(1.2)
#> Error: All non-missing values in valid youthvars_aqol6d_adol object must be less than or equal to 1.

Behavioural Activation for Depression Scale (BADS)

The youthvars_bads class is defined for integer vectors with a minimum value of 0 and maximum value of 150.

youthvars_bads(143L)
#> [1] 143
#> attr(,"class")
#> [1] "youthvars_bads" "integer"

youthvars_bads(as.integer(c(1,15,150)))
#> [1]   1  15 150
#> attr(,"class")
#> [1] "youthvars_bads" "integer"

Non-integers and values outside these ranges will produce errors.

youthvars_bads(22.5)
#> Error in make_new_youthvars_bads(x): is.integer(x) is not TRUE

youthvars_bads(-1L)
#> Error: All non-missing values in valid youthvars_bads object must be greater than or equal to 0.

youthvars_bads(160L)
#> Error: All non-missing values in valid youthvars_bads object must be less than or equal to 150.

Generalised Anxiety Disorder Scale (GAD-7)

The youthvars_gad7 class is defined for integer vectors with a minimum value of 0 and a maximum value of 21.

youthvars_gad7(15L)
#> [1] 15
#> attr(,"class")
#> [1] "youthvars_gad7" "integer"

youthvars_gad7(as.integer(c(0,14,21)))
#> [1]  0 14 21
#> attr(,"class")
#> [1] "youthvars_gad7" "integer"

Non-integers and values outside these ranges will produce errors.

youthvars_gad7(14.6)
#> Error in make_new_youthvars_gad7(x): is.integer(x) is not TRUE

youthvars_gad7(-1L)
#> Error: All non-missing values in valid youthvars_gad7 object must be greater than or equal to 0.

youthvars_gad7(22L)
#> Error: All non-missing values in valid youthvars_gad7 object must be less than or equal to 21.

Kessler Psychological Distress Scale (K6) - US Scoring System

The youthvars_k6 class is defined for integer vectors with a minimum value of 0 and a maximum value of 24.

youthvars_k6(21L)
#> [1] 21
#> attr(,"class")
#> [1] "youthvars_k6" "integer"

youthvars_k6(as.integer(c(0,13,24)))
#> [1]  0 13 24
#> attr(,"class")
#> [1] "youthvars_k6" "integer"

Non-integers and values outside these ranges will produce errors.

youthvars_k6(11.2)
#> Error in make_new_youthvars_k6(x): is.integer(x) is not TRUE

youthvars_k6(-1L)
#> Error: All non-missing values in valid youthvars_k6 object must be greater than or equal to 0.

youthvars_k6(25L)
#> Error: All non-missing values in valid youthvars_k6 object must be less than or equal to 24.

Overall Anxiety Severity and Impairment Scale (OASIS)

The youthvars_oasis class is defined for integer vectors with a minimum value of 0 and a maximum value of 20.

youthvars_oasis(15L)
#> [1] 15
#> attr(,"class")
#> [1] "youthvars_oasis" "integer"

youthvars_oasis(as.integer(c(0,12,20)))
#> [1]  0 12 20
#> attr(,"class")
#> [1] "youthvars_oasis" "integer"

Non-integers and values outside these ranges will produce errors.

youthvars_oasis(14.2)
#> Error in make_new_youthvars_oasis(x): is.integer(x) is not TRUE

youthvars_oasis(-1L)
#> Error: All non-missing values in valid youthvars_oasis object must be greater than or equal to 0.

youthvars_oasis(21L)
#> Error: All non-missing values in valid youthvars_oasis object must be less than or equal to 20.

Patient Health Questionnaire (PHQ-9)

The youthvars_phq9 class is defined for integer vectors with a minimum value of 0 and a maximum value of 27.

youthvars_phq9(11L)
#> [1] 11
#> attr(,"class")
#> [1] "youthvars_phq9" "integer"

youthvars_phq9(as.integer(c(0,13,27)))
#> [1]  0 13 27
#> attr(,"class")
#> [1] "youthvars_phq9" "integer"

Non-integers and values outside these ranges will produce errors.

youthvars_phq9(15.2)
#> Error in make_new_youthvars_phq9(x): is.integer(x) is not TRUE

youthvars_phq9(-1L)
#> Error: All non-missing values in valid youthvars_phq9 object must be greater than or equal to 0.

youthvars_phq9(28L)
#> Error: All non-missing values in valid youthvars_phq9 object must be less than or equal to 27.

Screen for Child Anxiety Related Disorders (SCARED)

The youthvars_scared class is defined for integer vectors with a minimum value of 0 and a maximum value of 82.

youthvars_scared(77L)
#> [1] 77
#> attr(,"class")
#> [1] "youthvars_scared" "integer"

youthvars_scared(as.integer(c(0,42,82)))
#> [1]  0 42 82
#> attr(,"class")
#> [1] "youthvars_scared" "integer"

Non-integers and values outside these ranges will produce errors.

youthvars_scared(33.2)
#> Error in make_new_youthvars_scared(x): is.integer(x) is not TRUE

youthvars_scared(-1L)
#> Error: All non-missing values in valid youthvars_scared object must be greater than or equal to 0.

youthvars_scared(83)
#> Error in make_new_youthvars_scared(x): is.integer(x) is not TRUE

Social and Occupational Functioning Assessment Scale (SOFAS)

The youthvars_sofas class is defined for integer vectors with a minimum value of 0 and a maximum value of 100.

youthvars_sofas(44L)
#> [1] 44
#> attr(,"class")
#> [1] "youthvars_sofas" "integer"

youthvars_sofas(as.integer(c(0,23,89)))
#> [1]  0 23 89
#> attr(,"class")
#> [1] "youthvars_sofas" "integer"

Non-integers and values outside these ranges will produce errors.

youthvars_sofas(73.2)
#> Error in make_new_youthvars_sofas(x): is.integer(x) is not TRUE

youthvars_sofas(-1L)
#> Error: All non-missing values in valid youthvars_sofas object must be greater than or equal to 0.

youthvars_sofas(103L)
#> Error: All non-missing values in valid youthvars_sofas object must be less than or equal to 100.

1.3 - Score health utility

Note: This vignette is illustrated with fake data. The dataset explored in this example should not be used to inform decision-making. Some of the methods illustrated in this AQoL-6D vignette can also be used to score other health utility instruments - see a vignette about scoring EQ-5D.

library(ready4)
library(scorz)

AQoL-6D scoring

To derive a health utility score from the raw responses to a multi-attribute utility instrument it is necessary to implement a scoring algorithm. Scoring algorithms for the Assessment of Quality of Life Six Dimension (AQoL-6D) are publicly available in SPSS format (https://www.aqol.com.au/index.php/scoring-algorithms).

However, to include scoring algorithms in reproducible research workflows, it is desirable to have these algorithms available in open science languages such as R. We therefore developed an R implementation of the adult and adolescent versions of the AQoL-6D scoring algorithms and have made them available as part of the scorz package.

Ingest data

To begin, we ingest an unscored dataset as an instance of the Ready4useDyad class (from the ready4use package). In this case we download our data from a remote repository.

X <- ready4use::Ready4useRepos(dv_nm_1L_chr = "fakes",
                               dv_ds_nm_1L_chr = "https://doi.org/10.7910/DVN/W95KED",
                               dv_server_1L_chr = "dataverse.harvard.edu") %>%
  ingest(fls_to_ingest_chr = "ymh_clinical_dyad_r4",
         metadata_1L_lgl = F) 

To make the ingested dataset easier to interpret, we can add labels from the dictionary.

X <- X %>%
  renew(type_1L_chr = "label")

We can now inspect our ingested dataset using the exhibit method.

exhibit(X,
        display_1L_chr = "head",
         scroll_box_args_ls = list(width = "100%"))

We now add meta-data that identifies our dataset as being longitudinal using the YouthvarsSeries class of the youthvars package.

X <- youthvars::YouthvarsSeries(a_Ready4useDyad = X,
                                id_var_nm_1L_chr = "fkClientID",
                                timepoint_var_nm_1L_chr = "round",
                                timepoint_vals_chr = levels(X@ds_tb$round))

We now use the data and meta-data we have created in the previous steps to create an instance of the ScorzAqol6Adol class. This class is specifically designed to facilitate scoring of the adolescent version of the AQoL-6D instrument.

Y <- ScorzAqol6Adol(a_YouthvarsProfile = X)

By default, instances of the ScorzAqol6Adol class are created with a slot specifying a value for the prefix for AQoL-6D questionnaire item responses.

procureSlot(Y,
            slot_nm_1L_chr = "itm_prefix_1L_chr")
#> [1] "aqol6d_q"

If this default value needs to be updated to match the prefix used in your dataset, use the renewSlot method.

# Not run
# Y <- renewSlot(Y, slot_nm_1L_chr = "itm_prefix_1L_chr", new_val_xx = "new_prefix")

Calculating scores

To calculate AQoL 6D adolescent utility scores, use the renew method.

Y <- renew(Y)

Viewing the updated dataset

We can inspect our updated dataset using the exhibit method. We can see that the updated dataset now has additional variables that include the intermediate and final calculations for AQoL-6D adolescent utility scores.

exhibit(Y,
        display_1L_chr = "head",
         scroll_box_args_ls = list(width = "100%"))

Creating summary plots

To create plots, we use the depict method.

We can create a list of summary plots by timepoint for all individual items.

plot_ls <- depict(Y, type_1L_chr = "item_by_time")

We can then select a desired item’s summary plot by using its index number.

plot_ls[[1]]

AQoL-6D Item 1 scores by data-collection round

Alternatively, we can generate individual plots by passing the item index number to the var_idcs_int argument of depict.

depict(Y, type_1L_chr = "item_by_time", var_idcs_int = 2L)

AQoL-6D Item 2 scores by data-collection round

We can also plot domain scores by time.

depict(Y, type_1L_chr = "domain_by_time", var_idcs_int = 1L)

AQoL-6D Independet Living Domain weighted scores by data-collection round

Total AQoL-6D scores can also be plotted using the same approach, where var_idcs_int = 1L is used to plot the weighted total distribution and var_idcs_int = 2L is used for plotting the unweighted total.

depict(Y, type_1L_chr = "total_by_time", var_idcs_int = 1L)

AQoL-6D item total weighted scores by data-collection round

Composite plots can be generated as well, though these are not currently optimised to reliably produce quality plots suitable for publication.

depict(Y, type_1L_chr = "comp_item_by_time")

AQoL-6D item responses by data-collection round

depict(Y, type_1L_chr = "comp_domain_by_time")

AQoL-6D weighted domain scores by data-collection round

Share output

We can now publicly share our scored dataset and its associated metadata, using Ready4useRepos and its share method as described in a vignette from the ready4use package.

Z <- ready4use::Ready4useRepos(gh_repo_1L_chr = "ready4-dev/scorz", # Replace with details of your repo.
                               gh_tag_1L_chr = "Documentation_0.0") # You must have write permissions.
Z <- share(Z,
           obj_to_share_xx = Y,
           fl_nm_1L_chr = "ymh_ScorzAqol6Adol")

Y is now available for download as the file ymh_ScorzAqol6Adol.RDS from the “Documentation_0.0” release of the scorz package.

1.4 - Explore candidate utility mapping models

Note: This vignette uses fake data - it is for illustrative purposes only and should not be used to inform decision making.

The steps in this exploratory analysis workflow may need to be performed iteratively, both in order to identify the optimal model types, predictors and covariates to use and modify default values to ensure model convergence.

library(ready4)
library(ready4show)
library(ready4use)
library(scorz)
library(betareg)
library(specific)

Import data

We start by ingesting our data. As this example uses EQ-5D data, we import a ScorzEuroQol5 ready4 framework module (created using the steps described in this vignette from the scorz pacakge) into a SpecificConverter Module and then apply the metamorphose method to convert it into a SpecificModel module.

X <- SpecificConverter(a_ScorzProfile = ready4use::Ready4useRepos(gh_repo_1L_chr = "ready4-dev/scorz", 
                                                                  gh_tag_1L_chr = "Documentation_0.0") %>%
                         ingest(fls_to_ingest_chr = "ymh_ScorzEuroQol5",
                                metadata_1L_lgl = F)) %>%
  metamorphose() 

class(X)
#> [1] "SpecificModels"
#> attr(,"package")
#> [1] "specific"

Inspect data

The dataset we are using has a total of 1786 records at two timepoints on 1068 study participants. The first six records are reproduced below.

To source dataset of X is contained in the a_YouthvarsProfile slot and is a YouthvarsSeries module. For more information about methods that can be used to explore this dataset, read this vignette from the youthvars package.

Specify parameters

In preparation for exploring our dataset, we need to declare a set of model parameters in a b_SpecificParameters slot of X. This can be done in one step, or in sequential steps. In this example, we will proceed sequentially.

Dependent variable

The dependent variable (total EQ-5D utility score) has already been specified when we imported the data from the ScorzEuroQol5 module.

procureSlot(X,
            "b_SpecificParameters@depnt_var_nm_1L_chr")
#> [1] "eq5d_total_w"

We can now add details of the allowable range of dependent variable values.

X <- renewSlot(X,
               "b_SpecificParameters@depnt_var_min_max_dbl",
               c(-1,1))

Candidate predictors

We can now specify the names of candidate predictor variables.

X <- renewSlot(X,
               "b_SpecificParameters@candidate_predrs_chr",
               new_val_xx = c("K10_int","Psych_well_int")) 

We next add meta-data about each candidate predictor variable in the form of a specific_predictors object.

X <- renewSlot(X, 
               "b_SpecificParameters@predictors_lup", 
               short_name_chr = c("K10_int","Psych_well_int"),
               long_name_chr = c("Kessler Psychological Distress - 10 Item Total Score",
                                 "Overall Wellbeing Measure (Winefield et al. 2012)"),
               min_val_dbl = c(10,18),
               max_val_dbl = c(50,90),
               class_chr = "integer",
               increment_dbl = 1,
               class_fn_chr = "as.integer",
               mdl_scaling_dbl = 0.01,
               covariate_lgl = F)

The specific_predictors object that we have added to X can be inspected using the exhibitSlot method.

exhibitSlot(X,
            "b_SpecificParameters@predictors_lup",
            scroll_box_args_ls = list(width = "100%"))

Covariates

We also specify the covariates that we aim to explore in conjunction with each candidate predictor.

X <- renewSlot(X, 
               "b_SpecificParameters@candidate_covars_chr",
               new_val_xx = c("d_sex_birth_s", "d_age",  "d_sexual_ori_s", "d_studying_working"))

Descriptive variables

We also specify variables that we will use for generating descriptive statistics about the dataset.

X <- renewSlot(X,
               "b_SpecificParameters@descv_var_nms_chr",
               c("d_age","Gender","d_relation_s",
                 "d_sexual_ori_s", "Region", "d_studying_working")) 

Temporal variables

The name of the dataset variable for data collection timepoint and all of its unique values were imported when converting the ScorzEuroQol5 module.

procureSlot(X,"a_YouthvarsProfile@timepoint_var_nm_1L_chr")
#> [1] "Timepoint"

procureSlot(X,"a_YouthvarsProfile@timepoint_vals_chr")
#> [1] "BL"  "FUP"

However, we also need to specify the name of the variable that contains the datestamp for each dataset record.

X <- renewSlot(X,
               "b_SpecificParameters@msrmnt_date_var_nm_1L_chr",
               "data_collection_dtm")

Candidate models

X was created with a default set of candidate models, stored as a specific_models sub-module, which can be inspected using the exhibitSlot method.

exhibitSlot(X,
            "b_SpecificParameters@candidate_mdls_lup",
            scroll_box_args_ls = list(width = "100%"))

We can choose to select just a subset of these to explore using the renewSlot method. As this is an illustrative example, we have restricted the models we will explore to just four types, passing the relevant row numbers to the slice_indcs_int argument.

X <- renewSlot(X,
               "b_SpecificParameters@candidate_mdls_lup",
               slice_indcs_int = c(1L,5L,7L,8L))

Other parameters

Depending on the type of analysis we plan on undertaking, we can also specify parameters such as the number of folds to use in cross validation, the maximum number of model runs to allow and a seed to ensure reproducibility of results. In this case we are going to use the default values generated when we first created X.

procureSlot(X,
            "b_SpecificParameters@folds_1L_int")
#> [1] 10

procureSlot(X,
            "b_SpecificParameters@max_mdl_runs_1L_int")
#> [1] 300

procureSlot(X,
            "b_SpecificParameters@seed_1L_int")
#> [1] 1234

Model testing

Before we start to use the data stored in X to undertake modelling, we must first validate that it contains all necessary (and internally consistent) data by using the ratify method. The call to ratify will update any variable names that are likely to cause problems when generating reports (e.g. through inclusion of characters like “_” in the variable name that can cause problems when rendering LaTeX documents).

X <- ratify(X)

Set-up workspace

We add details of the directory to which we will write all output. In this example we create a temporary directory (tempdir()), but in practice this would be an existing directory on your local machine.

X <- renewSlot(X,
               "paths_chr",
               tempdir())

It can be useful to save fake data (useful for demonstrating the generalisability and replicability of an analysis) and real data (required for write-up and reproducibility) is distinctly labelled directories. By default, X is created with a flag to save all output in a sub-directory “Real”. As we are using fake data, we can override this value.

X <- renewSlot(X,
               "b_SpecificParameters@fake_1L_lgl",
               T)

We can now write a number of sub-directories to our specified output directory.

X <- author(X,
            what_1L_chr = "workspace")

Descriptives

The first set of outputs we write to our output directories is a set of descriptive tables and plots.

X <- author(X,
            what_1L_chr = "descriptives",
            digits_1L_int = 3L)

Model comparisons

The investigate method can now be used to compare the candidate models we have specified earlier. In so doing it will transform X into a SpecificPredictors object.

X <- investigate(X,
                 depnt_var_max_val_1L_dbl = 0.99,
                 session_ls = sessionInfo())

class(X)
#> [1] "SpecificPredictors"
#> attr(,"package")
#> [1] "specific"

The investigate method will write each model to be tested to a new sub-directory of our output directory.

The investigate method also outputs a table summarising the performance of each of the candidate models.

exhibit(X,
        what_1L_chr = "mdl_cmprsn",
        type_1L_chr = "results"
        ) 

We can now identify the highest performing model in each category of candidate model based on the testing R² statistic.

procure(X,
        what_1L_chr = "prefd_mdls") # Fix for NA_ returns (one option within ctg)
#> [1] "BET_LGT" "OLS_NTF"

We can override these automated selections and instead incorporate other considerations (possibly based on judgments informed by visual inspection of the plots and the desirability of constraining predictions to a maximum value of one). We do this in the following command, specifying new preferred model types, in descending order of preference.

X <- renew(X,
           new_val_xx = c("BET_LGT", "OLS_CLL"),
           type_1L_chr = "results",
           what_1L_chr = "prefd_mdls")

Use most preferred model to compare all candidate predictors

We can now compare all of our candidate predictors (with and without candidate covariates) using the most preferred model type.

X <- investigate(X)

class(X)
#> [1] "SpecificFixed"
#> attr(,"package")
#> [1] "specific"

Now, we compare the performance of single predictor models of our preferred model type (in our case, a Beta Regression Model with Binomial distribution and logit link) for each candidate predictor. The last call to the investigate saved the tested models along with model plots in a sub-directory of our output directory. These results are also viewable as a table.

exhibit(X,
        what_1L_chr = "predr_cmprsn",
        type_1L_chr = "results",
        scroll_box_args_ls = list(width = "100%"))

The most recent call to the investigate method also saved single predictor R model objects (one for each candidate predictors) along with the two plots for each model in a sub-directory of our output directory. The performance of each single predictor model can also be summarised in a table.

exhibit(X,
        type_1L_chr = "results",
        what_1L_chr = "fxd_sngl_cmprsn")

Updated versions of each of the models in the previous step (this time with covariates added) are saved to a new subdirectory of the output directory and we can summarise the performance of each of the updated models, along with all signficant model terms, in a table.

exhibit(X,
        type_1L_chr = "results",
        what_1L_chr = "fxd_full_cmprsn",
        scroll_box_args_ls = list(width = "100%"))

We can now identify which, if any, of the candidate covariates we previously specified are significant predictors in any of the models.

procure(X,
        type_1L_chr = "results",
        what_1L_chr = "signt_covars")
#> [1] NA

We can override the covariates to select, potentially because we want to select only covariates that are significant for all or most of the models. However, in the below example we have opted not to do so and continue to use no covariates as selected by the algorithm in the previous step.

# X <- renew(X,
#             new_val_xx = c("COVARIATE OF YOUR CHOICE", "ANOTHER COVARIATE"),
#                                               type_1L_chr = "results",
#                                   what_1L_chr = "prefd_covars")

Test preferred model with preferred covariates for each candidate predictor

We now conclude our model testing by rerunning the previous step, except confining our covariates to those we prefer.

X <- investigate(X)

class(X)
#> [1] "SpecificMixed"
#> attr(,"package")
#> [1] "specific"

The previous call to the write_mdls_with_covars_cmprsn function saves the tested models along with two plots for each model in the “E_Predrs_W_Covars_Sngl_Mdl_Cmprsn” sub-directory of “Output”.

Apply preferred model types and predictors to longitudinal data

The next main step is to use the preferred model types and covariates identified from the preceding analysis of cross-sectional data in longitudinal analysis.

Longitudinal mixed modelling

Prior to undertaking longitudinal mixed modelling, we need to check the appropriateness of the default values for modelling parameters that are stored in X. These include the number of model iterations, and any custom control parameters and priors (by default, empty lists).

procureSlot(X,
            "b_SpecificParameters@iters_1L_int")
#> [1] 4000

In many cases there will be no need to specify any custom control parameters or priors and using the defaults may speed up execution.

procureSlot(X,
            "b_SpecificParameters@control_ls")
#> [[1]]
#> list()

procureSlot(X,
            "b_SpecificParameters@prior_ls")
#> [[1]]
#> list()

However, in this example using the default control parameters would result in warning messages suggesting a change to the adapt_delta control value (default = 0.8). Modifying the adapt_delta control parameter value can address this issue.

X <- renewSlot(X,
             slot_nm_1L_chr = "b_SpecificParameters@control_ls",
             new_val_xx = list(adapt_delta = 0.99))

X <- investigate(X)

class(X)
#> [1] "SpecificMixed"
#> attr(,"package")
#> [1] "specific"

The last call to investigate function wrote the models it tests to a sub-directory of the output directory along with plots for each model.

Create shareable outputs

The model objects created by the preceding analysis are not suitable for sharing as they contain duplicates of the source dataset. To create model objects that can be shared (where dataset copies are replaced with fake data) use the authorData method.

X <- authorData(X)

Purge dataset copies

For the purposes of efficient computation, multiple objects containing copies of the source dataset were saved to our output directory during the analysis process. We therefore need to delete all of these copies by supplying “purge_write” to the type_1L_chr argument of the author method.

author(X,
       type_1L_chr = "purge_write")

A copy of the module X is available for download as the file eq5d_ttu_SpecificMixed.RDS from the “Documentation_0.0” release of the specific package.

1.5 - Implement a utility mapping study

Note: This vignette uses fake data - it is for illustrative purposes only and should not be used to inform decision making.

Motivation

Youth mental health services do not typically collect health utility data from their clients, which makes it more difficult to place an economic values on outcomes attained in these services. One strategy for addressing this gap is to use data from similar samples of young people that contain both health utility and the types of outcome measures that are collected in clinical services. The TTU package provides a toolkit for conducting and reporting a utility mapping (or Transfer to Utility) study.

Implementation

TTU has been developed for use with the ready4 model and combines and extends multiple types of ready4 modules:

• Modules for labeling, validating and summarising youth mental health datasets from the youthvars package;
• Modules for scoring health utility from the scorz package;
• Modules for specifying and testing statistical models from the specific package;
• Modules for generating reports from the ready4show package; and
• Modules for sharing data via online data repositories from the ready4use package.

Additionally, TTU relies on two RMarkdown programs:

• ttu_mdl_ctlg: Generate a Template Utility Mapping (Transfer to Utility) Model Catalogue (https://doi.org/10.5281/zenodo.5936870)
• ttu_lng_ss: Create a Draft Scientific Manuscript For A Utility Mapping Study (https://doi.org/10.5281/zenodo.5976987)

Workflow

Background and citation

The following workflow illustrates (using fake data) the same steps we used in a real world study, a summary of which is available at https://doi.org/10.1101/2021.07.07.21260129). Citation information for that study is:

@article {Hamilton2021.07.07.21260129,
    author = {Hamilton, Matthew P and Gao, Caroline X and Filia, Kate M and Menssink, Jana M and Sharmin, Sonia and Telford, Nic and Herrman, Helen and Hickie, Ian B and Mihalopoulos, Cathrine and Rickwood, Debra J and McGorry, Patrick D and Cotton, Sue M},
    title = {Predicting Quality Adjusted Life Years in young people attending primary mental health services},
    elocation-id = {2021.07.07.21260129},
    year = {2021},
    doi = {10.1101/2021.07.07.21260129},
    publisher = {Cold Spring Harbor Laboratory Press},
    URL = {https://www.medrxiv.org/content/early/2021/07/12/2021.07.07.21260129},
    eprint = {https://www.medrxiv.org/content/early/2021/07/12/2021.07.07.21260129.full.pdf},
    journal = {medRxiv}
}

The program applied in that study, which this workflow closely resembles is available at https://doi.org/10.5281/zenodo.6116077 and can be cited as follows:

@software{hamilton_matthew_2022_6212704,
  author       = {Hamilton, Matthew and
                  Gao, Caroline},
  title        = {{Complete study program to reproduce all steps from 
                   data ingest through to results dissemination for a
                   study to map mental health measures to AQoL-6D
                   health utility}},
  month        = feb,
  year         = 2022,
  note         = {{Matthew Hamilton and Caroline Gao  (2022). 
                   Complete study program to reproduce all steps from
                   data ingest through to results dissemination for a
                   study to map mental health measures to AQoL-6D
                   health utility. Zenodo.
                   https://doi.org/10.5281/zenodo.6116077. Version
                   0.0.9.3}},
  publisher    = {Zenodo},
  version      = {0.0.9.3},
  doi          = {10.5281/zenodo.6212704},
  url          = {https://doi.org/10.5281/zenodo.6212704}
}

Load required packages

We begin by loading our required packages.

library(ready4)
library(ready4use)
library(youthvars)
library(scorz)
library(TTU)

Add dataset metadata

We use the Ready4useDyad and Ready4useRepos modules to retrieve and ingest and to then pair a dataset and its data dictionary.

A <- Ready4useDyad(ds_tb = Ready4useRepos(dv_nm_1L_chr = "fakes",
                                          dv_ds_nm_1L_chr = "https://doi.org/10.7910/DVN/HJXYKQ",
                                          dv_server_1L_chr = "dataverse.harvard.edu") %>%
                     ingest(fls_to_ingest_chr = c("ymh_clinical_tb"),
                            metadata_1L_lgl = F) %>%
                     youthvars::transform_raw_ds_for_analysis(),
                   dictionary_r3 = Ready4useRepos(dv_nm_1L_chr = "TTU", 
                                                  dv_ds_nm_1L_chr = "https://doi.org/10.7910/DVN/DKDIB0", 
                                                  dv_server_1L_chr = "dataverse.harvard.edu") %>%
                     ingest(fls_to_ingest_chr = c("dictionary_r3"),
                            metadata_1L_lgl = F)) %>%
  renew(type_1L_chr = "label")

We use the YouthvarsSeries module to supply metadata about out a longitudinal dataset vignette.

A <- YouthvarsSeries(a_Ready4useDyad = A,
                     id_var_nm_1L_chr = "fkClientID",
                     timepoint_var_nm_1L_chr = "round",
                     timepoint_vals_chr = levels(procureSlot(A,
                                                             "ds_tb")$round))

Score health utility

We next use the ScorzAqol6Adol module to score adolescent AQoL-6D health utility.

A <- TTUProject(a_ScorzProfile = ScorzAqol6Adol(a_YouthvarsProfile = A))
A <- renewSlot(A, "a_ScorzProfile")
#> Joining, by = c("fkClientID", "match_var_chr")

Evaluate candidate models

Over the next few steps we will use modules from the specific package to to specify and assess a number of candidate utility mapping models.

A <- renewSlot(A, "b_SpecificParameters", SpecificConverter(a_ScorzProfile = A@a_ScorzProfile) %>%
                 metamorphose() %>%
                 procureSlot("b_SpecificParameters"))

A <- renewSlot(A, "b_SpecificParameters@predictors_lup", Ready4useRepos(dv_nm_1L_chr = "TTU", 
                                                                        dv_ds_nm_1L_chr = "https://doi.org/10.7910/DVN/DKDIB0", 
                                                                        dv_server_1L_chr = "dataverse.harvard.edu") %>%
                 ingest(fls_to_ingest_chr = c("predictors_r3"),
                        metadata_1L_lgl = F)) 

We can inspect the metadata on candidate predictors that we have just ingested.

exhibitSlot(A, "b_SpecificParameters@predictors_lup",
         scroll_box_args_ls = list(width = "100%"))

We add additional metadata about variables in our dataset that will be used in exploratory modelling.

A <- renewSlot(A, "b_SpecificParameters@depnt_var_min_max_dbl", c(0.03,1)) %>% # Inherit From TTUAqolAdol
  renewSlot("b_SpecificParameters@candidate_predrs_chr", c("BADS","GAD7", "K6", "OASIS", "PHQ9", "SCARED")) %>%
  renewSlot("b_SpecificParameters@candidate_covars_chr", c("d_sex_birth_s", "d_age",  "d_sexual_ori_s", 
                                                           "d_studying_working", "c_p_diag_s", "c_clinical_staging_s",
                                                           "SOFAS")) %>%
  renewSlot("b_SpecificParameters@descv_var_nms_chr", c("d_age","Gender","d_relation_s", "d_sexual_ori_s", 
                                                        "Region", "d_studying_working", "c_p_diag_s", 
                                                        "c_clinical_staging_s","SOFAS")) %>%
  renewSlot("b_SpecificParameters@msrmnt_date_var_nm_1L_chr", "d_interview_date") 

A <-  renewSlot(A, "b_SpecificParameters@fake_1L_lgl", T) 

A <- renewSlot(A, "c_SpecificProject", SpecificModels(a_YouthvarsProfile = A@a_ScorzProfile@a_YouthvarsProfile,
                                                      b_SpecificParameters = A@b_SpecificParameters,
                                                      paths_chr = tempdir())) 

A <- ratifySlot(A, "c_SpecificProject")

A <- renewSlot(A, "c_SpecificProject", 
               authorSlot(A, "c_SpecificProject", what_1L_chr = "workspace"))

We now generate tables and charts that describe our dataset. These are saved in a sub-directory of our output data directory, a copy of which is available for download. One of the plots is also reproduced here.

A <- renewSlot(A, "c_SpecificProject",
               authorSlot(A, "c_SpecificProject", what_1L_chr = "descriptives",
                          digits_1L_int = 3L))

We next compare the performance of different model types. This step saves model objects and plots to a sub-directory of our output directory, a copy of which is available for download.

A <- renewSlot(A, "c_SpecificProject",
               investigateSlot(A, "c_SpecificProject",
                               depnt_var_max_val_1L_dbl = 0.99,
                               session_ls = sessionInfo()))

After inspecting the output of the previous command, we can now specify the preferred model types to use from this point onwards.

A <- renewSlot(A, "c_SpecificProject",
               renew(procureSlot(A, "c_SpecificProject"),
                     new_val_xx = c("GLM_GSN_LOG", "OLS_CLL"),
                     type_1L_chr = "results",
                     what_1L_chr = "prefd_mdls"))

Next we assess multiple versions of our preferred model type - one single predictor model for each of our candidate predictors and the same models with candidate covariates added. A number of model/plot objects saved to a sub-directory of our output directory, a copy of which is available for download.

A <- renewSlot(A, "c_SpecificProject",
               investigateSlot(A,"c_SpecificProject"))

After reviewing the output of the previous step, we specify the covariates we wish to add to the models.

A <- renewSlot(A, "c_SpecificProject",
               renew(procureSlot(A, "c_SpecificProject"),
                     new_val_xx = "SOFAS",
                     type_1L_chr = "results",
                     what_1L_chr = "prefd_covars"))

We now assess the multivariate models. More model/plot objects are saved to a sub-directory of our output directory, a copy of which is available for download.

A <- renewSlot(A, "c_SpecificProject",
               investigateSlot(A, "c_SpecificProject"))

We next reformulate the models we finalised in the previous step so that they are suitable for modelling longitudinal change.

For our primary analysis, we use the longitudinal formulation of the models we previously selected. A series of large model files are written to the local output data directory.

A <- renewSlot(A, "c_SpecificProject",
               investigateSlot(A, "c_SpecificProject"))

For our secondary analyses, we specify alternative combinations of predictors and covariates.

A <- renewSlot(A, "c_SpecificProject",
               investigateSlot(A, "c_SpecificProject",
                               scndry_anlys_params_ls = make_scndry_anlys_params(candidate_predrs_chr = c("SOFAS"),
                                                                                 candidate_covar_nms_chr = c("d_sex_birth_s", 
                                                                                                             "d_age", 
                                                                                                             "d_sexual_ori_s",
                                                                                                             "d_studying_working"),
                                                                                 prefd_covars_chr = NA_character_) %>%
                                 make_scndry_anlys_params(candidate_predrs_chr = c("SCARED","OASIS","GAD7"),
                                                          candidate_covar_nms_chr = c("PHQ9", "SOFAS", 
                                                                                      "d_sex_birth_s", 
                                                                                      "d_age", 
                                                                                      "d_sexual_ori_s",
                                                                                      "d_studying_working"),
                                                          prefd_covars_chr = "PHQ9")))

Report and disseminate findings

Create shareable models

The model objects created and saved in our working directory by the preceding steps are not suitable for public dissemination. They are both too large in file size and, more importantly, include copies of our source dataset. We can overcome these limitations by creating shareable versions of the models. Two types of shareable version are created - copies of the original model objects in which fake data overwrites the original source data and summary tables of model coefficients.

A <- renewSlot(A, "c_SpecificProject",
               authorData(procureSlot(A, "c_SpecificProject")))

Specify study reporting metadata

We create a TTUSynopsis object that contains the fields necessary to render and share reports.

A <- renewSlot(A, "d_TTUReports",
               {
                 Y <- metamorphoseSlot(A, "c_SpecificProject")
                 Y <- TTUSynopsis(a_Ready4showPaths = Y@a_Ready4showPaths,
                                  b_SpecificResults = Y@b_SpecificResults,
                                  c_SpecificParameters = Y@c_SpecificParameters,
                                  d_YouthvarsProfile = Y@d_YouthvarsProfile,
                                  rmd_fl_nms_ls = Y@rmd_fl_nms_ls)
                 Y <- TTUReports(a_TTUSynopsis = Y)
                 Y
                 }
               )

We add metadata relevant to the reports that we will be generating to these fields. Note that the data we supply to the Ready4useRepos object below must relate to a repository to which we have write permissions (otherwise subsequent steps will fail).

A <- renewSlot(A, "d_TTUReports@a_TTUSynopsis",
               procureSlot(A, "d_TTUReports@a_TTUSynopsis") %>% 
                 renewSlot("authors_r3", ready4show::authors_tb) %>%
                 renewSlot("institutes_r3", ready4show::institutes_tb) %>%
                 renewSlot("digits_int", c(3L,3L)) %>%
                 renewSlot("outp_formats_chr", c("PDF","PDF")) %>%
                 renewSlot("title_1L_chr", "A hypothetical utility mapping study using fake data") %>%
                 renewSlot("correspondences_r3", old_nms_chr = c("PHQ9", "GAD7"), new_nms_chr = c("PHQ-9", "GAD-7")) %>%
                 renewSlot("e_Ready4useRepos", Ready4useRepos(dv_nm_1L_chr = "fakes", 
                                                              dv_ds_nm_1L_chr = "https://doi.org/10.7910/DVN/D74QMP", 
                                                              dv_server_1L_chr = "dataverse.harvard.edu"))) 

Author model catalogues

We download a program for generating a catalogue of models and use it to summarising the models created under each study analysis (one primary and two secondary). The catalogues are saved locally.

authorSlot(A, "d_TTUReports", what_1L_chr = "Catalogue", download_tmpl_1L_lgl = T)

Share model catalogue

We share the catalogues that we created, uploading a copy to our study online repository. To run this step you will need write permissions to the online repository.

shareSlot(A, "d_TTUReports@a_TTUSynopsis", type_1L_chr = "Report", what_1L_chr = "Catalogue") 

Share models

We share tables of coefficients and other meta-data about the models we have created by posting them to the online repository. The object we create and share is designed to be used in conjunction with the youthu package to make it easier to make predictions with these models using new data. Again, you will need write permissions to the online repository.

shareSlot(A, "d_TTUReports@a_TTUSynopsis", type_1L_chr = "Models", what_1L_chr = "ingredients")

Author manuscript

We add some content about the manuscript we wish to author.

A <- renewSlot(A, "d_TTUReports@a_TTUSynopsis",
               procureSlot(A, "d_TTUReports@a_TTUSynopsis") %>% 
                 renewSlot("background_1L_chr", "Quality Adjusted Life Years (QALYs) are often used in economic evaluations, yet utility weights for deriving them are rarely directly measured in mental health services.") %>%
                 renewSlot("coi_1L_chr", "None declared") %>%
                 renewSlot("conclusion_1L_chr","Nothing should be concluded from this study as it is purely hypothetical.") %>%
                 renewSlot("ethics_1L_chr", "The study was reviewed and granted approval by no-one." ) %>%
                 renewSlot("funding_1L_chr", "The study was funded by no-one.") %>%
                 renewSlot("interval_chr", "three months") %>%
                 renewSlot("keywords_chr", c("anxiety", "AQoL","depression", "psychological distress", "QALYs", "utility mapping")) %>%
                 renewSlot("sample_desc_1L_chr", "The study sample is fake data.") )

We create a summary of results that can be interpreted by the program that authors the manuscript.

A <- renewSlot(A, "d_TTUReports@a_TTUSynopsis@abstract_args_ls",
               manufactureSlot(A,"d_TTUReports@a_TTUSynopsis", what_1L_chr = "abstract_args_ls",
                               depnt_var_nms_chr = c("AQoL-6D", "Adolescent AQoL Six Dimension"))) 

A <- enhanceSlot(A, "d_TTUReports@a_TTUSynopsis", with_1L_chr = "results_ls",
                 depnt_var_nms_chr = c("AQoL-6D", "Adolescent AQoL Six Dimension")) 

We create and save the plots that will be used in the manuscript.

authorSlot(A, "d_TTUReports", type_1L_chr = "Plots",
           depnt_var_desc_1L_chr = A@d_TTUReports@a_TTUSynopsis@b_SpecificResults@a_SpecificShareable@shareable_outp_ls$results_ls$study_descs_ls$health_utl_nm_1L_chr)

We download a program for generating a template manuscript and run it to author a first draft of the manuscript.

authorSlot(A, "d_TTUReports", type_1L_chr = "Report", what_1L_chr = "Manuscript_Auto", download_tmpl_1L_lgl = T)

We can copy the RMarkdown files that created the template manuscript to a new director (which we call “Manuscript_Submission”) so that we can then manually edit those files to produce a manuscript that we can submit for publication. Note that in this example we have not made any edits to the template manuscript.

R.utils::copyDirectory(paste0(A@d_TTUReports@a_TTUSynopsis@a_Ready4showPaths@outp_data_dir_1L_chr,
                              "/",
                              A@d_TTUReports@a_TTUSynopsis@a_Ready4showPaths@mkdn_data_dir_1L_chr,
                              "/Manuscript_Auto"),
                       paste0(A@d_TTUReports@a_TTUSynopsis@a_Ready4showPaths@outp_data_dir_1L_chr,
                              "/",
                              A@d_TTUReports@a_TTUSynopsis@a_Ready4showPaths@mkdn_data_dir_1L_chr,
                              "/Manuscript_Submission"))

Once any edits to the RMarkdown files for creating the submission manuscript have been finalised, we can run the following command to author the manuscript. The below commands will generate a Microsoft Word format manuscript and a PDF technical appendix. Unlike the template manuscript, the figures and tables are positioned after (and not within) the main body of the manuscript. Note that the Word version of the manuscript generated by these commands will require some minor formatting edits (principally to the display of tables and numbering of sections).

A <- renewSlot(A, "d_TTUReports",
               procureSlot(A, "d_TTUReports") %>%
                 renewSlot("a_TTUSynopsis@tables_in_body_lgl",  F) %>%
                 renewSlot("a_TTUSynopsis@figures_in_body_lgl", F) %>%
                 renewSlot("a_TTUSynopsis@outp_formats_chr", c("Word","PDF")))
authorSlot(A, "d_TTUReports", what_1L_chr = "Manuscript_Submission", download_tmpl_1L_lgl = F)

Tidy workspace

The preceding steps saved multiple objects (mostly R model objects) that have embedded within them copies of the source dataset. We can now purge all such copies from our output data directory.

author(procureSlot(A,"c_SpecificProject"),
       type_1L_chr = "purge_write") 

1.6 - Find and deploy utility mapping models

library(youthu)

This vignette outlines a workflow for:

• Searching, selecting and retrieving transfer to utility models;
• Preparing a prediction dataset for use with a selected transfer to utility model; and
• Applying the selected transfer to utility model to a prediction dataset to predict Quality Adjusted Life Years (QALYs).

The practical value of implementing such a workflow is discussed in the economic analysis vignette and a scientific manuscript. Note, this example uses fake data - it should should not be used to inform decision making.

Search, select and retrieve transfer to utility models

To identify datasets that contain transfer to utility models compatible with youthu (ie those developped with the TTU package), you can use the get_ttu_dv_dss function. The function searches specified dataverses (in the below example, the TTU dataverse) for datasets containing output from the TTU package.

ttu_dv_dss_tb <- get_ttu_dv_dss("TTU")

The ttu_dv_dss_tb table summarises some pertinent details about each dataset containing TTU models found by the preceding command. These details include a link to any scientific summary (the “Article” column) associated with a dataset.

To identify models that predict a specified type of health utility from one or more of a specified subset of predictors, use:

mdls_lup <- get_mdls_lup(ttu_dv_dss_tb = ttu_dv_dss_tb,
                         utility_type_chr = "AQoL-6D",
                         mdl_predrs_in_ds_chr = c("PHQ9 total score",
                                                  "SOFAS total score"))

The preceding command will produce a lookup table with information that includes the catalogue names of models, the predictors used in each model and the analysis that generated each one.

To review the summary information about the predictive performance of a specific model, use:

get_dv_mdl_smrys(mdls_lup,
                 mdl_nms_chr = "PHQ9_SOFAS_1_OLS_CLL")
#> $PHQ9_SOFAS_1_OLS_CLL
#>        Parameter Estimate    SE          95% CI
#> 1 SD (Intercept)    0.348 0.017   0.312 , 0.382
#> 2      Intercept    0.428 0.129   0.174 , 0.686
#> 3  PHQ9 baseline   -9.115 0.249 -9.601 , -8.618
#> 4    PHQ9 change   -7.331 0.339 -8.007 , -6.665
#> 5 SOFAS baseline    0.960 0.172   0.616 , 1.292
#> 6   SOFAS change    1.146 0.235   0.674 , 1.607
#> 7             R2    0.767 0.012   0.743 , 0.788
#> 8           RMSE    0.925 0.004   0.922 , 0.928
#> 9          Sigma    0.406 0.012   0.384 , 0.429

More information about a selected model can be found in the online model catalogue, the link to which can be obtained with the following command:

get_mdl_ctlg_url(mdls_lup,
                 mdl_nm_1L_chr = "PHQ9_SOFAS_1_OLS_CLL")

Prepare a prediction dataset for use with a selected transfer to utility model

Import data

You can now import and inspect the dataset you plan on using for prediction. In the below example we use fake data.

data_tb <- make_fake_ds_one()

Confirm dataset can be used as a prediction dataset

The prediction dataset must contain variables that correspond to all the predictors of the model you intend to apply. The allowable range and required class of each predictor variable are described in the min_val_dbl, max_val_dbl and class_chr columns of the model predictors lookup table, which can be accessed with a call to the get_predictors_lup function.

predictors_lup <- get_predictors_lup(mdls_lup = mdls_lup,
                                     mdl_nm_1L_chr = "PHQ9_SOFAS_1_OLS_CLL")

The prediction dataset must also include both a unique client identifier variable and a measurement time-point identifier variable (which must be a factor with two levels). The dataset also needs to be in long format (ie where measures at different time-points for the same individual are stacked on top of each other in separate rows). We can confirm these conditions hold by creating a dataset metadata object using the make_predn_metadata_ls function. In creating the metadata object, the function checks that the dataset can be used in conjunction with the model specified at the mdl_nm_1L_chr argument. If the prediction dataset uses different variable names for the predictors to those specified in the predictors_lup lookup table, a named vector detailing the correspondence between the two sets of variable names needs to be passed to the predr_vars_nms_chr argument. Finally, if you wish to specify a preferred variable name to use for the predicted utility values when applying the model, you can do this by passing this name to the utl_var_nm_1L_chr argument.

predn_ds_ls <- make_predn_metadata_ls(data_tb,
                                      id_var_nm_1L_chr = "UID",
                                      msrmnt_date_var_nm_1L_chr = "Date",
                                      predr_vars_nms_chr = c(PHQ9 = "PHQ_total",SOFAS = "SOFAS_total"),
                                      round_var_nm_1L_chr = "Timepoint",
                                      round_bl_val_1L_chr = "Baseline",
                                      utl_var_nm_1L_chr = "AQoL6D_HU",
                                      mdls_lup = mdls_lup,
                                      mdl_nm_1L_chr = "PHQ9_SOFAS_1_OLS_CLL")

Apply the selected transfer to utility model to a prediction dataset to predict Quality Adjusted Life Years (QALYs)

Predict health utility at baseline and follow-up timepoints

To generate utility predictions we use the add_utl_predn function. The function needs to be supplied with the prediction dataset (the value passed to argument data_tb) and the validated prediction metadata object we created in the previous step.

data_tb <- add_utl_predn(data_tb,
                         predn_ds_ls = predn_ds_ls)
#> Joining, by = c("UID", "Timepoint")

By default the add_utl_predn function samples model parameter values based on a table of model coefficients when making predictions and constrains predictions to an allowed range. You can override these defaults by adding additional arguments new_data_is_1L_chr = "Predicted" (which uses mean parameter values), force_min_max_1L_lgl = F (removes range constraint) and (if the source dataset makes available downloadable model objects) make_from_tbl_1L_lgl = F. These settings will produce different predictions. It is strongly recommended that you consult the model catalogue (see above) to understand how such decisions may affect the validity of the predicted values that will be generated.

Our health utility predictions are now available for use and are summarised below.

summary(data_tb$AQoL6D_HU)
#>    Min. 1st Qu.  Median    Mean 3rd Qu. 
#> 0.06646 0.42781 0.63403 0.62335 0.83351 
#>    Max. 
#> 1.00000

Calculate QALYs

The last step is to calculate Quality Adjusted Life Years, using a method assuming a linear rate of change between timepoints.

data_tb <- data_tb %>% add_qalys_to_ds(predn_ds_ls = predn_ds_ls,
                                       include_predrs_1L_lgl = F,
                                       reshape_1L_lgl = F)

1.7 - Use utility mapping algorithms to help implement cost-utility analyses

library(youthu)
set.seed(1234)

This vignette illustrates the rationale for and practical decision-making utility of youthu’s QALYs prediction workflow. Note, this example is illustrated with fake data and should not be used to inform decision-making.

Motivation

The main motivation behind the youthu package is to extend the types of economic analysis that can be undertaken with both single group (e.g. pilot study, health service records) and matched groups (e.g. trial) longitudinal datasets that do not include measures of health utility. This article focuses on its application to matched group datasets.

Example dataset

First, we must first import our data. In this example we will use a fake dataset.

ds_tb <- make_fake_ds_two()
#> Joining, by = c("fkClientID", "study_arm_chr")

Our dataset includes 268 matched comparisons, with each comparison containing baseline and follow-up records for one intervention arm participant and one control arm participant. The first few records are as follows.

This dataset contains features that make it possible to use in conjunction with youthu’s economic analysis functions. These requirements are described in the vignette about finding and using models compatible models to predict QALYs;

The dataset also contains a cost variable, which is a requirement for most, though not all, of the economic analyses that can be undertaken with youthu.

Limitations of datasets without measures of health utility

A notable omission from the dataset is any measure of utility. This omission means that, in the absence of using mapping algorithms such as those included with youthu, the most feasible types of economic evaluation to apply to this dataset would likely be cost-consequence analysis (where a synopsis of the differences in a range of measures are presented alongside cost differences) and cost-effectiveness analysis (where a summary statistic - the incremental cost-effectiveness ratio or ICER - is calculated by dividing differences in costs by differences in a single outcome measure).

These types of economic analyses can be relatively simple to interpret if either the intervention or control arm is simultaneously cheaper and more effective across all included outcome measures. However, these conditions don’t hold in our sample data.

summary((ds_tb %>% dplyr::filter(study_arm_chr == "Control" & round == "Baseline"))[5:6])
#>       PHQ9          SOFAS      
#>  Min.   : 0.0   Min.   :39.00  
#>  1st Qu.: 7.0   1st Qu.:60.00  
#>  Median :12.0   Median :66.00  
#>  Mean   :10.9   Mean   :66.13  
#>  3rd Qu.:15.0   3rd Qu.:72.00  
#>  Max.   :19.0   Max.   :89.00

summary((ds_tb %>% dplyr::filter(study_arm_chr == "Control" & round == "Follow-up"))[5:7])
#>       PHQ9            SOFAS         costs_dbl     
#>  Min.   : 0.000   Min.   :39.00   Min.   : 889.9  
#>  1st Qu.: 4.000   1st Qu.:64.00   1st Qu.:1321.1  
#>  Median : 8.000   Median :71.00   Median :1486.7  
#>  Mean   : 8.493   Mean   :70.65   Mean   :1489.0  
#>  3rd Qu.:13.000   3rd Qu.:77.00   3rd Qu.:1627.0  
#>  Max.   :27.000   Max.   :98.00   Max.   :2216.5

summary((ds_tb %>% dplyr::filter(study_arm_chr == "Intervention" & round == "Baseline"))[5:6])
#>       PHQ9           SOFAS      
#>  Min.   : 0.00   Min.   :36.00  
#>  1st Qu.: 7.00   1st Qu.:61.00  
#>  Median :11.00   Median :67.00  
#>  Mean   :10.81   Mean   :66.74  
#>  3rd Qu.:15.00   3rd Qu.:72.25  
#>  Max.   :19.00   Max.   :88.00

summary((ds_tb %>% dplyr::filter(study_arm_chr == "Intervention" & round == "Follow-up"))[5:7])
#>       PHQ9            SOFAS      costs_dbl     
#>  Min.   : 0.000   Min.   :40   Min.   : 923.4  
#>  1st Qu.: 2.000   1st Qu.:60   1st Qu.:1625.6  
#>  Median : 6.500   Median :68   Median :1777.3  
#>  Mean   : 6.851   Mean   :68   Mean   :1807.8  
#>  3rd Qu.:11.000   3rd Qu.:77   3rd Qu.:1996.0  
#>  Max.   :25.000   Max.   :93   Max.   :2872.7

The pattern of results summarised above create some significant barriers to meaningfully interpreting economic evaluations that are based on cost-consequence or cost-effectiveness analysis:

• A cost-effectiveness analysis in which change in PHQ-9 was the benefit measure would be difficult to interpret as the Intervention arm is both more effective and more costly, which begs the question is it worth paying the extra dollars for this improvement? Also - would a judgment of cost-effectiveness remain the same if the study had measured a slightly different incremental benefit or recorded change over a longer or shorter time horizon? It is likely that there is no commonly used value for money benchmark for improvements measured in PHQ-9, nor is there any time weighting associated with the measure. Furthermore, if the potential funding for the intervention is from a budget that is allocated to non-depressive illnesses (e.g. physical health), results from a cost-effectiveness analysis using PHQ-9 as its benefit measure are not readily comparable with economic evaluations of interventions from other illness groups using different benefit measures that are potentially competing for the same scarce funding.

• A cost consequence analyses that summarised the differences in costs with the differences in changes in PHQ-9 and SOFAS score would be difficult to interpret because while the intervention is more effective than control for improvements measured on PHQ-9 (where lower scores are better), the control group is superior if benefits are based on functioning improvements as measured by SOFAS scores (where higher scores are better). The lack of any formal weighting for how to trade off clinical symptoms and functioning means that interpretation of this analysis will be highly subjective and likely to change across potential decision makers.

These types of short-comings can be significantly addressed by undertaking cost-utility analyses (CUAs) as:

• they use a measure of benefit - the Quality Adjusted Life Year (QALY) - that captures multiple domains of health, weighted by time and population preferences in a single index measure that can be applied across health conditions;
• there are published benchmark willingness to pay values for QALYs that are routinely used by decision makers in many countries to make ICER statistics readily interpretable in the context of health budget allocation.

The rest of this article demonstrates how youthu functions can be used to undertake CUA based analyses on the type of data we have just profiled.

Using youthu in a cost-utility analysis workflow

Predict adolescent AQoL-6D health utility

Our first step is to identify which youthu models we will use to predict adolescent AQoL-6D and apply these models to our data. This step was explained in more detail in another vignette article about finding and using transfer to utility models, so will be dealt with briefly here.

First we make sure that our dataset can be used as a prediction dataset in conjunction with the model we intend using.

predn_ds_ls <- make_predn_metadata_ls(ds_tb,
                                      cmprsn_groups_chr = c("Intervention", "Control"),
                                      cmprsn_var_nm_1L_chr = "study_arm_chr",
                                      costs_var_nm_1L_chr = "costs_dbl",
                                      id_var_nm_1L_chr = "fkClientID",
                                      msrmnt_date_var_nm_1L_chr = "date_psx",
                                      round_var_nm_1L_chr = "round",
                                      round_bl_val_1L_chr = "Baseline",
                                      utl_var_nm_1L_chr = "AQoL6D_HU",
                                      mdls_lup = get_mdls_lup(utility_type_chr = "AQoL-6D",
                                                              mdl_predrs_in_ds_chr = c("PHQ9 total score",
                                                                                       "SOFAS total score"),
                                                              ttu_dv_nms_chr = "TTU"),
                                      mdl_nm_1L_chr =  "PHQ9_SOFAS_1_OLS_CLL")

We now use our preferred model to predict health utility from the measures in our dataset.

ds_tb <- add_utl_predn(ds_tb,
                       predn_ds_ls = predn_ds_ls) %>%
  dplyr::select(fkClientID, round, study_arm_chr, date_psx, duration_prd, dplyr::everything())
#> Joining, by = c("fkClientID", "round")

Calculate QALYs

Next we combine the health utility data with the interval between measurement data to calculate QALYs and add them to the dataset.

ds_tb  <- ds_tb %>% add_qalys_to_ds(predn_ds_ls = predn_ds_ls,
                                    include_predrs_1L_lgl = T,
                                    reshape_1L_lgl = T)

Analyse results

Now we can run the main economic analysis. This is implemented by the make_hlth_ec_smry function, which first bootstraps the dataset (implemented by the boot function from the boot package) before passing the mean values for costs and QALYs from each bootstrap sample to with bcea function of the BCEA package to calculate a range of health economic statistics. For this example we pass a value of 50,000 for the willingness to pay parameter, as this is the dollar amount commonly used in Australia as a benchmark for the value of a QALY.

Note, for this illustrative example we only request 1000 bootstrap iterations - in practice this number may be higher.

he_smry_ls <- ds_tb %>% make_hlth_ec_smry(predn_ds_ls = predn_ds_ls,
                                                 wtp_dbl = 50000,
                                                 bootstrap_iters_1L_int = 1000L)

As part of the output of the make_hlth_ec_smry function is a BCEA object, we can use the BCEA package to produce a number of graphical summaries of economic results. One of the most important is the production of a cost-effectiveness plane. This plot highlights that, with an ICER of $-98,145.56, less than half of the bootstrapped iteration incremental cost and QALY pairs fall within the zone of cost-effectiveness (green). In fact, at the cost-effectiveness threshold we supplied, the results suggest there is a 8% probability that the intervention is cost-effective.

library(ggplot2)
BCEA::ceplane.plot(he_smry_ls$ce_res_ls, wtp =50000,    
                   area_color = "green",
                    graph = "ggplot2",
          theme = ggplot2::theme_light())
#> Warning: Duplicated aesthetics after name standardisation: colour

1.8 - Develop choice models

library(mychoice)

The tools in mychoice are designed to make it easier to develop and use choice models with ready4 - an open source health economic model of the systems shaping mental health and wellbeing in young people.

This development version of the mychoice package has been made available as part of the process of testing and documenting the package.

Currently there are no vignettes available. However, examples of the application of mychoice functions to a real world discrete choice experiment are in programs available at https://doi.org/10.5281/zenodo.6626256 (design of a discrete choice experiment survey) and https://doi.org/10.5281/zenodo.7223286 (analysis of discrete choice experiment survey responses). PDF versions of each program, along with the artefacts produced by each are available in the online dataset at https://doi.org/10.7910/DVN/VGPIPS.

2 - Modules for modelling places

3 - Modules for modelling platforms

4 - Modules for modelling programs