Documentation

• 1: Overview
• 2: Examples
• 3: Getting started
• 3.1: Motivation
• 3.2: Concepts
• 3.2.1: Model
• 3.2.2: Module
• 3.2.3: Modelling project
• 3.2.4: Reproducible research
• 3.2.5: Transferability
• 3.3: Users
• 3.3.1: Coders
• 3.3.2: Modellers
• 3.3.3: Planners
• 3.4: Stakeholders
• 3.4.1: Funders
• 3.4.2: Researchers
• 3.4.3: Young people
• 3.5: Software
• 3.5.1: System requirements
• 3.5.2: Code repositories
• 3.5.3: Release statuses
• 3.5.3.1: Unreleased code
• 3.5.3.2: Development releases
• 3.5.3.3: Production releases
• 3.5.3.4: Archived releases
• 3.5.4: Code libraries
• 3.5.4.1: Current libraries
• 3.5.4.1.1: Framework libraries
• 3.5.4.1.2: Model module libraries
• 3.5.4.2: Code library documentation
• 3.5.4.3: Dependencies
• 3.5.4.4: Installation and set-up
• 3.5.4.4.1: Installing the ready4 framework foundation library
• 3.5.4.4.2: Installing authoring tools
• 3.5.4.4.2.1: Installing tools for authoring model modules
• 3.5.4.4.2.2: Installing tools for authoring and managing model datasets
• 3.5.4.4.2.3: Installing tools for authoring reproducible analyses
• 3.5.4.4.3: Installing ready4 computational model modules
• 3.5.5: Executables
• 3.5.5.1: Programs
• 3.5.5.2: Subroutines
• 3.5.6: User interfaces
• 3.5.7: Terms of use
• 3.5.7.1: Open source licensing
• 3.5.7.2: Citing ready4
• 3.5.7.3: Disclaimer
• 4: Framework
• 4.1: Standards
• 4.2: Implementation
• 4.2.1: Paradigms
• 4.2.1.1: Why ready4 is object oriented
• 4.2.1.2: The role of functional programming in ready4 development
• 4.2.2: Modularity
• 4.2.3: Syntax
• 5: Model
• 5.1: Finding modules and sub-modules
• 5.2: Using ready4 modules
• 5.2.1: Modules for modelling people
• 5.2.1.1: Add metadata to datasets of individual human records
• 5.2.1.2: Validate variable total scores
• 5.2.1.3: Score health utility
• 5.2.1.4: Explore candidate utility mapping models
• 5.2.1.5: Implement a utility mapping study
• 5.2.1.6: Find and deploy utility mapping models
• 5.2.1.7: Use utility mapping algorithms to help implement cost-utility analyses
• 5.2.1.8: Develop choice models
• 5.2.2: Modules for modelling places
• 5.2.3: Modules for modelling platforms
• 5.2.4: Modules for modelling programs
• 5.3: Modules pipeline
• 5.3.1: Pipeline of people modules
• 5.3.2: Pipeline of places modules
• 5.3.3: Pipeline of platforms modules
• 5.3.4: Pipeline of programs modules
• 5.4: Authoring ready4 modules
• 5.4.1: Authoring model data structures
• 5.4.2: Authoring model algorithms
• 5.4.3: Dissemating citable, documented and quality assured model module libraries
• 6: Data
• 6.1: Finding and using open access data
• 6.1.1: Search open access data collections
• 6.1.2: Ingest data from an open access repository
• 6.2: Authoring model data
• 6.2.1: Share data via online repositories
• 6.2.2: Add a data dictionary to a dataset
• 7: Analyses
• 7.1: Decision aids
• 7.1.1: Predicting the spatial epidemiology of emerging mental disorders
• 7.2: Code to reproduce and/or replicate scientific studies
• 7.2.1: Model youth choices
• 7.2.1.1: Design a Discrete Choice Experiment
• 7.2.1.2: Analyse the results of a Discrete Choice Experiment
• 7.2.2: Create synthetic populations
• 7.2.2.1: Create a synthetic population of young people attending primary mental health services
• 7.2.3: Model health utility
• 7.2.3.1: Develop health utility mapping algorithms
• 7.2.3.2: Predict health utility
• 7.3: Subroutine templates for authoring analysis reports
• 7.3.1: Make a catalogue of utility mapping models
• 7.3.2: Author a template scientific manuscript
• 7.3.3: Author a draft scientific manuscript for a utility mapping study
• 7.3.4: Make results summary for a Discrete Choice Experiment
• 7.4: Authoring reproducible analyses
• 7.4.1: Authoring scientific manuscripts
• 8: Contributions
• 8.1: Our priorities
• 8.1.1: Priority 1: Launch ready4 Minimum Viable Product (MVP) systems model
• 8.1.2: Priority 2: Maintain ready4
• 8.1.3: Priority 3: Apply ready4 to undertake replications and transfers
• 8.1.4: Priority 4: Grow a ready4 community
• 8.1.5: Priority 5: Extend the scope of the ready4 model
• 8.1.6: Priority 6: Integrate ready4 with other open source tools
• 8.2: Contribution types
• 8.2.1: Provide advice
• 8.2.2: Contribute code
• 8.2.3: Fund projects
• 8.2.4: Undertake projects
• 8.2.5: Support the ready4 community
• 8.3: Contributor covenant (code of conduct)

1 - Overview

What is ready4?

ready4 is a modular computational model of the systems that shape young people’s mental health that is being developed as an open source project led by researchers at Orygen and Monash University.

What is a computational model?

A computational model is a simplified representation of a system of interest that is implemented in computer code.

What makes it modular?

The paradigms we use for computational model development allow individual model components (modules) to be independently reused (in other models) and safely and flexibly combined (to model more extensive systems).

How is that achieved?

To facilitate interoperability between modules developed by multiple modelling teams, we have defined a framework for model development that comprises a set of explicit standards and the software (currently, written in R) to implement modules that adhere to those standards.

Why is that useful?

Modelling youth mental health systems is a complex task - that means it is easy to make mistakes and is more likely to be done well if models are implemented to be accountable, reusable and updatable. We hope that ready4 will support these goals and in so doing help to share data, improve the replicability and transferability of modelling analyses and generate valuable insights for youth mental health policymakers and system planners.

Who is it for?

ready4 is designed to be primarily used by coders, modellers and planners, working collaboratively on computational modelling projects in which other stakeholders such as funders, researchers and young people make essential contributions.

Can I use it?

ready4 is free for you to assess (to verify and validate), apply (to generate novel insights into decision problems of interest to you) and to derive your own derivative works from (to leverage and enhance the work of others) under liberal terms of use.

Can I help?

ready4 is a collaborative project and we’d love your help in progressing our priorty project goals! You can help fund our development, contribute code improvements, enhance our documentation and community support, give us advice and/or lead a modelling project.

Why is it called ready4?

ready4 is shorthand for “readyforwhatsnext”, the title of the research project that catalysed this model and a reference to both how good mental health can equip young people for better lives and how computational models can improve the preparedness of decision makers and system planners.

Where should I go next?

We’d recommend reading the documentation in the order in which sections appear in the table of contents (so go next to Examples, then to Getting started and so on).

2 - Examples

Some examples of practical applications of ready4 include:

• Mapping psychological and functional measures to health utility for young people using primary mental health services (a pre-print manuscript, a replication and results dataset, replication code and R packages for developing and applying mapping models);
• Assessing the heterogeneity of quality of life in a clinical sample of young people (a peer-reviewed manuscript and R package for exploring and characteristing heterogeneity in quality of life data);
• Predicting the online helpseeking choices of socially anxious young people (an R package for designing and analysing discrete choice experiments, a replication dataset and replication code - a manuscript will also be available shortly); and
• Predicting the future spatial distribution of mental disorder in young people (a policy briefing, an app and a dataset of sample reports generated by the app).

3 - Getting started

3.1 - Motivation

Problem

Improving the mental health and wellbeing of young people is a global public health priority. However, identifying the optimal policy and system design choices to meet this challenge is not straightforward. Models are a potentially useful tool to help decision makers navigate complexity, but can have significant limitations such as:

• Mistakes: Errors, common in even relatively simple health economic models, become both more likely to occur and more difficult to detect as model complexity grows;

• Poor transparency: the validity of model analyses can be difficult to adequately ascertain if it is not clear who contributed to the model, what assumptions they made, how model algorithms were implemented, how those algorithms were tested and what data they were applied to;

• Contested legitimacy: the value judgments of the model development team (e.g. what types of question are most important for a model to address, what parts of the workings of the system of interest to represent and in what detail, what outcome variables to include, which stakeholders to consult, etc) may differ from those using or affected by model outputs;

• Narrow applicability: a model might be too simple to adequately explore some problems and too complex to reliably address others and be hard to transfer beyond a very specific decision context (e.g. within a specific jurisdiction);

• Limited interoperability: different approaches to model implementation, dissemination, ownership and reporting makes it more difficult for multiple models to be efficiently and safely combined;

• Ease of misuse: in the absence of clear documentation and prominent caveats, a model can easily be applied to decision problems to which it is poorly suited (potentially supporting decisions with serious negative consequences);

• Restricted access: a potential overcompensation for fear of model misuse is constructing high barriers to accessing model code and data - thus limiting model testing, reuse and refinement; and

• Growing stale: health economic models are rarely updated, meaning they can lose validity with time (e.g. input data becomes less relevant, new better performing algorithms are not incorporated, sudden major changes in environment / epidemiology / policy / service system are not accounted for).

Reponse

To help address these issues, ready4 is being implemented as a modular and open source computational model of youth mental health that aims to be accountable, reusable and updatable.

Accountable

Model code and data are publicly available in online code repositories and data collections. Algorithms are documented and transparently and regularly tested. Model development occurs in the open and invites community participation, with each individual’s contribution publicly identifiable. Analyses are reproducible and replicable.

Reusable

Model modules and datasets originally developed in one modelling project can be independently reused in other projects. As they share a common framework, model modules can be combined in other models and analyses to address multiple topics. Due to ready4’s code implementation paradigms, model modules are easier to transfer for use in other decision contexts.

Updatable

Model code, data and analyses are versioned, with an ongoing program of making new and updated releases. Software is maintained, with opportunities for users and contributors to flag issues, request features and supply code contributions.

3.2 - Concepts

3.2.1 - Model

A model is a simplified representation of a system of interest. In the way we use the term, we also mean that a model is:

• abstract and general (i.e. largely free of non-modifiable data, including numeric values, that are assumption- or context- specific) and
• a tool (i.e. a model can be used to help undertake an analysis, it is not the analysis itself).

If a model is developed primarily to inform a decision or set of decisions (e.g. relating to youth mental health policy and system design) it can be called a decision model.

Ideally, a model should have three inter-related representations - conceptual, mathematical and computational.

Conceptual Model

A conceptual model refers to underlying theory and beliefs about a system of interest that can be described in words and pictures.

Mathematical Model

A mathematical model formalises a conceptual model as a set of equations.

Computational Model

A computational model implements the conceptual and mathematical models of a system of interest as computer code.

ready4 is a computational model of youth mental health. More specifically, the ready4 computational model is the complete set of ready4 modules.

3.2.2 - Module

A modular computational model is one that constructed from multiple self-contained components, called modules. Each ready4 module describes a data structure and the set of algorithms that can be applied to it.

The advantages of developing ready4 as a modular model include:

• each module can be independently re-used in other computational models of the people, places, platforms and programs important to the mental health of young people;

• a complex model can be developed iteratively, beginning with a simple representation that is easier to validate and then progressively expanding in scope and functionality through the addition of new modules, to be validated independently and jointly; and

• long term and resource intensive model development is more feasible through combining multiple independently managed and financed modelling projects.

To ensure that all ready4 modules can be safely and flexibly combined, each module is created from a template using authoring tools that support standardisation.

3.2.3 - Modelling project

As a complex, collaborative and long-term undertaking, it is not feasible for ready4 to be financed by a single funder or progressed as a single project. Instead, our mode of development is via multiple independent modelling projects, each with their own project governance and funding, but which adopt a common framework.

A ready4 modelling project will involve the three steps of:

• Developing and validating a computational model;

• Adding context-specific data to that computational model; and

• Applying the computational model to the supplied data to undertake analyses.

The key components of each step are summarised here.

3.2.4 - Reproducible research

Although there is widespread support from the scientific community on the importance of reproducible research, the definition of key terms such as reproducibility and replicability can vary across disciplines and methodologies (e.g. the extent to which computational modelling is used). In some cases, entirely different terms (e.g. repeatability) are preferred. The meanings we intend when using these terms are described below.

Reproduction and Replication

The distinctions we make between reproduction and replication have been guided by the approach outlined in a report by the National Academies of Sciences, Engineering and Medicine. However, we have adapted their definitions slightly as the meanings in that report were framed in terms of study findings / outcomes, whereas our usage relates more to intended objectives when deploying tools.

Meanings

Reproduction

Applying the same analysis code to the same input data with the expectation of generating identical outputs (with the exception of trivial artefacts like datestamps for when analysis reports were produced).

Replication

Applying analysis code used in a study to new input data. The analysis code is reused with only minimal edits that are necessary to account for differences in input data paths and variable names and to study metadata (e.g. investigator names, sample descriptions). The new data can be real or fake, but will include the same structure and concepts / measures as those found in the original study’s dataset. If the new data is a sample from the same population as the original study, then the expectation when undertaking replications is for results across studies to be broadly similar.

Examples

Examples of both reproduction and replication code are available. When publishing analysis code we try to adopt (there are exceptions) the following rules of thumb:

1. If the data required to re-run a study analysis are publicly available (or declared by the analysis program itself), then we publish the code as a reproduction program (e.g. this program for creating a synthetic population).

2. If the data required to re-run a study analysis are not publicly available, we publish the replication version of the code. The replication version of the code may be configured to ingest a synthetic (fake) representation of the study dataset as with this utility mapping replication program. Details of the (minimal) steps required to revert the replication code to a version that can be used for reproduction purposes are typically embedded within the program itself.

3.2.5 - Transferability

It is common for discussions of scientific studies to consider the extent to which findings can be generalised (e.g. if a well conducted study concludes with high confidence that an intervention is cost-effective in Australia, is it valid to infer that it is likely to be cost-effective in the United Kingdom?). However, we are more interested in the transferability of computational models (e.g. the extent to which the data-structures and algorithms from a computational model developed for an Australian context can be used to explore similar topics in the United Kingdom). Our usage of the term “transferring” (and by extension “transferability”, “transferable”, “transfers”) reflects this motivation.

Transferring - our meaning

Adapting a computational model, in whole or in part, to extend the types of data and/or research questions to which it can be applied. The new types of data will possess some differences in structure and / or concepts from that to which the computational model had previously been applied and these differences may be why research questions need to be reformulated.

When we use the term transferring, we are typically referring to either (a) authoring or (b) using on of the following:

1. An analysis program (or sub-routine) that has been adapted from an executable from another study to account for differences in the input data / research question.

2. Inheriting data-structures and algorithms that selectively re-use, discard and replace elements of a study’s computational model based on an alternative use-case.

3. (Multi-purpose) function libraries that have been created by decomposing a study’s (single purpose) analysis program.

Examples

The scorz module library was originally developed to provide an R implementation of algorithms in other languages for scoring adolescent AQoL-6D health utility as part of a utility mapping study (which also used the analysis program mentioned above). Examples of all three approaches mentioned in the previous section can be seen by examining the documentation and source code of the scorz library:

1. Two vignette programs from the scorz library website score different utility instruments. The first program scores adolescent AQoL-6D health utility and acts as a template for the second, which has been modified to score EQ-5D health utility.

2. Inspecting those example programs shows that one of the key adaptations in the EQ-5D program is to use the ScorzEuroQol5 module instead of the ScorzAqol6Adol module. Both of these modules inherit from ScorzProfile. This arrangement means that all three modules share some features (in terms of both structure and algorithms) but selectively differ (e.g. aspects that are necessarily different for scoring different instruments).

3. The algorithms attached to each module from the scorz library are principally implemented by functions (the source code for which can be viewed here) that were created when decomposing an early draft of the above mentioned study algorithm. These functions are called by module methods (source code viewable here).

3.3 - Users

3.3.1 - Coders

If you are a coder, you may be a data scientist or software engineer by training or are perhaps a quantitative researcher who spends a high proportion of their time writing code to undertake their work.

Role

The primary role of coders in ready4 modelling projects is to author modules that implement computational models.

Benefits of using ready4

ready4 provides an opportunity to write software that matters! Our aim is to help improve the lives of young people through empowering decision makers with better models. If you already write code for youth mental health modelling projects, ready4 may also be helpful to you by enhancing your impact (providing a framework for modular models that facilitate re-use by others) and efficiency (through partial automation of code development and quality-assurance workflows).

Contributing to ready4

The types of contribution you can make to ready4 include:

• testing, improving and authoring code;
• enhancing technical documentation and help; and
• providing advice and feedback.

3.3.2 - Modellers

If you are a modeller, you are responsible for the overall implementation of a modelling study from initial conceptualisation through to analysis and reporting. You are likely to be an economist, epidemiologist or statistician and are probably reasonably comfortable with writing analysis scripts in statistical software (potentially including R), without necessarily being a coding wizard.

Role

The primary role of modellers in ready4 modelling projects is to use modules to undertake analyse as part of modelling projects.

Benefits of using ready4

We hope that ready4 can be of benefit to you by helping you to efficiently build on work by other modellers, to implement more reproducible workflows, and to share your work so that it can be reused.

Contributing to ready4

The types of contribution you can make to ready4 include:

• leading or participating in a modelling project;
• enhancing technical documentation and help; and
• providing advice and feedback.

3.3.3 - Planners

If you are an planner, you contribute to policy development or service planning to help immprove the mental health of young people. You probably value the role of modelling to inform your work, but are likely to rely on others to provide much of the technical expertise to implement computational models.

Role

The primary role of planners in ready4 modelling projects is co-design of decision aids that provide user-interfaces for allow easy configuration of models to run bespoke analyses.

Benefits of using ready4

We hope that ready4 can provide you with accountable, reusable and updatable decision support.

Contributing to ready4

The types of contribution you can make to ready4 include:

• leading or participating in a modelling project;
• providing advice and feedback; and
• funding a project.

3.4 - Stakeholders

3.4.1 - Funders

There are six main types of funder that can provide cash and/or in kind support to ready4:

1. Grant making research bodies can support modelling project proposals submitted to their existing funding schemes. These types of funder could also consider making a number of changes to how they work including the assessment weightings and levels of financial support given to the reproducibility, replicability and transferability components of research proposals and initiating targeted calls for proposals to improve the accountability, flexibility and maintenance of models to inform policy.

2. Government departments can support the development of ready4 as part of programs to enhance data analysis and modelling capability in youth mental health by providing support to develop core ready4 infrastructure (e.g. our software maintenance and community development priorities). When commissioning new modelling projects, governments could make providing open access to code and (to the greatest extent feasible, balancing confidentiality considerations) data a requirement of all applicants.

3. Youth mental health service commissioners can commission data analysis and modelling projects that develop novel decision aids and to apply existing ready4 modules to undertake new analyses.

4. Philanthropic donors can help accelerate our development and enhance our impact by supporting us to bring our existing in-development software to launch and to further extend the ready4 model.

5. Corporate sponsors can provide cash, expertise and free product licenses to support both our core open-source infrastructure and individual modelling projects.

6. Individual givers can provide support by donating to Orygen (please remember to specify www.ready4-dev.com as the reference for the project you would like to support!).

3.4.2 - Researchers

Researchers in multiple discipline enhance prior, current or planned future projects related to how economic, environmental, service, social and technical systems shape young people’s mental health by using ready4 to:

• label and share non-confidential data (e.g. tables of summary statistics / parameter values) from prior and current work to extend access and impact;
• working with modellers to undertake modelling analyses;
• working with coders to redevelop study analysis scripts by authoring modules that are transferable to other contexts;
• supervising student projects to assist with ready4’s current priorities; and/or
• develop research proposals that incorporate a modelling project.

Researchers considering using ready4 should ensure they understand the development status of the tools they wish to use. If the required software is not yet a production release (a process we are working on!) we’d suggest only using it for testing or exploratory work that is not designed to inform decision making. All our software is free and open source so you don’t need to ask our permission to use it. We are however, very happy to discuss ideas for potential collaborations - contact the project lead to arrage a chat.

We also welcome advice from researchers about how we can make ready4 more relevant and useful.

3.4.3 - Young people

Young people have an important role to play in both the development of the overarching ready4 model and the applicability of ready4 to specific decision problems.

One of the main contributions that young people can make is to provide advice. To date, the advice we have elicited from young people has related to shaping the design and conduct of individual modelling projects. The process we have previously used to engage young people in modelling projects normally begins with the advertisement for expressions of interest via a range of social media platforms (always including those maintained by Orygen). We plan to supplement these opportunities to shape individual project with opportunities to shape the overall development of the ready4 model though growing a ready4 support community.

3.5 - Software

3.5.1 - System requirements

Currently, all ready4 software is written in R (for libraries), R Markdown (for programs and sub-routines) and JavaScript (for the user interface component of Shiny applications). Therefore:

• to use ready4 libraries and programs / subroutines you must have an up to date version of R installed on your machine and it is recommended that you install the RStudio IDE; and

• the requirements for using ready4 user interfaces depend on whether you are running a version we have deployed to the web (in which case you just need a supported browser) or whether you are running the app on your local machine (in which case you will need both R and RStudio).

3.5.2 - Code repositories

Currently:

• the most up to date development releases for all ready4 software is available in the ready4-dev GitHub organisation;

• citable and uniquely identified (with a DOI) archived releases of ready4 software are available in the ready4 Zenodo community and are indexed in OpenAire Explore; and

• ready4 user interfaces that have web-based deployments are hosted on Shinyapps.io.

We have yet to issue production releases of any of our R packages, but once these are finalised we will be submitting them to the Comprehensive R Archive Network.

3.5.3 - Release statuses

3.5.3.1 - Unreleased code

Currently, a new ready4 software project initiated by the ready4 core team will by default be made public as a pre-release version in the ready4 GitHub organisation. However, there are some important exceptions. Principally, these exceptions relate to code that we authored in the initial phase of ready4’s development to which some or all of the following apply:

• the code is highly unstable because it has not been (fully) updated to account to major changes implemented in core dependencies;
• the code uses outdated naming conventions and is potentially confusing when used in conjunction with other elements of the ready4 suite; and/or
• the code repository has yet to be cleansed of artefacts that are not yet appropriate for public dissemination (e.g. renders of draft scientific manuscripts).

Depending on which of the above issues apply to a code repository, that repository will either be:

• a private repository (not accessible to anyone outside the core development team); or
• a public repository stored in a location other than the ready4 GitHub organisation.

3.5.3.2 - Development releases

A complete record of all publicly available versions of a ready4 software project’s code over its entire development history (including the most up to date version) is stored in the ready4 GitHub organisation. We refer to these comprehensive publicly available source code resources as “development releases” (even though these records will include versions of our code that we have not formally labelled as “releases”).

Public access to development releases allows individuals to install, test and preview code in advance of production versions being released. Development releases also provide transparency as to who contributed what to a software project and when these contributions were made. Accessing the latest development version of the code is particularly useful to people who wish to contribute bug fixes or new features to our code.

Limitations of development releases include the likelihood that some or all of this code may be inadequately documented or tested. In peer reviewed publications, it is generally considered preferable to avoid citing the copies of code stored in GitHub repositories as these repositories are impermanent (they can be moved, renamed or deleted at any time).

3.5.3.3 - Production releases

Production releases of our code are intended for end-users and signal that they have undergone a number of quality assurance checks and have some supporting documentation.

We have yet to make any production releases of ready4 software, but plan to do this in 2023. Production releases of ready4 R packages will be submitted to CRAN. Unless and until a software item is submitted to a production code repository like CRAN, the recommended platform from which to install our software is that software’s GitHub repository.

3.5.3.4 - Archived releases

Software items that we have formally issued as “releases” are archived as permanent, uniquely identified (with DOI) and citable records within the ready4 Zenodo community. Archived releases of ready4 software are useful as they are snapshots of a project at key milestones in its development (e.g. at the time an analysis was undertaken). As these milestones are purposely selected, archived releases are more likely to have undergone some testing and documenting prior to being released than code not selected for release.

A limitation of archived code libraries is that a greater knowledge of R is required to appropriately install R packages from Zenodo compared to the simpler installation process for the versions of code libraries stored on either GitHub or CRAN.

3.5.4 - Code libraries

3.5.4.1 - Current libraries

3.5.4.1.1 - Framework libraries

The two types of framework library are:

• - the foundational ready4 module and syntax; and

• - tools to implement standardised, semi-automated workflows for authoring and documenting computational models.

Currently available framework libraries are summarised below.

Type	Package	Purpose	Documentation	Code	Examples
		Implement a Modular, Open Source Computational Model of Youth	Citation , Website , Manual - Short (PDF) , Manual - Full (PDF)	Dev , Archive	1, 2, 3, 4
		Retrieve, Label and Share Ready4 Datasets	Citation , Website , Manual - Short (PDF) , Manual - Full (PDF)	Dev , Archive	5, 6, 7
		Author Literate Programs to Implement and Report Ready4 Analyses	Citation , Website , Manual - Short (PDF) , Manual - Full (PDF)	Dev , Archive	8
		Author R Packages of Ready4 Model Modules	Citation , Website , Manual - Short (PDF) , Manual - Full (PDF)	Dev , Archive	9
		Author and Document Functions to Implement Ready4 Algorithms	Citation , Website , Manual - Short (PDF) , Manual - Full (PDF)	Dev , Archive	10
		Author Ready4 Model Modules	Citation , Website , Manual - Short (PDF) , Manual - Full (PDF)	Dev, Archive	11

3.5.4.1.2 - Model module libraries

Computational models developed with ready4 are intended to be both transferable (they are tools that can be used in multiple decision contexts) and modular (they are comprised of self-contained components, each of which performs a narrow sub-set of tasks). For these reasons, ready4 computational models are developed and distributed as libraries of modules.

The three types of computational module libraries are:

• - modules for describing and quality assuring model data;

• - modules to specify, assess and report statisitical models; and

• - modules for making predictions.

Currently available libraries of computational model modules are summarised below.

Type	Package	Purpose	Documentation	Code	Examples
		Describe and Validate Ready4 Person Record Datasets	Citation , Website , Manual - Short (PDF) , Manual - Full (PDF)	Dev , Archive	1, 2
		Score Ready4 Model Datasets	Citation , Website , Manual - Short (PDF) , Manual - Full (PDF)	Dev , Archive	3, 4
		Model Youth Choice Behaviours with Ready4	Citation , Website , Citation	Dev , Archive
		Implement Transfer to Utility Mapping Algorithms	Citation , Website , Manual - Short (PDF) , Manual - Full (PDF)	Dev , Archive	5
		Explore and Characterise Heterogeneity in Quality of Life Data	Citation , Website , Manual - Short (PDF) , Manual - Full (PDF)	Dev , Archive
		Specify Inverse Problems to Solve with Ready4	Citation , Website , Manual - Short (PDF) , Manual - Full (PDF)	Dev , Archive	6
		Transform Youth Outcomes to Health Utility Predictions with Ready4	Citation , Website , Manual - Short (PDF) , Manual - Full (PDF)	Dev , Archive	7

3.5.4.2 - Code library documentation

All ready4 code libraries have:

• a website, the URL of which typically takes the form of "https://ready4-dev.github.io/PACKAGE_NAME" (for example the website for the foundational ready4 R package is https://ready4-dev.github.io/ready4 );

• a brief PDF manual itemising the subset of classes, methods and functions that will be of interest to most end users, the URL of which takes the form of "https://github.com/ready4-dev/PACKAGE_NAME/releases/download/Documentation_0.0/PACKAGE_NAME_User.pdf" (for example, https://github.com/ready4-dev/ready4/releases/download/Documentation_0.0/ready4_User.pdf);

• a more comprehensive PDF manual itemising all included classes, methods and functions, including those that may only be of interest to developers of ready4 libraries, the URL of which takes the form of "https://github.com/ready4-dev/PACKAGE_NAME/releases/download/Documentation_0.0/PACKAGE_NAME_Developer.pdf" (for example, https://github.com/ready4-dev/ready4/releases/download/Documentation_0.0/ready4_Developer.pdf); and

• a publicly accessible record of the results of all continuous integration checks performed on each version of the source code, the url of which takes the form of "https://github.com/ready4-dev/PACKAGE_NAME/actions" (for example, https://github.com/ready4-dev/ready4/actions).

All ready4 code libraries include interactive help. Once you have installed and loaded a library, you can view its contents by using the command library(help="PACKAGE_NAME") (for example, library(help="ready4")).

Note that the manuals and files used by the interactive help are currently all automatically authored by tools from our ready4fun package and are therefore quite basic (and in some cases use clumsy English). In the future we hope to augment this machine generated documentation with human-authored documentation.

Most ready4 libraries (the exceptions are those at very early stages of development) have one or more vignette articles that provide examples of how to use it. These are available from the “Articles” section of each library’s website.

3.5.4.3 - Dependencies

As an open-source project, ready4 depends on the software created and shared by others. Using the DependenciesGraphs R package, we have created the Shiny app below to:

• explore the inter-dependencies between ready4 libraries;
• highlight how our software depends on other R packages;
• itemise the contents each ready4 library;
• display function help files; and
• map function inter-dependencies across multiple ready4 libraries.

To use the app, choose one of the two potential pathways:

• For Pathway 1, start at Step 1 (choose the libraries you wish to profile from the drop down menu and click on the Go button), before proceeding to Step 2 (click on one library that you wish to view the contents of), then Step 3 (click on the view functions button) and finally Step 4 (click on the function for which you would like to view documentation);
• For Pathway 2, start at Step 1 (choose libraries from the drop down menu), then Step 2 (click on the Find functions button), then Step 3 (select functions from the drop down menu) and finally Step 4 (click on the Make graph button).

Note, as the app is displayed on this page via an iFrame, it may be difficult to view on a phone. If so, you can try the following link: https://orygen.shinyapps.io/dependencies/

3.5.4.4 - Installation and set-up

ready4 libraries are currently only available as development releases, so you will need to use a tool like devtools to assist with installing ready4 R packages directly from our GitHub organisation. If you do not have devtools on your machine you can install it with the following command.

utils::install.packages("devtools")

3.5.4.4.1 - Installing the ready4 framework foundation library

Before you install

If you plan to use ready4 for any purpose, you will need to install the ready4 foundation library.

However, please note that the ready4 library is not yet available as a production release. You should therefore understand the limitations of using ready4 software development releases before you make the decision to install this software.

As all software in the ready4 suite depends on the ready4 library, so in most cases you do not need to install this library directly (it will come bundled with whatever other ready4 suite software you install).

If you can run the following command without producing an error message, then you already have it.

find.package("ready4")

Installation

You can install the ready4 library directly from its GitHub repository.

devtools::install_github("ready4-dev/ready4")

Try it out!

Before you apply ready4 tools to your own project, you should make sure you can run some or all of the example code included in the package vignettes.

3.5.4.4.2 - Installing authoring tools

3.5.4.4.2.1 - Installing tools for authoring model modules

Before you install

If you are a coder planning on using ready4 to author model modules, then you may wish to install the ready4class, ready4fun and ready4pack libraries.

However, please note that none of these libraries are yet available as a production release. You should therefore understand the limitations of using ready4 software development releases before you make the decision to install this software. We use these authoring tools intensively to help us write highly standardised model modules. However, we feel that these tools are most likely to be helpful to you once much more comprehensive documentation and training resources become available. Without this training and support, the requirements for complying with our house-style, file-naming, directory structure and workflow standards are unlikely to be sufficiently clear. We will be making these improvements, but for the mean time we recommend that, if you wish to use these authoring tools, you first get in touch with the project lead.

Installation

As ready4class and ready4fun are bundled as dependencies of ready4pack, you can install all three from our GitHub organisation using one command.

devtools::install_github("ready4-dev/ready4pack")

Configuration

To use these computational model authoring tools, you will need to have set-up and appropriately configured your own accounts in:

• GitHub (you will need write permissions to a GitHub organisation and to then enable GitHub actions and GitHub pages support for the repositories you create in that organisation);
• Zenodo (you will need to have linked each GitHub repository used for your ready4 projects to your Zenodo account); and
• Codecov (linked to your GitHub organisation).

The machine onto which you install ready4pack will also need to be securely storing your GitHub credentials (i.e. the value for the GITHUB_PAT token).

Try it out!

It should be noted that the development workflow supported by our computational model authoring tools is not yet well documented. We don’t recommend undertaking R package development with these tools until this has been rectified. However, if you still want to try these tools out, the best place to start is review the examples in the ready4class, ready4fun and ready4pack vignettes.

3.5.4.4.2.2 - Installing tools for authoring and managing model datasets

Before you install

If you are a coder or modeller planning to create, share and access model datasets with ready4, then you will need the ready4use library.

However, please note that ready4use is not yet available as a production release. You should therefore understand the limitations of using ready4 software development releases before you make the decision to install this software.

You may already have ready4use installed on your machine (e.g. if you have previously installed other ready4 framework and module libraries that include ready4use as a dependency). If you can run the following command without producing an error message, then you already have it.

find.package("ready4use")

Installation

You can install ready4use directly from its GitHub repository.

devtools::install_github("ready4-dev/ready4use")

Configuration

If one of your intended uses of ready4use is to share outputs in online datasets, you will need to have set up an account on a Dataverse installation (we recommend using the Harvard Dataverse). Some of the key terms and concepts relating to using a Dataverse installation in conjunction with ready4use are described in this tutorial.

You need to ensure that you have write permissions to any Dataverse Datasets that you plan to use to post files to. Furthermore, the machine on which you install ready4use should also securely store your Dataverse account credentials (specifically, values for the DATAVERSE_KEY and DATAVERSE_SERVER tokens). Details of how to do this are described in documentation for the dataverse R package, an important third party dependency package for ready4use.

Try it out

You should now be able to run the example code included in the package vignettes. To run all of this code you will need to replace the details of the Dataverse Dataset to which files are being written to those of your own Dataverse Dataset.

3.5.4.4.2.3 - Installing tools for authoring reproducible analyses

Before you install

If you are a coder or modeller planning to implement a reproducible analysis with ready4, you will need to install the ready4show library.

However, please note that ready4show is not yet available as a production release. You should therefore understand the limitations of using ready4 software development releases before you make the decision to install this software.

If you have installed other ready4 libraries, then ready4show may have already been installed as a dependency. If you can run the following command without producing an error message, then you already have it.

find.package("ready4show")

Installation

The ready4show library can be installed directly from its GitHub repository.

devtools::install_github("ready4-dev/ready4show")

Try it out!

Before you apply ready4show tools to your own project, you should make sure you can run some or all of the example code included in the package vignettes.

3.5.4.4.3 - Installing ready4 computational model modules

Before you install

If you plan on using existing ready4 modules for a modelling project, you can review currently available module libraries, to identify which libraries are relevant to your project.

However, please note that no ready4 module library is yet available as a [production release](/docs/getting-started/software/status/production-releases/. You should therefore understand the limitations of using ready4 software development releases before you make the decision to install this software.

Installation

The command to install each ready4 module takes the following format.

devtools::install_github("ready4-dev/PACKAGE_NAME")

For example, if you are planning to predict health utility using some of the mapping algorithms that we have previously developed, you can install the youthu library with the following command.

devtools::install_github("ready4-dev/youthu")

Configuration

A small number of ready4 modules require that you configure some of the dependencies installed with them before they can be used. In particular:

• if you are using modules from the TTU package to undertake a utility mapping study, you will need to have both installed and configured the cmdstanr R package as per the instructions on that package’s documentation website; and

• if you are using the mychoice package to undertake a discrete choice experiment study and are using a Mac, you need to ensure that you have a Fortran compiler installed. Some relevant advice on this: https://mac.r-project.org/tools/ .

Try it out!

Before you apply ready4 modules to your own project, you should make sure you can run some or all of the example code included in relevant library vignette articles. The package website URL takes the form of https://ready4-dev.github.io/PACKAGE_NAME/articles/ (e.g. the vignettes for the youthvars package are available at https://ready4-dev.github.io/youthvars/articles/).

3.5.5 - Executables

Currently all ready4 programs and subroutines are written in R Markdown. Each ready4 program and subroutine depends on at least one ready4 framework library as well as one or more ready4 module libraries. The required libraries will vary based on the purpose of the program. ready4 programs and subroutines typically generate reporting documents in file formats such as PDF, Word and HTML.

3.5.5.1 - Programs

What are ready4 programs?

Programs can be executed in their current form without the need for additional input data and, unless modified or run interactively (prompting a user for inputs during execution), will always generate the exact same output. They are typically deployed for configuring the run specifications of a computational model, specifying the data to which it will be applied and reporting analysis results.

Why are they useful?

ready4 programs can be used for the following purposes:

• to reproduce a study analysis, in which case you will need access to the original study data, and may also need to modify the program to specify the path to this data from your machine;
• to replicate a study analysis (ie to apply the study algorithm to similar but different input data [this can be a new sample from the same population or, if used for demonstration purposes, fake data representative of the original study dataset]), in which case you will need to modify the program to specify the path to this data; and
• to transfer a study analysis, in which case you use the program as a template to be modified to reflect key differences between the original study and your study.

Current ready4 programs

Currently available ready4 programs are summarised in the below table.

Documentation

ready4 programs are typically self-documenting, meaning that each section of code is integrated with plain English descriptions of the purpose it fulfills. The only programs that are not self-documenting are those whose primary purpose is to produce a document (normally an analysis report). Self-documenting programs and sub-routines will be typically documented as a PDF or HTML render of the RMarkdown source file. This rendered document will be bundled with the program, but in some cases may also be shared in online data repositories.

3.5.5.2 - Subroutines

What are ready4 subroutines?

Sub-routines need to be called by parent programs that supply them with input data. Sub-routines can be called by multiple programs and will produce output that varies based on the input values they are supplied with. They are typically deployed to implement parts of a model’s analysis and reporting algorithm.

Why are they useful?

ready4 subroutines can be used for the following purposes:

• to help execute a program or function written by a third party (in which case you probably won’t need to modify the subroutine and may not even be aware that it is being used);
• to help execute a program or function that you write (in which case, you shouldn’t have to modify the subroutine, but may find it useful to customise it to your purposes); and
• to serve as a template for subroutines you write yourself that perform similar tasks (in which case you will be rewriting the subroutine’s code).

Current ready4 subroutines

Currently available ready4 subroutines are summarised in the below table.

Documentation

ready4 programs are currently minimally documented, typically in the form as notes contained in a README file in the source code bundle.

3.5.6 - User interfaces

ready4 user-interfaces (UIs) enable individuals, especially planners, to explore, configure and use ready4 tools without needing to enter any computer code. Our UIs are typically deployed as Shiny apps an may have deployments via the web (accessed through your browser) or to your desktop (installed as part of a ready4 library).

The main purposes of ready4 user-interfaces are:

• to make it easy for non-technical users to configure a computational model and apply it to selected input data; and

• as an accessible and interactive means of demonstrating key concepts relating to ready4.

The user interface for exploring the dependencies of ready4 libraries is an example of the latter use.

We do not have any current releases of ready4 user interfaces for running models. However, an old and deprecated user interface for running the Springtides model is available for purely illustrative purposes.

3.5.7 - Terms of use

3.5.7.1 - Open source licensing

To help ensure the models we develop are as transparent as possible and to make their algorithms as useful to others as possible, all ready4 software is free and open-source. You are encouraged to make as widespread use of our software, including the creation of derivative works, as you see fit, so long as it is consistent with each item’s license. Our software is typically licensed under GPL-3, a copy-left open-source licensing regime.

3.5.7.2 - Citing ready4

To make it easier to cite our software, each software item bundle includes a CITATION.cff file. Inclusion of this file means that the repositories storing our software can generate appropriate citations in the format of most relevance to you.

Currently:

• Zenodo provides a free text field under the heading “Cite as” which enables you to generate a wide range of citation manager and journal specific citation outputs. There is also an “Export” tool that will generate citation metadata in multiple output formats;
• OpenAire Explore has a “Cite this software” button that allows you to generate a citation in multiple journal formats or to download BibTeX or RIS files;
• Github repositories have a “Cite this repository” button that can generate both BibTeX and APA output as well as link to the Citation.cff file.

Additionally, we have included a CITATION file in each of our R libraries so that you can generate a citation from within an R session using the citation function (for example: citation("ready4").

3.5.7.3 - Disclaimer

All ready4 software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Furthermore, no ready4 software is yet sufficiently well documented and tested to be given a production release. All ready4 software should therefore viewed as experimental development releases.

4 - Framework

4.1 - Standards

An essential component of the ready4 framework is standardisation. Defining and adhering to a common set of standards is important because:

• explicit, measurable standards are good modelling practice;

• to implement a modular computational model, ready4 model modules need to be interoperable (ie they can easily and safely be combined);

• as an open source project with multiple users and contributors, consistency in implementation and documentation facilitates collaboration and ease of use;

• standardised workflows are easier to partially automate.

We have therefore developed a set of standards that we believe support good practice in the development of open source computational models. We are describing these standards, their rationale and how ready4 implements them, in a manuscript being prepared for publication. When it is publicly available we will provide a link in an updated version of this page.

4.2 - Implementation

4.2.1 - Paradigms

4.2.1.1 - Why ready4 is object oriented

library(ready4)

Motivation

The practical utility and ease of use of computational models of mental health systems are in part shaped by the choice of programming paradigm used to develop them. ready4 adopts an object oriented programming (OOP) paradigm which in practice means that the framework principally consists of classes (representations of data structures useful for modelling mental health systems) and methods (algorithms that can be applied to these data-structures to generate insight useful for policy-making). Adopting an OOP approach is particular useful for making the ready4 model both modular and transparent.

Implementation

Modular Computational Models

Two commonly noted features of OOP - encapsulation and inheritance are particularly useful when developing modular computational models.

Encapsulation

Encapsulation allows us to define the data structures (“classes”) used in computational modelling projects in a manner that allows them to be safely combined. For example, assume there are two computational models, one (A) focused on predicting the types and intensity of services used by individuals that present to mental health services and the other (B) that predicts outcomes for recipients of these services. It may be desirable to develop a new model (C) that combines A and B to model both service use and outcomes. Using encapsulated code allows all of the features and functionality of A can be made available to B in a manner that protects the integrity of A. Specifically, B can only interact with A using the algorithms (“methods”) that have been already defined for A.

Furthermore, if appropriately implemented, methods associated with a class will work with any combination of input values that can be encapsulated by that class - making computational models more transferable. For example, imagine a class (X) that is used to structure summary data relevant to mental health systems. Methods associated with X (e.g. a method to derive an unmet need statistic) can then applied to two instances of X - one containing data relevant to the Australian context and one with data from the UK context.

Inheritence

The examples highlighted in the previous section have some potential limitations. What if the developers of A didn’t define methods that would allow B to interact with it in the desired way? Or what if there are a number of differences between the Australian and UK system that need to be accounted for when transfering a method from the former to the latter? These types of issues can be addressed by another key feature of OOP - inheritance. Inheritance allows for a “child” class to be created from a “parent” class. By default, the “child” inherits all of the features of the “parent” including all methods associated with the “parent” class. Importantly however, alternative or additional features can also be specified for the “child” to allow it to implement different methods where necessary. For example, when developing our new computational model C we could create a number of new classes that are children of the classes defined in A. We can then define any additional/alternative methods for these classes that overcome any integration issues between the classes and methods of A and B. In this way, we can enjoy the best of both worlds - leveraging all relevant algorithms from A and B (as there is no need to re-invent the wheel), while ensuring that we transparently develop the additional code required for C. This approach also ensures that the respective contributions of the (potentially different) authorship teams behind A, B and C is clearer.

Similarly, inheritance would allow re-use of much of the code from a model of the Australian mental health system when exploring similar topics within the UK context, while making it straightforward to develop additional code that addresses relevant divergence in features between the two jurisdictions. In practical terms, this would mean developing two child classes of X - class Y for use with Australian data and class Z for use in the UK system. All methods that are not specific to a particular jurisdiction are defined for X and inherited by Y and Z. Methods that are only appropriate for use in the Australian context are defined for Y, while UK specific methods are defined for Z.

Transparent Computational Models

To make analyses implemented using the ready4 model more readily understood, the ready4 package provides the model’s simple and consistent syntax. Such simplified approaches are facilitated by two other commonly noted features of OOP - polymorphism and abstraction.

Polymorphism

Polymorphism allows for similar concepts to be represented using consistent syntax. The same top level code can therefore be transferred to multiple model implementations, making algorithms simpler to understand and easier to re-use.

Returning to a previous example, the exact same command (e.g. a call to the method exhibit) can be applied to both Y (used for Australian data) and Z (used for UK data). However, the algorithm implemented by that command can vary based on the class that each method is applied to (ie a different algorithm is applied when the data is specified as being from the UK compared to being specified as Australian).

Abstraction

The simplicity enabled by polymorphism is enhanced by Abstraction, which basically means that only the briefest and easiest to comprehend parts of the code are exposed by default to potential users. Once an instance of a class is created, the entire program to ingest model data, analyse it and produce a scientific summary can be represented in a few brief lines of code, readily comprehensible to non-coders. When using open source languages, the elegance and simplicity of abstraction does not restrict the ability of more technically minded users exploring the detailed workings of the underpinning code.

4.2.1.2 - The role of functional programming in ready4 development

Although the object-oriented programming (OOP) approach ready4 implements has many advantages, it can also have some limitations. Some of these limitations have been colorfully highlighted by a popular quote attributed to Joe Armstrong:

“The problem with object-oriented languages is they’ve got all this implicit environment that they carry around with them. You wanted a banana but what you got was a gorilla holding the banana and the entire jungle.”

In practical terms, this means that if not carefully planned, using OOP can create barriers to code-reuse as algorithms come bundled with artefacts of no/low relevance to many potential users. To help maximise the accessibility and re-usability of ready4 algorithms, these algorithms are primarily written using the functional programming paradigm. Only once an algorithm has been implemented using functions are they then linked to a data-structure by means of a calling method. The typical development workflow for a ready4 computational modelling project might therefore look something the following three step process:

1. A modelling study algorithm is implemented as a program.

2. To help transfer the methods used in the study algorithm, it is decomposed into functions, which are bundled as a code library (or libraries). The program is updated to use the newly authored functions.

3. A ready4 module is authored to define a data-structure along with a method (or methods) that call the functions to implement the transferable version of the study algorithm. The new module is added to the previously created code library and the program is again updated so that the algorithm is now implemented by supplying data to the ready4 module and then calling the desired method(s).

Modellers using ready4 for the most part will only use ready4 modules and will rarely interact directly with the functions that implement module methods. However, these functions are potentially of significant usefulness to coders authoring new algorithms. A helpful way of exploring currently available functions is to use the ready4 dependencies app. All ready4 functions are created with minimal, but consistent documentation with the aid of tools from the ready4fun library.

4.2.2 - Modularity

library("ready4") 

Motivation

A potentially attractive approach to modelling complex youth mental health systems is to begin with a relatively simple computational model and to progressively extend its scope and sophistication. Such an approach could be described as “modular” if it is possible to readily combine multiple discrete modelling projects (potentially developed by different modelling teams) that each independently describe distinct aspects of the system being modelled. This modular and collaborative approach is being used in the development of ready4 - an open source health economic model of the systems shaping mental health and wellbeing in young people. The ready4 package provides the foundational tools to support the development and application of the ready4 modular model.

Implementation

The ready4 model is being implemented in R and its modular nature is enabled by the encapsulation and inheritance features of Object Oriented Programming (OOP). Specifically, ready4 uses two of R’s systems for implementing OOP - S3 and S4. An in-depth explanation of R’s different class system is beyond the scope of this article, but is explored in Hadley Wickham’s Advanced R handbook. However, it is useful to know some very high level information about S3 and S4 classes:

• S4 classes are frequently said to be “formal”, “strict” or “rigorous”. The elements of an S4 class are called slots and the type of data that each slot is allowed to contain is specified in the class definition. An S4 class can be comprised of slots that contain different types of data (e.g. a slot that contains a character vector and another slot that contains tabular data).

• S3 classes are often described as “simple”, “informal” and “flexible”. S3 objects attach an attribute label to base type objects (e.g. a character vector, a data.frame, a list), which in turn is used to work out what methods should be applied to the class.

ready4 Model Modules

A ready4 model module is a data-structure and associated algorithms that is used to model a discrete component of a system relevant to young people’s mental health. Each ready4 model module is created using the ready4 package’s Ready4Module class. We can create an instance (X) of Ready4Module using the following command.

X <- ready4::Ready4Module()

However, if we inspect X we can see it is of limited use as it contains no data other than an empty element called dissemination_1L_chr.

str(X)
#> Formal class 'Ready4Module' [package "ready4"] with 1 slot
#>   ..@ dissemination_1L_chr: chr NA

The Ready4Module class is therefore not intended to be called directly. Instead, the purpose of Ready4Module is to be the parent-class of all ready4 model modules. Ready4Module and all of its child-classes (ie all ready4 model modules) are “S4” classes.

ready4 includes two child classes of Ready4Module. These are Ready4Public and Ready4Private and both are almost as minimally informative as their parent (the only difference being that their instances have the values “Public” or “Private” assigned to the dissemination_1L_chr slot).

Y <- Ready4Public()
str(Y)
#> Formal class 'Ready4Public' [package "ready4"] with 1 slot
#>   ..@ dissemination_1L_chr: chr "Public"

Z <- Ready4Private()
str(Z)
#> Formal class 'Ready4Private' [package "ready4"] with 1 slot
#>   ..@ dissemination_1L_chr: chr "Private"

Like the Ready4Module class they inherit from, the purpose of Ready4Public and Ready4Private is to be used as parent classes. Using either of Ready4Public and Ready4Private can be a potentially efficient way of partially automating access policies for model data. If all the data contained in a module can always be shared publicly, it may be convenient to note this by using a module that has been created as a child-class of Ready4Public. Similarly, if at least some of the data contained in a module will always be unsuitable for public dissemination, it can be useful to use a module that is a child of Ready4Private. When the dissemination policy for data contained in a module will vary depending on user or context, it is more appropriate to use a module that inherits from Ready4Module without being a child of either Ready4Public and Ready4Private. In this latest case, users may choose to add descriptive information about the data access policy themselves using the renewSlot method. The dissemination policy can be inspected with the procureSlot method.

X <- renewSlot(X,
               "dissemination_1L_chr",
               "Staff and students of research institutes")
procureSlot(X,
            "dissemination_1L_chr")
#> [1] "Staff and students of research institutes"

ready4 Model Sub-modules

In ready4, S3 classes are principally used to help define the structural properties of slots (array elements) of model modules and the methods that can be applied to these slots. S3 classes created for these purposes are called sub-modules.

Module and Sub-module Methods

All methods associated with ready4 modules and sub-modules adopt a common syntax. However, the algorithms implemented by each command in that syntax will vary depending on which module it is applied to. A limited number of methods are defined at the level of the Ready4Module parent class and are therefore inherited by all ready4 modules. Currently, the only methods defined for Ready4Module are slot-methods and these can be itemised using the get_methods function.

get_methods()
#>  [1] "authorSlot"        "characterizeSlot"  "depictSlot"       
#>  [4] "enhanceSlot"       "exhibitSlot"       "ingestSlot"       
#>  [7] "investigateSlot"   "manufactureSlot"   "metamorphoseSlot" 
#> [10] "procureSlot"       "prognosticateSlot" "ratifySlot"       
#> [13] "reckonSlot"        "renewSlot"         "shareSlot"

4.2.3 - Syntax

library("ready4") 

Motivation

Transparency is one of the underpinning principles of open science. One way to improve the transparency of the ready4 model is to ensure that the programs implementing analyses using this model can be meaningfully inspected by readers with different levels of technical expertise. Even non-technical readers should be able to follow the high-level logic implemented by model algorithms. By using a simple programming syntax that can be consistently used across all model analyses programs, ready4 can help ensure that readers need to contend with relatively few new concepts when reviewing analysis code.

Implementation

ready4 provides a simple syntax that can be consistently applied to all ready4 model modules. It does so by taking advantage of the polymorphism and abstraction features of Object Oriented Programing and R’s use of generic functions. Generic functions don’t obviously do anything by themselves - their most salient features are a name and a high level description of the type of task that any method using that name should perform. Whenever a method is defined for classes that use R’s S4 and S3 systems (the types used for ready4 model modules and sub-modules), it is assigned to the generic that is the best match for the algorithm it implements.

Finding ready4 Methods

A table that summarises the syntax used by ready4 model module methods, can be generated by web-scraping using make_methods_tb (which produces up to date results but can be a little slow to excecute) or alternatively be downloaded from a periodically updated database using get_methods_tb (which is quicker to implement, but may miss the most recent additions).

# Not run
# x <- make_methods_tb()

x <- get_methods_tb()

Core generics

ready4 includes a number of core generic functions which describe the main types of method to be implemented by ready4 model modules. Notably, the ready4 package does not define methods for any of these core generics. Instead, methods are defined for these generics in R packages that contain ready4 modules. A HTML table of the core generics bundled with ready4 and examples of methods that implement each generic can be displayed using the print_methods function, using the return_1L_chr = "core" argument.

print_methods(x,
              return_1L_chr = "core",
              scroll_width_1L_chr = "100%") 

Slot generics and methods

Each of the “core” generics also has a “slot” version, for use when applying a core method to a specified slot of a class. The ready4 package defines methods for each of these “slot” generics for the Ready4Module class. Two of these “slot” methods can also be used for additional purposes:

• procureSlot is a “getter” method - its default behaviour is to return the value of a specified slot. If the argument use_procure_mthd_1L_lgl = T is included in the method call, procureSlot will instead apply the procure method to a specified slot.

• renewSlot is a “setter” method - if any value other than “use_renew_mthd” (the default) is passed to the new_val_xx argument, that value will be assigned to the specified slot.

A HTML table of the slot generics bundled with ready4 can be displayed using the print_methods function, using the return_1L_chr = "slot" argument.

print_methods(x,
              return_1L_chr = "slot",
              scroll_width_1L_chr = "100%")

Extended author generics

Finally, there are a small number of other generics that are more general extensions of the core functions. Currently, these extended generics are all variants on the author generics, with each specifying the type of output to be authored by the method. The ready4 package does not include methods for any of these extended generics. A HTML table of the extended generics bundled with ready4 can be displayed using the print_methods function, using the return_1L_chr = "extended" argument.

print_methods(x,
              exclude_mthds_for_chr = "Ready4Module",
              return_1L_chr = "extended",
              scroll_width_1L_chr = "100%")

5 - Model

5.1 - Finding modules and sub-modules

You can search for ready4 modules and sub-modules using tools from the ready4 R package.

library(ready4)

To search for themed collections of modules, you can review the current list of module libraries.

An itemised list of individual ready4 model modules and sub-modules can be generated by scraping the websites of these libraries make_modules_tb function (this may take a couple of minutes).

modules_tb <-  make_modules_tb()

A slightly quicker method to achieve a similar (but potentially less up to date) result is to use the get_modules_tb function.

# Not run
# modules_tb <- get_modules_tb()

If you wish to display the table of modules as HTML, you can use the print_modules function. You can choose to display only ready4 sub-modules (which always use R’s “S3” class type).

print_modules(modules_tb,
              what_1L_chr = "S3")

To display only ready4 modules, restrict returns to R’s “S4” class type.

print_modules(modules_tb,
              what_1L_chr = "S4")

5.2 - Using ready4 modules

5.2.1 - Modules for modelling people

5.2.1.1 - Add metadata to datasets of individual human records

Note: This vignette is illustrated with fake data. The dataset explored in this example should not be used to inform decision-making.

library(ready4)
library(youthvars)

Youthvars provides a two classes - YouthvarsProfile and YouthvarsSeries that are useful for describing features of datasets. The tools in youthvars build on the metadata included in a Ready4useDyad.

Ingest data

To start we ingest X, a Ready4useDyad (dataset and data dictionary pair) that we can download from a remote repository.

X <- ready4use::Ready4useRepos(dv_nm_1L_chr = "fakes",
                               dv_ds_nm_1L_chr = "https://doi.org/10.7910/DVN/W95KED",
                               dv_server_1L_chr = "dataverse.harvard.edu") %>%
  ingest(fls_to_ingest_chr = "ymh_clinical_dyad_r4",
         metadata_1L_lgl = F)

Add metadata

We could add metadata about X, such as the unique identifier variable name, by transforming it to a YouthvarsProfile instance.

## Not run
# X <- YouthvarsProfile(a_Ready4useDyad = X,
#                       id_var_nm_1L_chr = "fkClientID")

However, in this case the data we ingested includes a longitudinal dataset. It is therefore preferable to transform X into a YouthvarsSeries instance. YouthvarsSeries objects contain all of the fields of YouthvarsProfile objects, but also include additional fields that are specific for longitudinal datasets (e.g. timepoint_var_nm_1L_chr and timepoint_vals_chr that respectively specify the data-collection timepoint variable name and values and participation_var_1L_chr that specifies the desired name of a yet to be created variable that will summarise the data-collection timepoints for which each unit record supplied data).

X <- YouthvarsSeries(a_Ready4useDyad = X,
                     id_var_nm_1L_chr = "fkClientID",
                     participation_var_1L_chr = "participation",
                     timepoint_vals_chr = c("Baseline","Follow-up"),
                     timepoint_var_nm_1L_chr = "round")

YouthvarsSeries methods

Currently, only methods for YouthvarsSeries (and not yet YouthvarsProfile) have been included with the youthvars package. These methods are summarised in the following sections.

Validate data

We use the ratify method to ensure that X has been appropriately configured for methods examining datasets reporting measures at two timepoints.

X <- ratify(X,
            type_1L_chr = "two_timepoints")

Inspect data

We can now specify the variables that we would like to prepare descriptive statistics for using the renewSlot and renew methods. The variables to be profiled are specified in arguments beginning with “compare_”. Use compare_ptcpn_chr to compare variables based on whether cases reported data at one or both timepoints and compare_by_time_chr to compare the summary statistics of variables by timepoints, e.g at baseline and follow-up. If you wish these comparisons to report p values, then use the compare_ptcpn_with_test_chr and compare_by_time_with_test_chr arguments.

X <- renewSlot(X,
               "descriptives_ls",
               compare_by_time_chr = c("d_age","d_sexual_ori_s","d_studying_working"),
               compare_by_time_with_test_chr = c("k6_total", "phq9_total", "bads_total"),
               compare_ptcpn_with_test_chr = c("k6_total", "phq9_total", "bads_total")) %>%
  renew(type_1L_chr = "characterize")

The tables generated in the preceding step can be inspected using the exhibit method.

X %>%
  exhibit(profile_idx_int = 1L,
          scroll_box_args_ls = list(width = "100%"))

X %>%
  exhibit(profile_idx_int = 2L,
          scroll_box_args_ls = list(width = "100%"))

X %>%
  exhibit(profile_idx_int = 3L,
          scroll_box_args_ls = list(width = "100%"))

The depict method can create plots, comparing numeric variables by timepoint.

depict(X,
       type_1L_chr = "by_time",
       var_nms_chr = c("c_sofas"),
       label_fill_1L_chr = "Time",#
       labels_chr = c("SOFAS"),#
       y_label_1L_chr = "")
#> Warning: `stat(width * density)` was deprecated in ggplot2 3.4.0.
#> ℹ Please use `after_stat(width * density)` instead.
#> ℹ The deprecated feature was likely used in the youthvars package.
#>   Please report the issue to the authors.

SOFAS total scores by data collection round

Share data

If and only if the dataset you are working with is appropriate for public dissemination (e.g. is synthetic data), you can use the following workflow for sharing it. We can share the dataset we created for this example using the share method, specifying the repository to which we wish to publish the dataset (and for which we have write permissions) in a (Ready4useRepos object).

Y <- Ready4useRepos(gh_repo_1L_chr = "ready4-dev/youthvars", # Replace with your repository 
                          gh_tag_1L_chr = "Documentation_0.0"), # (need write permissions).
Y <- share(Y,
           obj_to_share_xx = X,
           fl_nm_1L_chr = "ymh_YouthvarsSeries")

X is now available for download as the file ymh_YouthvarsSeries.RDS from the “Documentation_0.0” release of the youthvars package.

5.2.1.2 - Validate variable total scores

library(ready4)
library(youthvars)

Variable classes and data integrity

The youthvars package defines a number of vector based classes that can be used to quality assure the data recorded for individual variables. youthvars variable classes are potentially useful for:

1. facilitating automated data integrity checks that verify no impermissible values (e.g. utility scores greater than one) are present in source data, transformed data or results; and
2. automating the selection of the appropriate method to apply to each data type.

Included classes

The initial set of classes included in the youthvars package are one class for Assessment of Quality of Life (Adolescent) health utility and one for each of the predictors used in the utility prediction algorithms included in the related youthu package.

Assessment of Quality of Life Six Dimension (Adolescent) Health Utility

The youthvars_aqol6d_adol class is defined for numeric vectors with a minimum value of 0.03 and maximum value of 1.0.

youthvars_aqol6d_adol(0.4)
#> [1] 0.4
#> attr(,"class")
#> [1] "youthvars_aqol6d_adol"
#> [2] "numeric"

youthvars_aqol6d_adol(c(0.03,0.2,1))
#> [1] 0.03 0.20 1.00
#> attr(,"class")
#> [1] "youthvars_aqol6d_adol"
#> [2] "numeric"

Non numeric objects and values outside these ranges will produce errors.

youthvars_aqol6d_adol("0.5")
#> Error in make_new_youthvars_aqol6d_adol(x): is.numeric(x) is not TRUE

youthvars_aqol6d_adol(-0.1)
#> Error: All non-missing values in valid youthvars_aqol6d_adol object must be greater than or equal to 0.03.

youthvars_aqol6d_adol(1.2)
#> Error: All non-missing values in valid youthvars_aqol6d_adol object must be less than or equal to 1.

Behavioural Activation for Depression Scale (BADS)

The youthvars_bads class is defined for integer vectors with a minimum value of 0 and maximum value of 150.

youthvars_bads(143L)
#> [1] 143
#> attr(,"class")
#> [1] "youthvars_bads" "integer"

youthvars_bads(as.integer(c(1,15,150)))
#> [1]   1  15 150
#> attr(,"class")
#> [1] "youthvars_bads" "integer"

Non-integers and values outside these ranges will produce errors.

youthvars_bads(22.5)
#> Error in make_new_youthvars_bads(x): is.integer(x) is not TRUE

youthvars_bads(-1L)
#> Error: All non-missing values in valid youthvars_bads object must be greater than or equal to 0.

youthvars_bads(160L)
#> Error: All non-missing values in valid youthvars_bads object must be less than or equal to 150.

Generalised Anxiety Disorder Scale (GAD-7)

The youthvars_gad7 class is defined for integer vectors with a minimum value of 0 and a maximum value of 21.

youthvars_gad7(15L)
#> [1] 15
#> attr(,"class")
#> [1] "youthvars_gad7" "integer"

youthvars_gad7(as.integer(c(0,14,21)))
#> [1]  0 14 21
#> attr(,"class")
#> [1] "youthvars_gad7" "integer"

Non-integers and values outside these ranges will produce errors.

youthvars_gad7(14.6)
#> Error in make_new_youthvars_gad7(x): is.integer(x) is not TRUE

youthvars_gad7(-1L)
#> Error: All non-missing values in valid youthvars_gad7 object must be greater than or equal to 0.

youthvars_gad7(22L)
#> Error: All non-missing values in valid youthvars_gad7 object must be less than or equal to 21.

Kessler Psychological Distress Scale (K6) - US Scoring System

The youthvars_k6 class is defined for integer vectors with a minimum value of 0 and a maximum value of 24.

youthvars_k6(21L)
#> [1] 21
#> attr(,"class")
#> [1] "youthvars_k6" "integer"

youthvars_k6(as.integer(c(0,13,24)))
#> [1]  0 13 24
#> attr(,"class")
#> [1] "youthvars_k6" "integer"

Non-integers and values outside these ranges will produce errors.

youthvars_k6(11.2)
#> Error in make_new_youthvars_k6(x): is.integer(x) is not TRUE

youthvars_k6(-1L)
#> Error: All non-missing values in valid youthvars_k6 object must be greater than or equal to 0.

youthvars_k6(25L)
#> Error: All non-missing values in valid youthvars_k6 object must be less than or equal to 24.

Overall Anxiety Severity and Impairment Scale (OASIS)

The youthvars_oasis class is defined for integer vectors with a minimum value of 0 and a maximum value of 20.

youthvars_oasis(15L)
#> [1] 15
#> attr(,"class")
#> [1] "youthvars_oasis" "integer"

youthvars_oasis(as.integer(c(0,12,20)))
#> [1]  0 12 20
#> attr(,"class")
#> [1] "youthvars_oasis" "integer"

Non-integers and values outside these ranges will produce errors.

youthvars_oasis(14.2)
#> Error in make_new_youthvars_oasis(x): is.integer(x) is not TRUE

youthvars_oasis(-1L)
#> Error: All non-missing values in valid youthvars_oasis object must be greater than or equal to 0.

youthvars_oasis(21L)
#> Error: All non-missing values in valid youthvars_oasis object must be less than or equal to 20.

Patient Health Questionnaire (PHQ-9)

The youthvars_phq9 class is defined for integer vectors with a minimum value of 0 and a maximum value of 27.

youthvars_phq9(11L)
#> [1] 11
#> attr(,"class")
#> [1] "youthvars_phq9" "integer"

youthvars_phq9(as.integer(c(0,13,27)))
#> [1]  0 13 27
#> attr(,"class")
#> [1] "youthvars_phq9" "integer"

Non-integers and values outside these ranges will produce errors.

youthvars_phq9(15.2)
#> Error in make_new_youthvars_phq9(x): is.integer(x) is not TRUE

youthvars_phq9(-1L)
#> Error: All non-missing values in valid youthvars_phq9 object must be greater than or equal to 0.

youthvars_phq9(28L)
#> Error: All non-missing values in valid youthvars_phq9 object must be less than or equal to 27.

Screen for Child Anxiety Related Disorders (SCARED)

The youthvars_scared class is defined for integer vectors with a minimum value of 0 and a maximum value of 82.

youthvars_scared(77L)
#> [1] 77
#> attr(,"class")
#> [1] "youthvars_scared" "integer"

youthvars_scared(as.integer(c(0,42,82)))
#> [1]  0 42 82
#> attr(,"class")
#> [1] "youthvars_scared" "integer"

Non-integers and values outside these ranges will produce errors.

youthvars_scared(33.2)
#> Error in make_new_youthvars_scared(x): is.integer(x) is not TRUE

youthvars_scared(-1L)
#> Error: All non-missing values in valid youthvars_scared object must be greater than or equal to 0.

youthvars_scared(83)
#> Error in make_new_youthvars_scared(x): is.integer(x) is not TRUE

Social and Occupational Functioning Assessment Scale (SOFAS)

The youthvars_sofas class is defined for integer vectors with a minimum value of 0 and a maximum value of 100.

youthvars_sofas(44L)
#> [1] 44
#> attr(,"class")
#> [1] "youthvars_sofas" "integer"

youthvars_sofas(as.integer(c(0,23,89)))
#> [1]  0 23 89
#> attr(,"class")
#> [1] "youthvars_sofas" "integer"

Non-integers and values outside these ranges will produce errors.

youthvars_sofas(73.2)
#> Error in make_new_youthvars_sofas(x): is.integer(x) is not TRUE

youthvars_sofas(-1L)
#> Error: All non-missing values in valid youthvars_sofas object must be greater than or equal to 0.

youthvars_sofas(103L)
#> Error: All non-missing values in valid youthvars_sofas object must be less than or equal to 100.

5.2.1.3 - Score health utility

Note: This vignette is illustrated with fake data. The dataset explored in this example should not be used to inform decision-making. Some of the methods illustrated in this AQoL-6D vignette can also be used to score other health utility instruments - see a vignette about scoring EQ-5D.

library(ready4)
library(scorz)

AQoL-6D scoring

To derive a health utility score from the raw responses to a multi-attribute utility instrument it is necessary to implement a scoring algorithm. Scoring algorithms for the Assessment of Quality of Life Six Dimension (AQoL-6D) are publicly available in SPSS format (https://www.aqol.com.au/index.php/scoring-algorithms).

However, to include scoring algorithms in reproducible research workflows, it is desirable to have these algorithms available in open science languages such as R. We therefore developed an R implementation of the adult and adolescent versions of the AQoL-6D scoring algorithms and have made them available as part of the scorz package.

Ingest data

To begin, we ingest an unscored dataset as an instance of the Ready4useDyad class (from the ready4use package). In this case we download our data from a remote repository.

X <- ready4use::Ready4useRepos(dv_nm_1L_chr = "fakes",
                               dv_ds_nm_1L_chr = "https://doi.org/10.7910/DVN/W95KED",
                               dv_server_1L_chr = "dataverse.harvard.edu") %>%
  ingest(fls_to_ingest_chr = "ymh_clinical_dyad_r4",
         metadata_1L_lgl = F) 

To make the ingested dataset easier to interpret, we can add labels from the dictionary.

X <- X %>%
  renew(type_1L_chr = "label")

We can now inspect our ingested dataset using the exhibit method.

exhibit(X,
        display_1L_chr = "head",
         scroll_box_args_ls = list(width = "100%"))

We now add meta-data that identifies our dataset as being longitudinal using the YouthvarsSeries class of the youthvars package.

X <- youthvars::YouthvarsSeries(a_Ready4useDyad = X,
                                id_var_nm_1L_chr = "fkClientID",
                                timepoint_var_nm_1L_chr = "round",
                                timepoint_vals_chr = levels(X@ds_tb$round))

We now use the data and meta-data we have created in the previous steps to create an instance of the ScorzAqol6Adol class. This class is specifically designed to facilitate scoring of the adolescent version of the AQoL-6D instrument.

Y <- ScorzAqol6Adol(a_YouthvarsProfile = X)

By default, instances of the ScorzAqol6Adol class are created with a slot specifying a value for the prefix for AQoL-6D questionnaire item responses.

procureSlot(Y,
            slot_nm_1L_chr = "itm_prefix_1L_chr")
#> [1] "aqol6d_q"

If this default value needs to be updated to match the prefix used in your dataset, use the renewSlot method.

# Not run
# Y <- renewSlot(Y, slot_nm_1L_chr = "itm_prefix_1L_chr", new_val_xx = "new_prefix")

Calculating scores

To calculate AQoL 6D adolescent utility scores, use the renew method.

Y <- renew(Y)

Viewing the updated dataset

We can inspect our updated dataset using the exhibit method. We can see that the updated dataset now has additional variables that include the intermediate and final calculations for AQoL-6D adolescent utility scores.

exhibit(Y,
        display_1L_chr = "head",
         scroll_box_args_ls = list(width = "100%"))

Creating summary plots

To create plots, we use the depict method.

We can create a list of summary plots by timepoint for all individual items.

plot_ls <- depict(Y, type_1L_chr = "item_by_time")

We can then select a desired item’s summary plot by using its index number.

plot_ls[[1]]

AQoL-6D Item 1 scores by data-collection round

Alternatively, we can generate individual plots by passing the item index number to the var_idcs_int argument of depict.

depict(Y, type_1L_chr = "item_by_time", var_idcs_int = 2L)

AQoL-6D Item 2 scores by data-collection round

We can also plot domain scores by time.

depict(Y, type_1L_chr = "domain_by_time", var_idcs_int = 1L)

AQoL-6D Independet Living Domain weighted scores by data-collection round

Total AQoL-6D scores can also be plotted using the same approach, where var_idcs_int = 1L is used to plot the weighted total distribution and var_idcs_int = 2L is used for plotting the unweighted total.

depict(Y, type_1L_chr = "total_by_time", var_idcs_int = 1L)

AQoL-6D item total weighted scores by data-collection round

Composite plots can be generated as well, though these are not currently optimised to reliably produce quality plots suitable for publication.

depict(Y, type_1L_chr = "comp_item_by_time")

AQoL-6D item responses by data-collection round

depict(Y, type_1L_chr = "comp_domain_by_time")

AQoL-6D weighted domain scores by data-collection round

Share output

We can now publicly share our scored dataset and its associated metadata, using Ready4useRepos and its share method as described in a vignette from the ready4use package.

Z <- ready4use::Ready4useRepos(gh_repo_1L_chr = "ready4-dev/scorz", # Replace with details of your repo.
                               gh_tag_1L_chr = "Documentation_0.0") # You must have write permissions.
Z <- share(Z,
           obj_to_share_xx = Y,
           fl_nm_1L_chr = "ymh_ScorzAqol6Adol")

Y is now available for download as the file ymh_ScorzAqol6Adol.RDS from the “Documentation_0.0” release of the scorz package.

5.2.1.4 - Explore candidate utility mapping models

Note: This vignette uses fake data - it is for illustrative purposes only and should not be used to inform decision making.

The steps in this exploratory analysis workflow may need to be performed iteratively, both in order to identify the optimal model types, predictors and covariates to use and modify default values to ensure model convergence.

library(ready4)
library(ready4show)
library(ready4use)
library(scorz)
library(betareg)
library(specific)

Import data

We start by ingesting our data. As this example uses EQ-5D data, we import a ScorzEuroQol5 ready4 framework module (created using the steps described in this vignette from the scorz pacakge) into a SpecificConverter Module and then apply the metamorphose method to convert it into a SpecificModel module.

X <- SpecificConverter(a_ScorzProfile = ready4use::Ready4useRepos(gh_repo_1L_chr = "ready4-dev/scorz", 
                                                                  gh_tag_1L_chr = "Documentation_0.0") %>%
                         ingest(fls_to_ingest_chr = "ymh_ScorzEuroQol5",
                                metadata_1L_lgl = F)) %>%
  metamorphose() 

class(X)
#> [1] "SpecificModels"
#> attr(,"package")
#> [1] "specific"

Inspect data

The dataset we are using has a total of 1786 records at two timepoints on 1068 study participants. The first six records are reproduced below.

To source dataset of X is contained in the a_YouthvarsProfile slot and is a YouthvarsSeries module. For more information about methods that can be used to explore this dataset, read this vignette from the youthvars package.

Specify parameters

In preparation for exploring our dataset, we need to declare a set of model parameters in a b_SpecificParameters slot of X. This can be done in one step, or in sequential steps. In this example, we will proceed sequentially.

Dependent variable

The dependent variable (total EQ-5D utility score) has already been specified when we imported the data from the ScorzEuroQol5 module.

procureSlot(X,
            "b_SpecificParameters@depnt_var_nm_1L_chr")
#> [1] "eq5d_total_w"

We can now add details of the allowable range of dependent variable values.

X <- renewSlot(X,
               "b_SpecificParameters@depnt_var_min_max_dbl",
               c(-1,1))

Candidate predictors

We can now specify the names of candidate predictor variables.

X <- renewSlot(X,
               "b_SpecificParameters@candidate_predrs_chr",
               new_val_xx = c("K10_int","Psych_well_int")) 

We next add meta-data about each candidate predictor variable in the form of a specific_predictors object.

X <- renewSlot(X, 
               "b_SpecificParameters@predictors_lup", 
               short_name_chr = c("K10_int","Psych_well_int"),
               long_name_chr = c("Kessler Psychological Distress - 10 Item Total Score",
                                 "Overall Wellbeing Measure (Winefield et al. 2012)"),
               min_val_dbl = c(10,18),
               max_val_dbl = c(50,90),
               class_chr = "integer",
               increment_dbl = 1,
               class_fn_chr = "as.integer",
               mdl_scaling_dbl = 0.01,
               covariate_lgl = F)

The specific_predictors object that we have added to X can be inspected using the exhibitSlot method.

exhibitSlot(X,
            "b_SpecificParameters@predictors_lup",
            scroll_box_args_ls = list(width = "100%"))

Covariates

We also specify the covariates that we aim to explore in conjunction with each candidate predictor.

X <- renewSlot(X, 
               "b_SpecificParameters@candidate_covars_chr",
               new_val_xx = c("d_sex_birth_s", "d_age",  "d_sexual_ori_s", "d_studying_working"))

Descriptive variables

We also specify variables that we will use for generating descriptive statistics about the dataset.

X <- renewSlot(X,
               "b_SpecificParameters@descv_var_nms_chr",
               c("d_age","Gender","d_relation_s",
                 "d_sexual_ori_s", "Region", "d_studying_working")) 

Temporal variables

The name of the dataset variable for data collection timepoint and all of its unique values were imported when converting the ScorzEuroQol5 module.

procureSlot(X,"a_YouthvarsProfile@timepoint_var_nm_1L_chr")
#> [1] "Timepoint"

procureSlot(X,"a_YouthvarsProfile@timepoint_vals_chr")
#> [1] "BL"  "FUP"

However, we also need to specify the name of the variable that contains the datestamp for each dataset record.

X <- renewSlot(X,
               "b_SpecificParameters@msrmnt_date_var_nm_1L_chr",
               "data_collection_dtm")

Candidate models

X was created with a default set of candidate models, stored as a specific_models sub-module, which can be inspected using the exhibitSlot method.

exhibitSlot(X,
            "b_SpecificParameters@candidate_mdls_lup",
            scroll_box_args_ls = list(width = "100%"))

We can choose to select just a subset of these to explore using the renewSlot method. As this is an illustrative example, we have restricted the models we will explore to just four types, passing the relevant row numbers to the slice_indcs_int argument.

X <- renewSlot(X,
               "b_SpecificParameters@candidate_mdls_lup",
               slice_indcs_int = c(1L,5L,7L,8L))

Other parameters

Depending on the type of analysis we plan on undertaking, we can also specify parameters such as the number of folds to use in cross validation, the maximum number of model runs to allow and a seed to ensure reproducibility of results. In this case we are going to use the default values generated when we first created X.

procureSlot(X,
            "b_SpecificParameters@folds_1L_int")
#> [1] 10

procureSlot(X,
            "b_SpecificParameters@max_mdl_runs_1L_int")
#> [1] 300

procureSlot(X,
            "b_SpecificParameters@seed_1L_int")
#> [1] 1234

Model testing

Before we start to use the data stored in X to undertake modelling, we must first validate that it contains all necessary (and internally consistent) data by using the ratify method. The call to ratify will update any variable names that are likely to cause problems when generating reports (e.g. through inclusion of characters like “_” in the variable name that can cause problems when rendering LaTeX documents).

X <- ratify(X)

Set-up workspace

We add details of the directory to which we will write all output. In this example we create a temporary directory (tempdir()), but in practice this would be an existing directory on your local machine.

X <- renewSlot(X,
               "paths_chr",
               tempdir())

It can be useful to save fake data (useful for demonstrating the generalisability and replicability of an analysis) and real data (required for write-up and reproducibility) is distinctly labelled directories. By default, X is created with a flag to save all output in a sub-directory “Real”. As we are using fake data, we can override this value.

X <- renewSlot(X,
               "b_SpecificParameters@fake_1L_lgl",
               T)

We can now write a number of sub-directories to our specified output directory.

X <- author(X,
            what_1L_chr = "workspace")

Descriptives

The first set of outputs we write to our output directories is a set of descriptive tables and plots.

X <- author(X,
            what_1L_chr = "descriptives",
            digits_1L_int = 3L)

Model comparisons

The investigate method can now be used to compare the candidate models we have specified earlier. In so doing it will transform X into a SpecificPredictors object.

X <- investigate(X,
                 depnt_var_max_val_1L_dbl = 0.99,
                 session_ls = sessionInfo())

class(X)
#> [1] "SpecificPredictors"
#> attr(,"package")
#> [1] "specific"

The investigate method will write each model to be tested to a new sub-directory of our output directory.

The investigate method also outputs a table summarising the performance of each of the candidate models.

exhibit(X,
        what_1L_chr = "mdl_cmprsn",
        type_1L_chr = "results"
        ) 

We can now identify the highest performing model in each category of candidate model based on the testing R² statistic.

procure(X,
        what_1L_chr = "prefd_mdls") # Fix for NA_ returns (one option within ctg)
#> [1] "BET_LGT" "OLS_NTF"

We can override these automated selections and instead incorporate other considerations (possibly based on judgments informed by visual inspection of the plots and the desirability of constraining predictions to a maximum value of one). We do this in the following command, specifying new preferred model types, in descending order of preference.

X <- renew(X,
           new_val_xx = c("BET_LGT", "OLS_CLL"),
           type_1L_chr = "results",
           what_1L_chr = "prefd_mdls")

Use most preferred model to compare all candidate predictors

We can now compare all of our candidate predictors (with and without candidate covariates) using the most preferred model type.

X <- investigate(X)

class(X)
#> [1] "SpecificFixed"
#> attr(,"package")
#> [1] "specific"

Now, we compare the performance of single predictor models of our preferred model type (in our case, a Beta Regression Model with Binomial distribution and logit link) for each candidate predictor. The last call to the investigate saved the tested models along with model plots in a sub-directory of our output directory. These results are also viewable as a table.

exhibit(X,
        what_1L_chr = "predr_cmprsn",
        type_1L_chr = "results",
        scroll_box_args_ls = list(width = "100%"))

The most recent call to the investigate method also saved single predictor R model objects (one for each candidate predictors) along with the two plots for each model in a sub-directory of our output directory. The performance of each single predictor model can also be summarised in a table.

exhibit(X,
        type_1L_chr = "results",
        what_1L_chr = "fxd_sngl_cmprsn")

Updated versions of each of the models in the previous step (this time with covariates added) are saved to a new subdirectory of the output directory and we can summarise the performance of each of the updated models, along with all signficant model terms, in a table.

exhibit(X,
        type_1L_chr = "results",
        what_1L_chr = "fxd_full_cmprsn",
        scroll_box_args_ls = list(width = "100%"))

We can now identify which, if any, of the candidate covariates we previously specified are significant predictors in any of the models.

procure(X,
        type_1L_chr = "results",
        what_1L_chr = "signt_covars")
#> [1] NA

We can override the covariates to select, potentially because we want to select only covariates that are significant for all or most of the models. However, in the below example we have opted not to do so and continue to use no covariates as selected by the algorithm in the previous step.

# X <- renew(X,
#             new_val_xx = c("COVARIATE OF YOUR CHOICE", "ANOTHER COVARIATE"),
#                                               type_1L_chr = "results",
#                                   what_1L_chr = "prefd_covars")

Test preferred model with preferred covariates for each candidate predictor

We now conclude our model testing by rerunning the previous step, except confining our covariates to those we prefer.

X <- investigate(X)

class(X)
#> [1] "SpecificMixed"
#> attr(,"package")
#> [1] "specific"

The previous call to the write_mdls_with_covars_cmprsn function saves the tested models along with two plots for each model in the “E_Predrs_W_Covars_Sngl_Mdl_Cmprsn” sub-directory of “Output”.

Apply preferred model types and predictors to longitudinal data

The next main step is to use the preferred model types and covariates identified from the preceding analysis of cross-sectional data in longitudinal analysis.

Longitudinal mixed modelling

Prior to undertaking longitudinal mixed modelling, we need to check the appropriateness of the default values for modelling parameters that are stored in X. These include the number of model iterations, and any custom control parameters and priors (by default, empty lists).

procureSlot(X,
            "b_SpecificParameters@iters_1L_int")
#> [1] 4000

In many cases there will be no need to specify any custom control parameters or priors and using the defaults may speed up execution.

procureSlot(X,
            "b_SpecificParameters@control_ls")
#> [[1]]
#> list()

procureSlot(X,
            "b_SpecificParameters@prior_ls")
#> [[1]]
#> list()

However, in this example using the default control parameters would result in warning messages suggesting a change to the adapt_delta control value (default = 0.8). Modifying the adapt_delta control parameter value can address this issue.

X <- renewSlot(X,
             slot_nm_1L_chr = "b_SpecificParameters@control_ls",
             new_val_xx = list(adapt_delta = 0.99))

X <- investigate(X)

class(X)
#> [1] "SpecificMixed"
#> attr(,"package")
#> [1] "specific"

The last call to investigate function wrote the models it tests to a sub-directory of the output directory along with plots for each model.

Create shareable outputs

The model objects created by the preceding analysis are not suitable for sharing as they contain duplicates of the source dataset. To create model objects that can be shared (where dataset copies are replaced with fake data) use the authorData method.

X <- authorData(X)

Purge dataset copies

For the purposes of efficient computation, multiple objects containing copies of the source dataset were saved to our output directory during the analysis process. We therefore need to delete all of these copies by supplying “purge_write” to the type_1L_chr argument of the author method.

author(X,
       type_1L_chr = "purge_write")

A copy of the module X is available for download as the file eq5d_ttu_SpecificMixed.RDS from the “Documentation_0.0” release of the specific package.

5.2.1.5 - Implement a utility mapping study

Note: This vignette uses fake data - it is for illustrative purposes only and should not be used to inform decision making.

Motivation

Youth mental health services do not typically collect health utility data from their clients, which makes it more difficult to place an economic values on outcomes attained in these services. One strategy for addressing this gap is to use data from similar samples of young people that contain both health utility and the types of outcome measures that are collected in clinical services. The TTU package provides a toolkit for conducting and reporting a utility mapping (or Transfer to Utility) study.

Implementation

TTU has been developed for use with the ready4 model and combines and extends multiple types of ready4 modules:

• Modules for labeling, validating and summarising youth mental health datasets from the youthvars package;
• Modules for scoring health utility from the scorz package;
• Modules for specifying and testing statistical models from the specific package;
• Modules for generating reports from the ready4show package; and
• Modules for sharing data via online data repositories from the ready4use package.

Additionally, TTU relies on two RMarkdown programs:

• ttu_mdl_ctlg: Generate a Template Utility Mapping (Transfer to Utility) Model Catalogue (https://doi.org/10.5281/zenodo.5936870)
• ttu_lng_ss: Create a Draft Scientific Manuscript For A Utility Mapping Study (https://doi.org/10.5281/zenodo.5976987)

Workflow

Background and citation

The following workflow illustrates (using fake data) the same steps we used in a real world study, a summary of which is available at https://doi.org/10.1101/2021.07.07.21260129). Citation information for that study is:

@article {Hamilton2021.07.07.21260129,
    author = {Hamilton, Matthew P and Gao, Caroline X and Filia, Kate M and Menssink, Jana M and Sharmin, Sonia and Telford, Nic and Herrman, Helen and Hickie, Ian B and Mihalopoulos, Cathrine and Rickwood, Debra J and McGorry, Patrick D and Cotton, Sue M},
    title = {Predicting Quality Adjusted Life Years in young people attending primary mental health services},
    elocation-id = {2021.07.07.21260129},
    year = {2021},
    doi = {10.1101/2021.07.07.21260129},
    publisher = {Cold Spring Harbor Laboratory Press},
    URL = {https://www.medrxiv.org/content/early/2021/07/12/2021.07.07.21260129},
    eprint = {https://www.medrxiv.org/content/early/2021/07/12/2021.07.07.21260129.full.pdf},
    journal = {medRxiv}
}

The program applied in that study, which this workflow closely resembles is available at https://doi.org/10.5281/zenodo.6116077 and can be cited as follows:

@software{hamilton_matthew_2022_6212704,
  author       = {Hamilton, Matthew and
                  Gao, Caroline},
  title        = {{Complete study program to reproduce all steps from 
                   data ingest through to results dissemination for a
                   study to map mental health measures to AQoL-6D
                   health utility}},
  month        = feb,
  year         = 2022,
  note         = {{Matthew Hamilton and Caroline Gao  (2022). 
                   Complete study program to reproduce all steps from
                   data ingest through to results dissemination for a
                   study to map mental health measures to AQoL-6D
                   health utility. Zenodo.
                   https://doi.org/10.5281/zenodo.6116077. Version
                   0.0.9.3}},
  publisher    = {Zenodo},
  version      = {0.0.9.3},
  doi          = {10.5281/zenodo.6212704},
  url          = {https://doi.org/10.5281/zenodo.6212704}
}

Load required packages

We begin by loading our required packages.

library(ready4)
library(ready4use)
library(youthvars)
library(scorz)
library(TTU)

Add dataset metadata

We use the Ready4useDyad and Ready4useRepos modules to retrieve and ingest and to then pair a dataset and its data dictionary.

A <- Ready4useDyad(ds_tb = Ready4useRepos(dv_nm_1L_chr = "fakes",
                                          dv_ds_nm_1L_chr = "https://doi.org/10.7910/DVN/HJXYKQ",
                                          dv_server_1L_chr = "dataverse.harvard.edu") %>%
                     ingest(fls_to_ingest_chr = c("ymh_clinical_tb"),
                            metadata_1L_lgl = F) %>%
                     youthvars::transform_raw_ds_for_analysis(),
                   dictionary_r3 = Ready4useRepos(dv_nm_1L_chr = "TTU", 
                                                  dv_ds_nm_1L_chr = "https://doi.org/10.7910/DVN/DKDIB0", 
                                                  dv_server_1L_chr = "dataverse.harvard.edu") %>%
                     ingest(fls_to_ingest_chr = c("dictionary_r3"),
                            metadata_1L_lgl = F)) %>%
  renew(type_1L_chr = "label")

We use the YouthvarsSeries module to supply metadata about out a longitudinal dataset vignette.

A <- YouthvarsSeries(a_Ready4useDyad = A,
                     id_var_nm_1L_chr = "fkClientID",
                     timepoint_var_nm_1L_chr = "round",
                     timepoint_vals_chr = levels(procureSlot(A,
                                                             "ds_tb")$round))

Score health utility

We next use the ScorzAqol6Adol module to score adolescent AQoL-6D health utility.

A <- TTUProject(a_ScorzProfile = ScorzAqol6Adol(a_YouthvarsProfile = A))
A <- renewSlot(A, "a_ScorzProfile")
#> Joining, by = c("fkClientID", "match_var_chr")

Evaluate candidate models

Over the next few steps we will use modules from the specific package to to specify and assess a number of candidate utility mapping models.

A <- renewSlot(A, "b_SpecificParameters", SpecificConverter(a_ScorzProfile = A@a_ScorzProfile) %>%
                 metamorphose() %>%
                 procureSlot("b_SpecificParameters"))

A <- renewSlot(A, "b_SpecificParameters@predictors_lup", Ready4useRepos(dv_nm_1L_chr = "TTU", 
                                                                        dv_ds_nm_1L_chr = "https://doi.org/10.7910/DVN/DKDIB0", 
                                                                        dv_server_1L_chr = "dataverse.harvard.edu") %>%
                 ingest(fls_to_ingest_chr = c("predictors_r3"),
                        metadata_1L_lgl = F)) 

We can inspect the metadata on candidate predictors that we have just ingested.

exhibitSlot(A, "b_SpecificParameters@predictors_lup",
         scroll_box_args_ls = list(width = "100%"))

We add additional metadata about variables in our dataset that will be used in exploratory modelling.

A <- renewSlot(A, "b_SpecificParameters@depnt_var_min_max_dbl", c(0.03,1)) %>% # Inherit From TTUAqolAdol
  renewSlot("b_SpecificParameters@candidate_predrs_chr", c("BADS","GAD7", "K6", "OASIS", "PHQ9", "SCARED")) %>%
  renewSlot("b_SpecificParameters@candidate_covars_chr", c("d_sex_birth_s", "d_age",  "d_sexual_ori_s", 
                                                           "d_studying_working", "c_p_diag_s", "c_clinical_staging_s",
                                                           "SOFAS")) %>%
  renewSlot("b_SpecificParameters@descv_var_nms_chr", c("d_age","Gender","d_relation_s", "d_sexual_ori_s", 
                                                        "Region", "d_studying_working", "c_p_diag_s", 
                                                        "c_clinical_staging_s","SOFAS")) %>%
  renewSlot("b_SpecificParameters@msrmnt_date_var_nm_1L_chr", "d_interview_date") 

A <-  renewSlot(A, "b_SpecificParameters@fake_1L_lgl", T) 

A <- renewSlot(A, "c_SpecificProject", SpecificModels(a_YouthvarsProfile = A@a_ScorzProfile@a_YouthvarsProfile,
                                                      b_SpecificParameters = A@b_SpecificParameters,
                                                      paths_chr = tempdir())) 

A <- ratifySlot(A, "c_SpecificProject")

A <- renewSlot(A, "c_SpecificProject", 
               authorSlot(A, "c_SpecificProject", what_1L_chr = "workspace"))

We now generate tables and charts that describe our dataset. These are saved in a sub-directory of our output data directory, a copy of which is available for download. One of the plots is also reproduced here.

A <- renewSlot(A, "c_SpecificProject",
               authorSlot(A, "c_SpecificProject", what_1L_chr = "descriptives",
                          digits_1L_int = 3L))

We next compare the performance of different model types. This step saves model objects and plots to a sub-directory of our output directory, a copy of which is available for download.

A <- renewSlot(A, "c_SpecificProject",
               investigateSlot(A, "c_SpecificProject",
                               depnt_var_max_val_1L_dbl = 0.99,
                               session_ls = sessionInfo()))

After inspecting the output of the previous command, we can now specify the preferred model types to use from this point onwards.

A <- renewSlot(A, "c_SpecificProject",
               renew(procureSlot(A, "c_SpecificProject"),
                     new_val_xx = c("GLM_GSN_LOG", "OLS_CLL"),
                     type_1L_chr = "results",
                     what_1L_chr = "prefd_mdls"))

Next we assess multiple versions of our preferred model type - one single predictor model for each of our candidate predictors and the same models with candidate covariates added. A number of model/plot objects saved to a sub-directory of our output directory, a copy of which is available for download.

A <- renewSlot(A, "c_SpecificProject",
               investigateSlot(A,"c_SpecificProject"))

After reviewing the output of the previous step, we specify the covariates we wish to add to the models.

A <- renewSlot(A, "c_SpecificProject",
               renew(procureSlot(A, "c_SpecificProject"),
                     new_val_xx = "SOFAS",
                     type_1L_chr = "results",
                     what_1L_chr = "prefd_covars"))

We now assess the multivariate models. More model/plot objects are saved to a sub-directory of our output directory, a copy of which is available for download.

A <- renewSlot(A, "c_SpecificProject",
               investigateSlot(A, "c_SpecificProject"))

We next reformulate the models we finalised in the previous step so that they are suitable for modelling longitudinal change.

For our primary analysis, we use the longitudinal formulation of the models we previously selected. A series of large model files are written to the local output data directory.

A <- renewSlot(A, "c_SpecificProject",
               investigateSlot(A, "c_SpecificProject"))

For our secondary analyses, we specify alternative combinations of predictors and covariates.

A <- renewSlot(A, "c_SpecificProject",
               investigateSlot(A, "c_SpecificProject",
                               scndry_anlys_params_ls = make_scndry_anlys_params(candidate_predrs_chr = c("SOFAS"),
                                                                                 candidate_covar_nms_chr = c("d_sex_birth_s", 
                                                                                                             "d_age", 
                                                                                                             "d_sexual_ori_s",
                                                                                                             "d_studying_working"),
                                                                                 prefd_covars_chr = NA_character_) %>%
                                 make_scndry_anlys_params(candidate_predrs_chr = c("SCARED","OASIS","GAD7"),
                                                          candidate_covar_nms_chr = c("PHQ9", "SOFAS", 
                                                                                      "d_sex_birth_s", 
                                                                                      "d_age", 
                                                                                      "d_sexual_ori_s",
                                                                                      "d_studying_working"),
                                                          prefd_covars_chr = "PHQ9")))

Report and disseminate findings

Create shareable models

The model objects created and saved in our working directory by the preceding steps are not suitable for public dissemination. They are both too large in file size and, more importantly, include copies of our source dataset. We can overcome these limitations by creating shareable versions of the models. Two types of shareable version are created - copies of the original model objects in which fake data overwrites the original source data and summary tables of model coefficients.

A <- renewSlot(A, "c_SpecificProject",
               authorData(procureSlot(A, "c_SpecificProject")))

Specify study reporting metadata

We create a TTUSynopsis object that contains the fields necessary to render and share reports.

A <- renewSlot(A, "d_TTUReports",
               {
                 Y <- metamorphoseSlot(A, "c_SpecificProject")
                 Y <- TTUSynopsis(a_Ready4showPaths = Y@a_Ready4showPaths,
                                  b_SpecificResults = Y@b_SpecificResults,
                                  c_SpecificParameters = Y@c_SpecificParameters,
                                  d_YouthvarsProfile = Y@d_YouthvarsProfile,
                                  rmd_fl_nms_ls = Y@rmd_fl_nms_ls)
                 Y <- TTUReports(a_TTUSynopsis = Y)
                 Y
                 }
               )

We add metadata relevant to the reports that we will be generating to these fields. Note that the data we supply to the Ready4useRepos object below must relate to a repository to which we have write permissions (otherwise subsequent steps will fail).

A <- renewSlot(A, "d_TTUReports@a_TTUSynopsis",
               procureSlot(A, "d_TTUReports@a_TTUSynopsis") %>% 
                 renewSlot("authors_r3", ready4show::authors_tb) %>%
                 renewSlot("institutes_r3", ready4show::institutes_tb) %>%
                 renewSlot("digits_int", c(3L,3L)) %>%
                 renewSlot("outp_formats_chr", c("PDF","PDF")) %>%
                 renewSlot("title_1L_chr", "A hypothetical utility mapping study using fake data") %>%
                 renewSlot("correspondences_r3", old_nms_chr = c("PHQ9", "GAD7"), new_nms_chr = c("PHQ-9", "GAD-7")) %>%
                 renewSlot("e_Ready4useRepos", Ready4useRepos(dv_nm_1L_chr = "fakes", 
                                                              dv_ds_nm_1L_chr = "https://doi.org/10.7910/DVN/D74QMP", 
                                                              dv_server_1L_chr = "dataverse.harvard.edu"))) 

Author model catalogues

We download a program for generating a catalogue of models and use it to summarising the models created under each study analysis (one primary and two secondary). The catalogues are saved locally.

authorSlot(A, "d_TTUReports", what_1L_chr = "Catalogue", download_tmpl_1L_lgl = T)

Share model catalogue

We share the catalogues that we created, uploading a copy to our study online repository. To run this step you will need write permissions to the online repository.

shareSlot(A, "d_TTUReports@a_TTUSynopsis", type_1L_chr = "Report", what_1L_chr = "Catalogue") 

Share models

We share tables of coefficients and other meta-data about the models we have created by posting them to the online repository. The object we create and share is designed to be used in conjunction with the youthu package to make it easier to make predictions with these models using new data. Again, you will need write permissions to the online repository.

shareSlot(A, "d_TTUReports@a_TTUSynopsis", type_1L_chr = "Models", what_1L_chr = "ingredients")

Author manuscript

We add some content about the manuscript we wish to author.

A <- renewSlot(A, "d_TTUReports@a_TTUSynopsis",
               procureSlot(A, "d_TTUReports@a_TTUSynopsis") %>% 
                 renewSlot("background_1L_chr", "Quality Adjusted Life Years (QALYs) are often used in economic evaluations, yet utility weights for deriving them are rarely directly measured in mental health services.") %>%
                 renewSlot("coi_1L_chr", "None declared") %>%
                 renewSlot("conclusion_1L_chr","Nothing should be concluded from this study as it is purely hypothetical.") %>%
                 renewSlot("ethics_1L_chr", "The study was reviewed and granted approval by no-one." ) %>%
                 renewSlot("funding_1L_chr", "The study was funded by no-one.") %>%
                 renewSlot("interval_chr", "three months") %>%
                 renewSlot("keywords_chr", c("anxiety", "AQoL","depression", "psychological distress", "QALYs", "utility mapping")) %>%
                 renewSlot("sample_desc_1L_chr", "The study sample is fake data.") )

We create a summary of results that can be interpreted by the program that authors the manuscript.

A <- renewSlot(A, "d_TTUReports@a_TTUSynopsis@abstract_args_ls",
               manufactureSlot(A,"d_TTUReports@a_TTUSynopsis", what_1L_chr = "abstract_args_ls",
                               depnt_var_nms_chr = c("AQoL-6D", "Adolescent AQoL Six Dimension"))) 

A <- enhanceSlot(A, "d_TTUReports@a_TTUSynopsis", with_1L_chr = "results_ls",
                 depnt_var_nms_chr = c("AQoL-6D", "Adolescent AQoL Six Dimension")) 

We create and save the plots that will be used in the manuscript.

authorSlot(A, "d_TTUReports", type_1L_chr = "Plots",
           depnt_var_desc_1L_chr = A@d_TTUReports@a_TTUSynopsis@b_SpecificResults@a_SpecificShareable@shareable_outp_ls$results_ls$study_descs_ls$health_utl_nm_1L_chr)

We download a program for generating a template manuscript and run it to author a first draft of the manuscript.

authorSlot(A, "d_TTUReports", type_1L_chr = "Report", what_1L_chr = "Manuscript_Auto", download_tmpl_1L_lgl = T)

We can copy the RMarkdown files that created the template manuscript to a new director (which we call “Manuscript_Submission”) so that we can then manually edit those files to produce a manuscript that we can submit for publication. Note that in this example we have not made any edits to the template manuscript.

R.utils::copyDirectory(paste0(A@d_TTUReports@a_TTUSynopsis@a_Ready4showPaths@outp_data_dir_1L_chr,
                              "/",
                              A@d_TTUReports@a_TTUSynopsis@a_Ready4showPaths@mkdn_data_dir_1L_chr,
                              "/Manuscript_Auto"),
                       paste0(A@d_TTUReports@a_TTUSynopsis@a_Ready4showPaths@outp_data_dir_1L_chr,
                              "/",
                              A@d_TTUReports@a_TTUSynopsis@a_Ready4showPaths@mkdn_data_dir_1L_chr,
                              "/Manuscript_Submission"))

Once any edits to the RMarkdown files for creating the submission manuscript have been finalised, we can run the following command to author the manuscript. The below commands will generate a Microsoft Word format manuscript and a PDF technical appendix. Unlike the template manuscript, the figures and tables are positioned after (and not within) the main body of the manuscript. Note that the Word version of the manuscript generated by these commands will require some minor formatting edits (principally to the display of tables and numbering of sections).

A <- renewSlot(A, "d_TTUReports",
               procureSlot(A, "d_TTUReports") %>%
                 renewSlot("a_TTUSynopsis@tables_in_body_lgl",  F) %>%
                 renewSlot("a_TTUSynopsis@figures_in_body_lgl", F) %>%
                 renewSlot("a_TTUSynopsis@outp_formats_chr", c("Word","PDF")))
authorSlot(A, "d_TTUReports", what_1L_chr = "Manuscript_Submission", download_tmpl_1L_lgl = F)

Tidy workspace

The preceding steps saved multiple objects (mostly R model objects) that have embedded within them copies of the source dataset. We can now purge all such copies from our output data directory.

author(procureSlot(A,"c_SpecificProject"),
       type_1L_chr = "purge_write") 

5.2.1.6 - Find and deploy utility mapping models

library(youthu)

This vignette outlines a workflow for:

• Searching, selecting and retrieving transfer to utility models;
• Preparing a prediction dataset for use with a selected transfer to utility model; and
• Applying the selected transfer to utility model to a prediction dataset to predict Quality Adjusted Life Years (QALYs).

The practical value of implementing such a workflow is discussed in the economic analysis vignette and a scientific manuscript. Note, this example uses fake data - it should should not be used to inform decision making.

Search, select and retrieve transfer to utility models

To identify datasets that contain transfer to utility models compatible with youthu (ie those developped with the TTU package), you can use the get_ttu_dv_dss function. The function searches specified dataverses (in the below example, the TTU dataverse) for datasets containing output from the TTU package.

ttu_dv_dss_tb <- get_ttu_dv_dss("TTU")

The ttu_dv_dss_tb table summarises some pertinent details about each dataset containing TTU models found by the preceding command. These details include a link to any scientific summary (the “Article” column) associated with a dataset.

To identify models that predict a specified type of health utility from one or more of a specified subset of predictors, use:

mdls_lup <- get_mdls_lup(ttu_dv_dss_tb = ttu_dv_dss_tb,
                         utility_type_chr = "AQoL-6D",
                         mdl_predrs_in_ds_chr = c("PHQ9 total score",
                                                  "SOFAS total score"))

The preceding command will produce a lookup table with information that includes the catalogue names of models, the predictors used in each model and the analysis that generated each one.

To review the summary information about the predictive performance of a specific model, use:

get_dv_mdl_smrys(mdls_lup,
                 mdl_nms_chr = "PHQ9_SOFAS_1_OLS_CLL")
#> $PHQ9_SOFAS_1_OLS_CLL
#>        Parameter Estimate    SE          95% CI
#> 1 SD (Intercept)    0.348 0.017   0.312 , 0.382
#> 2      Intercept    0.428 0.129   0.174 , 0.686
#> 3  PHQ9 baseline   -9.115 0.249 -9.601 , -8.618
#> 4    PHQ9 change   -7.331 0.339 -8.007 , -6.665
#> 5 SOFAS baseline    0.960 0.172   0.616 , 1.292
#> 6   SOFAS change    1.146 0.235   0.674 , 1.607
#> 7             R2    0.767 0.012   0.743 , 0.788
#> 8           RMSE    0.925 0.004   0.922 , 0.928
#> 9          Sigma    0.406 0.012   0.384 , 0.429