diff --git a/DESCRIPTION b/DESCRIPTION index d7b4663..9b79bc9 100644 --- a/DESCRIPTION +++ b/DESCRIPTION @@ -30,11 +30,18 @@ URL: https://github.com/stmcg/learner BugReports: https://github.com/stmcg/learner/issues Imports: ScreeNOT, Rcpp (>= 1.0.11), - RcppEigen + RcppEigen, + graphics, + grDevices, + stats LinkingTo: Rcpp, RcppEigen Suggests: - testthat (>= 3.0.0) + testthat (>= 3.0.0), + ggplot2 (>= 3.4.0), + knitr, + rmarkdown Config/testthat/edition: 3 Depends: R (>= 2.10) +VignetteBuilder: knitr RoxygenNote: 7.3.2 diff --git a/NAMESPACE b/NAMESPACE index 1ab3843..02814b0 100644 --- a/NAMESPACE +++ b/NAMESPACE @@ -3,6 +3,10 @@ export(cv.learner) export(dlearner) export(learner) +export(plot_contributions) +export(plot_cv) +export(plot_objective) +export(plot_source_spaces) import(RcppEigen) importFrom(Rcpp,sourceCpp) useDynLib(learner, .registration = TRUE) diff --git a/NEWS.md b/NEWS.md index e9ac59c..013c5a8 100644 --- a/NEWS.md +++ b/NEWS.md @@ -6,6 +6,10 @@ populations with independently generated noise. * Add separate row- and column-space penalties in `learner()` and cross-validation over these penalties in `cv.learner()`. +* Add `dat_asymmetric` and an advanced tutorial illustrating multiple sources + and separate space penalties in two distinct workflows. +* Add plotting functions for objective function values, cross-validation + results, population latent spaces, and variable contributions. ### learner version 1.0.0 (2025-03-02) diff --git a/R/data-asymmetric.R b/R/data-asymmetric.R new file mode 100644 index 0000000..2634eb9 --- /dev/null +++ b/R/data-asymmetric.R @@ -0,0 +1,53 @@ +#' Simulated data set: One shared latent space +#' +#' One source and one target with the same true U-space but different true +#' V-spaces. This example illustrates why the two space penalties in LEARNER +#' can be selected separately. +#' +#' @docType data +#' @format A list of four aligned matrices, each with 100 rows and 50 columns: +#' \describe{ +#' \item{Y_source}{Observed source data.} +#' \item{Y_target}{Observed target data.} +#' \item{Theta_source}{True rank-3 source signal.} +#' \item{Theta_target}{True rank-3 target signal.} +#' } +#' @details The target signal is the rank-3 truncated SVD of a standard normal +#' matrix. The source keeps its U-space, reverses its three singular values, +#' and perturbs its V basis with independent uniform entries between +#' \eqn{-1/(2\sqrt{50})} and \eqn{1/(2\sqrt{50})}, followed by QR orthonormalization. +#' This adapts the U-shared, V-perturbed "low similarity" scenario in Section +#' 2.4.1 of the Supporting Information of McGrath et al. (2026) to a small +#' example (the paper uses 500 by 500 matrices of rank 4). It is not a +#' replication of that study. The Supporting Information calls this setting +#' "same column spaces but different row spaces": the span of U is the column +#' space of the signal matrix, and the span of V is its row space. +#' +#' Independent mean-zero normal noise is added to each signal, with variance +#' 0.10 for the target and 0.01 for the source. The exact shared-space statement +#' applies to the true signals; noise also affects estimated spaces. +#' True signals are supplied to illustrate estimation error, not for fitting +#' or penalty selection. Generation code is in \file{data-raw/gen-asymmetric.R} +#' in the source repository. +#' +#' @references +#' McGrath, S., Zhu, C., O'Dea, R., Guo, M. and Duan, R. (2026). \emph{LEARNER: A transfer learning method for low-rank matrix estimation}. Statistics in Medicine, 45(15-17), e70669. \doi{10.1002/sim.70669}. +#' +#' @examples +#' data(dat_asymmetric) +#' d <- dat_asymmetric +#' error <- function(fit) norm(fit$learner_estimate - d$Theta_target, "F") +#' # Equal penalties versus stronger borrowing on the shared U side. The values +#' # are illustrative and were not selected by cross-validation; see +#' # vignette("advanced-learner") for selecting them. +#' common <- learner(d$Y_source, d$Y_target, r = 3, lambda_1 = 10, +#' lambda_2 = 0.1, step_size = 0.003, +#' control = list(max_iter = 2000)) +#' separate <- learner(d$Y_source, d$Y_target, r = 3, lambda_1_row = 100, +#' lambda_1_col = 10, lambda_2 = 0.1, step_size = 0.003, +#' control = list(max_iter = 2000)) +#' c(common = error(common), separate = error(separate)) +#' @seealso \code{\link{dat_highsim}}, \code{\link{dat_multisource}}, +#' \code{\link{learner}}, \code{\link{cv.learner}} +#' @keywords datasets +"dat_asymmetric" diff --git a/R/plot-spaces.R b/R/plot-spaces.R new file mode 100644 index 0000000..d94799a --- /dev/null +++ b/R/plot-spaces.R @@ -0,0 +1,189 @@ +# Internal helpers shared by the space and contribution plots. +check_plot_matrix <- function(x, name) { + if (!is.matrix(x) || !is.numeric(x) || is.complex(x) || any(dim(x)==0) || any(!is.finite(x))) { + stop(name, ' must be a complete finite real numeric matrix.') + } +} + +plot_indices <- function(index, n, max_display, name) { + if(is.null(index)) return(unique(as.integer(round(seq(1,n,length.out=min(n,max_display)))))) + if(!is.numeric(index)||is.complex(index)||length(index)==0||any(!is.finite(index))|| + any(index!=floor(index))||any(index<1|index>n)||anyDuplicated(index)) { + stop(name, ' must contain distinct valid integer indices.') + } + as.integer(index) +} + +plot_svd <- function(x, r) { + s<-svd(x,nu=r,nv=r) + if(s$d[r] <= max(dim(x))*s$d[1]*.Machine$double.eps) { + stop('r includes numerically zero singular directions; use a smaller rank.') + } + s +} + +plot_limits_pair <- function(limits, automatic) { + if(is.null(limits))return(automatic) + if(!is.numeric(limits)||is.complex(limits)||length(limits)!=2L|| + any(!is.finite(limits))||any(limits<=0))stop('limits must contain two finite positive values: row then column.') + stats::setNames(as.numeric(limits),c('row','column')) +} + +# Use a raster to avoid drawing hundreds of thousands of individual rectangles. +draw_matrix_panel <- function(values, x_index, y_index, limits, colors, title, xlab, ylab) { + nx<-nrow(values);ny<-ncol(values) + clipped<-pmax(limits[1],pmin(limits[2],values)) + bins<-1L+floor((clipped-limits[1])/diff(limits)*(length(colors)-1L)) + pixels<-matrix(colors[pmax(1L,pmin(length(colors),bins))],nx,ny) + graphics::plot.new() + # Reserve a constant fraction of the panel width for the color key. + graphics::plot.window(xlim=c(0.5,nx+0.5+nx*0.15),ylim=c(0.5,ny+0.5),xaxs='i',yaxs='i') + graphics::rasterImage(grDevices::as.raster(t(pixels[,ny:1,drop=FALSE])), + 0.5,0.5,nx+0.5,ny+0.5,interpolate=FALSE) + graphics::rect(0.5,0.5,nx+0.5,ny+0.5,border='gray40',lwd=0.7) + ix<-unique(as.integer(round(seq(1,nx,length.out=min(5,nx))))) + iy<-unique(as.integer(round(seq(1,ny,length.out=min(8,ny))))) + graphics::axis(1,at=ix,labels=x_index[ix],cex.axis=0.85) + graphics::axis(2,at=iy,labels=y_index[iy],las=1,cex.axis=0.85) + graphics::title(main=title,xlab=xlab,ylab=ylab,cex.main=1.05,cex.lab=0.9) + draw_color_key(nx+0.5+nx*0.05,nx+0.5+nx*0.075,0.5,ny+0.5,limits,colors) +} + +#' Compare target and source latent spaces +#' +#' Compare patterns in the target and source latent spaces. Similar patterns +#' suggest that the populations share similar spaces at the chosen rank. +#' +#' @param Y_target Complete target matrix, or a completed estimate. Missing +#' values are not allowed. +#' @param Y_source Source matrix, or a list of source matrices, with the same +#' rows and columns as the target. +#' @param r Rank used for every matrix. +#' @param row_index,column_index Optional indices of the rows and columns to +#' display. The singular vectors are always computed from the full matrices. +#' @param max_display Maximum number of rows, and of columns, to display when +#' no indices are given (default 200). Evenly spaced indices are used. +#' @param limits Optional pair of positive color limits (row side, then column +#' side). Each scale is symmetric around zero and shared across populations. +#' Values beyond a limit get the extreme color, and their number is returned as +#' \code{clipped_count}. +#' @param row_label,column_label Axis labels. The defaults are generic, so use +#' labels that describe your rows and columns. +#' @param target_name Label for the target panel. +#' @return Invisibly, a list with the displayed projection matrices +#' (\code{row_projectors} and \code{column_projectors}, named by population), the +#' displayed \code{row_index} and \code{column_index}, \code{limits}, and +#' \code{clipped_count}. +#' @details Each heatmap shows a projection matrix computed from the leading +#' singular vectors. Column-side patterns appear above row-side patterns. +#' Red indicates negative values and blue positive values; populations share +#' a color scale within each side so their patterns can be compared. +#' +#' Rows and columns must already be aligned across populations. Selecting +#' indices changes only what is displayed; spaces are estimated from the full +#' matrices. These plots show each population separately, without source weights. +#' Similar colors are a visual comparison, not a statistical test of shared spaces. +#' +#' @examples +#' plot_source_spaces(dat_multisource$Y_target, dat_multisource$Y_source, +#' r = 3, max_display = 20) +#' @export +plot_source_spaces <- function(Y_target, Y_source, r, row_index=NULL, column_index=NULL, + max_display=200L, limits=NULL, row_label='Row index', + column_label='Column index', target_name='Target population') { + check_plot_matrix(Y_target,'Y_target') + sources<-prepare_sources(Y_source,Y_target,allow_missing_target=FALSE)$matrices + for(s in sources)check_plot_matrix(s,'Y_source') + r<-resolve_rank(r,Y_target);validate_count(max_display,'max_display') + ri<-plot_indices(row_index,nrow(Y_target),max_display,'row_index') + ci<-plot_indices(column_index,ncol(Y_target),max_display,'column_index') + labels<-names(sources) + if(is.null(labels))labels<-rep('',length(sources)) + labels[is.na(labels)]<-'' + labels<-ifelse(nzchar(labels),paste('Source',labels),paste('Source',seq_along(sources))) + populations<-c(list(Y_target),sources);names(populations)<-make.unique(c(target_name,labels)) + bases<-lapply(populations,plot_svd,r=r) + up<-lapply(bases,function(s)tcrossprod(s$u[ri,,drop=FALSE])) + vp<-lapply(bases,function(s)tcrossprod(s$v[ci,,drop=FALSE])) + auto<-c(row=max(vapply(up,function(p)max(abs(p)),numeric(1))), + column=max(vapply(vp,function(p)max(abs(p)),numeric(1)))) + auto[auto==0]<-1e-8 + cap<-plot_limits_pair(limits,auto) + counts<-list(row=vapply(up,function(p)sum(abs(p)>cap[1]),integer(1)), + column=vapply(vp,function(p)sum(abs(p)>cap[2]),integer(1))) + colors<-grDevices::colorRampPalette(c('firebrick3','white','dodgerblue3'))(101) + old<-graphics::par(no.readonly=TRUE);on.exit(graphics::par(old),add=TRUE) + for(page in split(seq_along(populations),ceiling(seq_along(populations)/3))) { + graphics::par(mfrow=c(2,length(page)),mar=c(3.5,3.8,2.5,4),oma=c(2,0,0,0),mgp=c(2.2,0.6,0)) + for(i in page)draw_matrix_panel(vp[[i]],ci,ci,c(-cap[2],cap[2]),colors,names(populations)[i],column_label,column_label) + for(i in page)draw_matrix_panel(up[[i]],ri,ri,c(-cap[1],cap[1]),colors,names(populations)[i],row_label,row_label) + caption<-paste('Red: negative; blue: positive. Shared scales within each row.', + if(sum(unlist(counts))>0)paste(sum(unlist(counts)),'displayed entries clipped across all panels.')else '') + graphics::mtext(caption,outer=TRUE,side=1,line=0.5,cex=0.8) + } + invisible(list(row_projectors=up,column_projectors=vp,row_index=ri,column_index=ci, + limits=cap,clipped_count=counts)) +} + +#' Plot variable contributions to latent components +#' +#' Show which rows and columns contribute to each latent component of a fitted +#' target matrix, using squared singular-vector entries. Higher scores are darker red. +#' +#' @param x A complete estimated matrix, or a \code{learner()} or +#' \code{dlearner()} result. The singular vectors are not rotated. +#' @param r Number of components. Required for a matrix; for a fitting result +#' the default is its rank. +#' @inheritParams plot_source_spaces +#' @param limits Optional pair of positive upper limits for the color scales +#' (row side, then column side). Each scale runs from 0 (white) to its limit +#' (dark red). Larger values get the darkest color, and their number is returned +#' as \code{clipped_count}. +#' @param main Plot title. +#' @return Invisibly, a list with the full \code{row_scores} and +#' \code{column_scores} (variables in rows, components in columns), the displayed +#' \code{row_index} and \code{column_index}, \code{limits}, and +#' \code{clipped_count}. The returned scores are not clipped. +#' @details Darker red indicates a larger contribution to a component. +#' Columns are shown on the left and rows on the right, with separate color +#' scales. Scores are squared singular-vector entries from the estimated matrix; +#' they sum to one over all variables on each side for each component. +#' +#' Scores describe contributions, not effect sizes or correlations. Selecting +#' variables changes only the display, without rescaling scores. Components are +#' not matched across populations, and components with tied singular values are +#' not unique. +#' +#' @examples +#' fit <- dlearner(dat_multisource$Y_source, dat_multisource$Y_target, r = 3) +#' plot_contributions(fit,max_display=50) +#' @export +plot_contributions <- function(x, r=NULL, row_index=NULL, column_index=NULL, + max_display=200L, limits=NULL, row_label='Row index', + column_label='Column index', main='Estimated target contributions') { + if(is.list(x)) { + estimate<-if(!is.null(x$learner_estimate))x$learner_estimate else x$dlearner_estimate + if(is.null(r))r<-x$r + } else estimate<-x + check_plot_matrix(estimate,'x') + if(is.null(r))stop('Supply r for a matrix without a stored rank.') + r<-resolve_rank(r,estimate);validate_count(max_display,'max_display') + ri<-plot_indices(row_index,nrow(estimate),max_display,'row_index') + ci<-plot_indices(column_index,ncol(estimate),max_display,'column_index') + s<-plot_svd(estimate,r) + row_scores<-s$u^2;col_scores<-s$v^2 + cap<-plot_limits_pair(limits,c(row=max(row_scores[ri,,drop=FALSE]),column=max(col_scores[ci,,drop=FALSE]))) + cap[cap==0]<-1e-8 + counts<-c(row=sum(row_scores[ri,,drop=FALSE]>cap[1]),column=sum(col_scores[ci,,drop=FALSE]>cap[2])) + colors<-grDevices::colorRampPalette(c('white','firebrick3'))(100) + old<-graphics::par(no.readonly=TRUE);on.exit(graphics::par(old),add=TRUE) + graphics::par(mfrow=c(1,2),mar=c(3.8,3.8,2.2,4),oma=c(2,0,2,0),mgp=c(2.2,0.6,0)) + draw_matrix_panel(col_scores[ci,,drop=FALSE],ci,seq_len(r),c(0,cap[2]),colors,'Column contributions',column_label,'Component') + draw_matrix_panel(row_scores[ri,,drop=FALSE],ri,seq_len(r),c(0,cap[1]),colors,'Row contributions',row_label,'Component') + graphics::mtext(main,outer=TRUE,side=3,line=0.4,font=2,cex=1.1) + caption<-paste('Squared singular-vector entries; separate color scales.', + if(sum(counts)>0)paste(sum(counts),'displayed entries clipped.')else '') + graphics::mtext(caption,outer=TRUE,side=1,line=0.4,cex=0.8) + invisible(list(row_scores=row_scores,column_scores=col_scores,row_index=ri,column_index=ci, + limits=cap,clipped_count=counts)) +} diff --git a/R/plotting.R b/R/plotting.R new file mode 100644 index 0000000..7bf3a82 --- /dev/null +++ b/R/plotting.R @@ -0,0 +1,382 @@ +#' Plot the objective function values of a LEARNER fit +#' +#' Plots the values of the objective function at each iteration of the +#' numerical optimization algorithm in \code{learner}. +#' +#' @param x A result returned by \code{learner}. +#' @param iterations Which iterations to show: \code{"all"} (the default), +#' \code{"near_optimum"} for up to 41 iterations around the smallest recorded +#' value, or a vector of iteration numbers such as \code{1:1000}. Use iteration +#' numbers if you want a different width around the minimum. +#' @param main Title of the plot. The default is \code{"Objective function"}, +#' with \code{"(near optimum)"} added for the near-optimum view. +#' +#' @return Invisibly, a data frame of the plotted iterations and objective +#' values. +#' +#' @details The line shows the recorded objective values, and each iteration is +#' also marked with a dot when 100 or fewer iterations are shown. The red point +#' marks the smallest recorded value when it is in the displayed range. +#' Initialization is not included in this history. A curve that zigzags between +#' two levels means that the objective alternates from one iteration to the next +#' instead of settling; a flat-looking curve alone does not establish +#' convergence. +#' +#' @examples +#' fit <- learner(dat_highsim$Y_source, dat_highsim$Y_target, +#' r = 3, lambda_1 = 1, lambda_2 = 1, step_size = 0.003, +#' control = list(max_iter = 1000)) +#' plot_objective(fit) +#' plot_objective(fit, iterations = "near_optimum") +#' plot_objective(fit, iterations = 1:100) +#' +#' @export +plot_objective <- function(x, iterations = "all", main = NULL) { + objective <- x$objective_values + if (!is.numeric(objective) || length(objective) == 0L || + !any(is.finite(objective))) { + stop('x must be a result returned by learner().') + } + n <- length(objective) + finite <- is.finite(objective) + minimum <- which.min(replace(objective, !finite, Inf)) + near <- identical(iterations, 'near_optimum') + if (identical(iterations, 'all')) { + shown <- seq_len(n) + } else if (near) { + width <- min(n, 41L) + start <- max(1L, min(minimum - 20L, n - width + 1L)) + shown <- seq.int(start, length.out = width) + } else if (is.numeric(iterations) && !is.complex(iterations) && + length(iterations) > 0L && all(is.finite(iterations)) && + all(iterations >= 1 & iterations == floor(iterations))) { + shown <- sort(unique(as.integer(iterations[iterations <= n]))) + if (length(shown) == 0L) { + stop('None of the requested iterations were recorded; the history contains ', n, ' iterations.') + } + } else { + stop("iterations must be 'all', 'near_optimum', or positive integer iteration numbers.") + } + if (!any(finite[shown])) { + stop('The selected iterations contain no finite objective values; choose a different range.') + } + if (is.null(main)) { + main <- if (near) 'Objective function (near optimum)' else 'Objective function' + } + values <- replace(objective[shown], !finite[shown], NA_real_) + + old <- graphics::par(mar = c(5, 4.5, 4, 1.5)) + on.exit(graphics::par(old), add = TRUE) + xlim <- range(shown) + graphics::plot(shown, values, type = 'n', xlim = xlim, xlab = 'Iteration', + ylab = 'Objective value', bty = 'l') + graphics::title(main, line = 2) + graphics::mtext(paste0('Minimum: ', format(objective[minimum], digits = 6), + ' at iteration ', minimum), + side = 3, line = 0.6, cex = 0.8, col = 'gray30') + graphics::lines(shown, values, col = '#2166AC') + # Mark the individual iterations when few are shown, so that values that + # alternate from one iteration to the next can be told apart. + if (length(shown) <= 100L) { + graphics::points(shown, values, pch = 16, cex = 0.6, col = '#2166AC') + } + if (minimum %in% shown) { + graphics::points(minimum, objective[minimum], pch = 16, cex = 1.1, col = '#B2182B') + } + invisible(data.frame(iteration = shown, objective = objective[shown])) +} + +#' Plot the LEARNER cross-validation scores +#' +#' Compare cross-validation errors across candidate penalties. Lighter cells +#' indicate lower error, and a black box marks the selected combination. +#' +#' @param x A result returned by \code{cv.learner()}. +#' @param lambda_1_all Candidate common penalties, in the same order as in +#' \code{cv.learner()}. Leave as \code{NULL} when separate penalties were used. +#' @param lambda_2_all Candidate balance penalties, in the same order as in +#' \code{cv.learner()}. +#' @param lambda_1_row_all,lambda_1_col_all Candidate row and column penalties, +#' in the same order as in \code{cv.learner()}. Both are needed when separate +#' penalties were used. +#' @param n_observed Number of observed target entries used in CV. If given, +#' the scores are shown as mean squared errors; otherwise as sums of squared +#' errors. Do not count missing entries. +#' @param lambda_2_index Which balance-penalty panels to draw when separate +#' penalties were used, as positions in \code{lambda_2_all}. The default draws +#' all of them. +#' @param log_labels If TRUE, label the axes with base-10 logarithms of the +#' candidates, which must then be positive. The default is FALSE. +#' @param relative_limit Cells whose error is more than this fraction above the +#' smallest error are drawn gray and labeled as above the cutoff (default 0.3, +#' or 30 percent). Use \code{NULL} to color every finite value. +#' @param show_values Whether to print scores inside cells. Defaults to TRUE. +#' @param main Plot title. +#' +#' @return Invisibly, a list describing the plot. It includes the \code{scores}, +#' the selected combination (\code{best_index}), flags for selections on a grid +#' boundary (\code{boundary}), and the ggplot pages (\code{plot} is the first +#' page and \code{plots} holds all of them). +#' +#' @details Supply the same penalty grids, in the same order, used in +#' \code{cv.learner()}. With a common space penalty, the axes show that penalty +#' and the balance penalty. With separate penalties, the axes show row and +#' column penalties, with one panel per balance penalty and shared colors. +#' +#' Colors and cell labels show the increase above the smallest CV error. +#' If the minimum is zero, absolute errors are shown instead. Gray cells mark +#' unavailable scores (NA) or errors above \code{relative_limit}; the cutoff +#' affects only the display. When several combinations tie, the first is marked. +#' A selected value at a grid boundary may suggest trying a wider grid. +#' +#' This plot does not check whether the fits converged. +#' Requires ggplot2 >= 3.4.0; install it with \code{install.packages("ggplot2")}. +#' Save a returned page with \code{ggplot2::ggsave(plot = result$plot, ...)}. +#' +#' @examples +#' if (requireNamespace("ggplot2", quietly = TRUE) && +#' utils::packageVersion("ggplot2") >= "3.4.0") { +#' row_grid <- c(0.1, 1) +#' col_grid <- c(0.1, 2) +#' balance_grid <- c(0.1, 1) +#' target <- dat_highsim$Y_target[1:12, 1:8] +#' set.seed(42) +#' cv <- cv.learner(dat_highsim$Y_source[1:12, 1:8], target, +#' r = 2, lambda_1_row_all = row_grid, +#' lambda_1_col_all = col_grid, lambda_2_all = balance_grid, +#' step_size = 0.0003, n_folds = 2, +#' control = list(max_iter = 20)) +#' plot_cv(cv, lambda_1_row_all = row_grid, lambda_1_col_all = col_grid, +#' lambda_2_all = balance_grid, n_observed = sum(!is.na(target))) +#' +#' } +#' +#' @export +plot_cv <- function(x, lambda_1_all = NULL, lambda_2_all = NULL, + lambda_1_row_all = NULL, lambda_1_col_all = NULL, + n_observed = NULL, lambda_2_index = NULL, log_labels = FALSE, + relative_limit = 0.3, show_values = TRUE, + main = 'Cross-validation error') { + d <- prepare_cv_plot(x, lambda_1_all, lambda_2_all, lambda_1_row_all, + lambda_1_col_all, n_observed, lambda_2_index, log_labels) + if (!is.null(relative_limit) && (!is.numeric(relative_limit) || + is.complex(relative_limit) || length(relative_limit) != 1L || + !is.finite(relative_limit) || relative_limit <= 0)) { + stop('relative_limit must be NULL or a finite positive number.') + } + if (!is.logical(show_values) || length(show_values) != 1L || is.na(show_values)) { + stop('show_values must be TRUE or FALSE.') + } + if (!is.character(main) || length(main) != 1L || is.na(main)) { + stop('main must be one nonmissing character string.') + } + if (!requireNamespace("ggplot2", quietly = TRUE)) { + stop('plot_cv() requires the optional ggplot2 package. Install it with install.packages("ggplot2").', + call. = FALSE) + } + if (utils::packageVersion("ggplot2") < "3.4.0") { + stop('plot_cv() requires ggplot2 >= 3.4.0. Update it with install.packages("ggplot2").', + call. = FALSE) + } + separate <- length(dim(d$scores)) == 3L + indices <- as.data.frame(arrayInd(seq_along(d$scores), dim(d$scores))) + names(indices) <- if (separate) c('x_index', 'y_index', 'slice') else c('x_index', 'y_index') + if (!separate) indices$slice <- 1L + indices$score <- as.vector(d$scores) + indices$available <- is.finite(indices$score) + # Relative increases are undefined when the best score is zero. + relative <- d$minimum > 0 + indices$display <- if (relative) indices$score / d$minimum - 1 else indices$score + indices$above_limit <- indices$available & relative & !is.null(relative_limit) + if (!is.null(relative_limit)) indices$above_limit <- indices$above_limit & indices$display > relative_limit + 8 * .Machine$double.eps * max(1, relative_limit) + indices$fill_value <- indices$display + indices$fill_value[!indices$available | indices$above_limit] <- NA_real_ + finite_fill <- indices$fill_value[is.finite(indices$fill_value)] + upper <- max(finite_fill) + limits <- c(0, if (upper > 0) upper else if (relative) 0.01 else 1e-8) + # One global scale across all slices, even if only a subset is displayed. + label_digits <- if (limits[2] <= 0.05) 2L else 1L + indices$label <- if (relative) sprintf(paste0('%.', label_digits, 'f%%'), 100 * indices$display) else + formatC(indices$display, digits = 4, format = 'g') + indices$label[!indices$available] <- 'NA' + if (any(indices$above_limit)) indices$label[indices$above_limit] <- + paste0('>', format(100 * relative_limit, trim = TRUE), '%') + indices$text_color <- ifelse(is.finite(indices$fill_value) & + indices$fill_value > limits[2] * 0.55, 'white', '#202020') + best_slice <- if (separate) d$best_index[3] else 1L + indices$selected <- indices$x_index == d$best_index[1] & + indices$y_index == d$best_index[2] & indices$slice == best_slice + panels <- if (separate) d$shown_lambda_2_index else 1L + labels_x <- if (separate) d$grids$row else d$grids$common + labels_y <- if (separate) d$grids$col else d$grids$balance + xlab <- if (separate) expression(lambda[1*','*plain(row)]) else expression(lambda[1]) + ylab <- if (separate) expression(lambda[1*','*plain(col)]) else expression(lambda[2]) + if (log_labels) { + labels_x <- round(log10(labels_x), 2) + labels_y <- round(log10(labels_y), 2) + xlab <- bquote(log[10](.(xlab[[1]]))) + ylab <- bquote(log[10](.(ylab[[1]]))) + } + metric <- if (is.null(n_observed)) 'SSE' else 'MSE' + selected_values <- vapply(seq_along(d$grids), function(i) d$grids[[i]][d$best_index[i]], numeric(1)) + selected_names <- if (separate) c('row', 'col', 'balance') else c('lambda1', 'lambda2') + subtitle <- paste0('Selected ', paste(paste0(selected_names, ' = ', + trimws(formatC(selected_values, digits = 4, format = 'g'))), collapse = ', '), + ' | Minimum ', metric, ' = ', formatC(d$minimum, digits = 6, format = 'g')) + caption <- paste0('Black box: selected parameters. Cell labels rounded.\n', metric, ' range: ', + formatC(d$score_range[1], digits = 6, format = 'g'), ' to ', + formatC(d$score_range[2], digits = 6, format = 'g'), '.') + if (any(d$boundary)) caption <- paste0(caption, ' Selection on grid boundary.') + if (d$exact_minima > 1) caption <- paste0(caption, '\n', d$exact_minima, + ' exact minima; first in CV order selected.') + if (any(indices$above_limit)) caption <- paste0(caption, '\nGray >', + format(100 * relative_limit, trim = TRUE), '%: high error, not a divergence diagnosis.') + if (d$unavailable_count > 0) caption <- paste0(caption, '\nGray NA: unavailable score.') + if (!relative) caption <- paste0(caption, '\nMinimum is zero; colors and labels show absolute ', metric, '.') + breaks <- seq(0, limits[2], length.out = 5L) + key_labels <- if (relative) sprintf(paste0('%.', label_digits, 'f%%'), 100 * breaks) else + formatC(breaks, digits = 4, format = 'g') + # Declare aesthetic bindings for package checks; ggplot evaluates them in data. + x_index <- y_index <- fill_value <- label <- text_color <- NULL + plots <- list() + for (page in split(panels, ceiling(seq_along(panels) / 6))) { + data <- indices[indices$slice %in% page, , drop = FALSE] + facet_labels <- paste0('lambda[2] == ', format(d$grids$balance[page], digits = 4, trim = TRUE)) + if (anyDuplicated(facet_labels)) { + facet_labels <- paste0('lambda[2] == ', format(d$grids$balance[page], digits = 17, trim = TRUE)) + } + # Grid indices identify panels; rounded display labels must not merge them. + data$panel <- factor(data$slice, levels = page) + panel_labels <- stats::setNames(facet_labels, as.character(page)) + plot <- ggplot2::ggplot(data, ggplot2::aes(x = x_index, y = y_index, fill = fill_value)) + + ggplot2::geom_tile(color = 'white', linewidth = 0.35, width = 1, height = 1) + + ggplot2::scale_fill_viridis_c(option = 'magma', direction = -1, + na.value = 'grey85', limits = limits, breaks = breaks, labels = key_labels, + name = if (relative) paste0('CV ', metric, '\nabove minimum') else paste('CV', metric)) + + ggplot2::scale_x_continuous(breaks = seq_along(labels_x), labels = labels_x, + expand = ggplot2::expansion(mult = 0)) + + ggplot2::scale_y_continuous(breaks = seq_along(labels_y), labels = labels_y, + expand = ggplot2::expansion(mult = 0)) + + ggplot2::labs(title = main, subtitle = subtitle, x = xlab, y = ylab, caption = caption) + + ggplot2::theme_minimal(base_size = 11) + + ggplot2::theme(panel.grid = ggplot2::element_blank(), + plot.title = ggplot2::element_text(face = 'bold', size = 14), + plot.subtitle = ggplot2::element_text(size = 10, margin = ggplot2::margin(b = 10)), + plot.caption = ggplot2::element_text(size = 9, hjust = 0, margin = ggplot2::margin(t = 10)), + legend.title = ggplot2::element_text(size = 10), + legend.text = ggplot2::element_text(size = 9), + axis.text = ggplot2::element_text(color = '#303030', size = 10), + strip.text = ggplot2::element_text(size = 11), + plot.margin = ggplot2::margin(12, 18, 10, 12)) + if (show_values) plot <- plot + + ggplot2::geom_text(ggplot2::aes(label = label, color = text_color), size = 2.8) + + ggplot2::scale_color_identity() + plot <- plot + ggplot2::geom_tile(data = data[data$selected, , drop = FALSE], + fill = NA, color = 'black', linewidth = 0.9, width = 0.94, height = 0.94) + if (separate) plot <- plot + ggplot2::facet_wrap(~panel, ncol = min(3L, length(page)), + labeller = ggplot2::as_labeller(panel_labels, default = ggplot2::label_parsed)) + print(plot) + plots[[length(plots) + 1L]] <- plot + } + d$display_data <- indices + d$display_limits <- limits + d$relative <- relative + d$above_limit_count <- sum(indices$above_limit) + d$plots <- plots + d$plot <- plots[[1L]] + invisible(d) +} + +prepare_cv_plot <- function(x, lambda_1_all = NULL, lambda_2_all = NULL, + lambda_1_row_all = NULL, lambda_1_col_all = NULL, + n_observed = NULL, lambda_2_index = NULL, log_labels = FALSE) { + if (!is.list(x) || !is.numeric(x$mse_all) || is.complex(x$mse_all) || + !(length(dim(x$mse_all)) %in% c(2L, 3L)) || any(dim(x$mse_all) < 1L)) { + stop('x must contain a nonempty two- or three-dimensional mse_all array.') + } + if (!is.logical(log_labels) || length(log_labels) != 1L || is.na(log_labels)) { + stop('log_labels must be TRUE or FALSE.') + } + scores <- x$mse_all + finite <- is.finite(scores) + if (!any(finite)) stop('No finite validation scores are available to plot.') + if (any(scores[finite] < 0)) stop('Validation squared-error scores cannot be negative.') + separate <- length(dim(scores)) == 3L + check_grid <- function(values, size, name) { + validate_penalty_grid(values, name) + if (!is.null(dim(values)) || length(values) != size) { + stop(name, ' must match its score-array dimension in the original CV order.') + } + } + if (separate) { + if (!is.null(lambda_1_all)) stop('Do not supply lambda_1_all for separate penalties.') + check_grid(lambda_1_row_all, dim(scores)[1], 'lambda_1_row_all') + check_grid(lambda_1_col_all, dim(scores)[2], 'lambda_1_col_all') + check_grid(lambda_2_all, dim(scores)[3], 'lambda_2_all') + grids <- list(row = lambda_1_row_all, col = lambda_1_col_all, + balance = lambda_2_all) + panels <- if (is.null(lambda_2_index)) seq_along(lambda_2_all) else lambda_2_index + if (!is.numeric(panels) || length(panels) == 0L || any(!is.finite(panels)) || + any(panels != floor(panels)) || any(panels < 1 | panels > length(lambda_2_all)) || + anyDuplicated(panels)) { + stop('lambda_2_index must contain distinct valid balance-grid indices.') + } + } else { + if (!is.null(lambda_1_row_all) || !is.null(lambda_1_col_all) || + !is.null(lambda_2_index)) stop('Separate-penalty arguments require a 3D score array.') + check_grid(lambda_1_all, dim(scores)[1], 'lambda_1_all') + check_grid(lambda_2_all, dim(scores)[2], 'lambda_2_all') + grids <- list(common = lambda_1_all, balance = lambda_2_all) + panels <- 1L + } + axis_grids <- if (separate) c(grids$row, grids$col) else c(grids$common, grids$balance) + if (log_labels && any(axis_grids <= 0)) { + stop('log_labels requires strictly positive candidates on both heatmap axes.') + } + selectable <- scores + selectable[!finite] <- Inf + best <- as.integer(arrayInd(which.min(selectable), dim(scores))) + expected <- if (separate) { + c(lambda_1_row_min = grids$row[best[1]], lambda_1_col_min = grids$col[best[2]], + lambda_2_min = grids$balance[best[3]]) + } else c(lambda_1_min = grids$common[best[1]], lambda_2_min = grids$balance[best[2]]) + for (name in names(expected)) { + if (!is.numeric(x[[name]]) || length(x[[name]]) != 1L || + !isTRUE(all.equal(unname(x[[name]]), unname(expected[[name]]), tolerance = 1e-12))) { + stop('Supplied grids do not match the selected penalties stored in x: ', name) + } + } + scale <- 'Validation SSE' + if (!is.null(n_observed)) { + validate_count(n_observed, 'n_observed', upper = Inf) + scores <- scores / n_observed + scale <- 'Validation MSE' + } + limits <- range(scores[finite]) + if (diff(limits) == 0) { + pad <- max(abs(limits[1]) * 0.01, 1e-8) + limits <- c(max(0, limits[1] - pad), limits[2] + pad) + } + score_range <- range(scores[finite]) + minimum <- min(scores[finite]) + boundary <- vapply(seq_along(grids), function(i) { + g <- grids[[i]] + length(unique(g)) > 1L && g[best[i]] %in% range(g) + }, logical(1)) + names(boundary) <- names(grids) + slice <- if (separate) scores[best[1], best[2], ] else scores[best[1], ] + list(scores = scores, scale = scale, grids = grids, best_index = best, + shown_lambda_2_index = if (separate) panels else NULL, + color_limits = limits, unavailable_count = sum(!finite), log_labels = log_labels, + score_range = score_range, minimum = minimum, exact_minima = sum(scores[finite] == minimum), + boundary = boundary, lambda_2_slice_range = range(slice[is.finite(slice)])) +} + +# Five ticks include the numerical endpoints even for a narrow score range. +draw_color_key <- function(left, right, bottom, top, limits, colors) { + edges <- seq(bottom, top, length.out=length(colors)+1L) + graphics::rect(left,edges[-length(edges)],right,edges[-1],col=colors,border=NA) + ticks <- seq(limits[1],limits[2],length.out=5) + graphics::axis(4,pos=right,at=seq(bottom,top,length.out=5), + labels=formatC(ticks,format='g',digits=5),las=1,cex.axis=0.9,tck=-0.015) +} diff --git a/README.Rmd b/README.Rmd index 595e3ff..4f8c385 100644 --- a/README.Rmd +++ b/README.Rmd @@ -4,11 +4,12 @@ output: github_document -```{r, include = FALSE} +```{r setup, include = FALSE} knitr::opts_chunk$set( collapse = TRUE, comment = "#>", - out.width = "100%" + out.width = "100%", fig.width = 8, fig.height = 5, + fig.path = "README_files/figure-gfm/" ) ``` @@ -36,10 +37,16 @@ You can install the development version of `learner` from [GitHub](https://githu devtools::install_github("stmcg/learner") ``` +`plot_cv()` needs the ggplot2 package (version 3.4.0 or later): + +``` r +install.packages("ggplot2") +``` + ## Example We illustrate an example of how `learner` can be used. We first load the package. -```{r, results="hide", message=FALSE} +```{r load-package, results="hide", message=FALSE} library(learner) ``` @@ -47,45 +54,60 @@ In this illustration, we will use one of the toy data sets in the package (`dat_ #### LEARNER Method -We can apply the LEARNER method via the `learner` function. +We can apply the LEARNER method via the `learner` function. This method allows for flexible patterns of heterogeneity across the source and target populations. It consequently requires specifying the tuning parameters `lambda_1` and `lambda_2` which control the degree of transfer learning between the populations. These values can be selected based on cross-validation via the `cv.learner` function. For example, we can specify candidate values of 1, 10, and 100 for `lambda_1` and `lambda_2` and select the optimal values based on cross-validation as follows: -```{r} -res_cv.learner <- cv.learner(Y_source = dat_highsim$Y_source, - Y_target = dat_highsim$Y_target, - lambda_1_all = c(1, 10, 100), - lambda_2_all = c(1, 10, 100), +```{r single-source-cv} +set.seed(815) +res_cv.learner <- cv.learner(Y_source = dat_highsim$Y_source, + Y_target = dat_highsim$Y_target, + lambda_1_all = c(1, 10, 100), + lambda_2_all = c(1, 10, 100), step_size = 0.003) res_cv.learner$lambda_1_min res_cv.learner$lambda_2_min ``` -Next, we apply the `learner` function with these values of `lambda_1` and `lambda_2`: -```{r} -res_learner <- learner(Y_source = dat_highsim$Y_source, +The heatmap shows the cross-validation error for each candidate combination. +Lighter cells indicate lower error; the black box marks the selected combination. +This plot requires the optional package `ggplot2` (version 3.4.0 or later). + +```{r cv-heatmap} +plot_cv(res_cv.learner, lambda_1_all = c(1, 10, 100), + lambda_2_all = c(1, 10, 100), + n_observed = sum(!is.na(dat_highsim$Y_target))) +``` + +Next, we apply the `learner` function with these values of `lambda_1` and `lambda_2`: +```{r single-source-fit} +res_learner <- learner(Y_source = dat_highsim$Y_source, Y_target = dat_highsim$Y_target, - lambda_1 = 100, lambda_2 = 1, + lambda_1 = res_cv.learner$lambda_1_min, + lambda_2 = res_cv.learner$lambda_2_min, step_size = 0.003) ``` The LEARNER estimate is given by the `learner_estimate` component in the output of the `learner` function, e.g., -```{r} +```{r single-source-estimate} res_learner$learner_estimate[1:5, 1:5] ``` -We can check the convergence of the numerical optimization algorithm by plotting the values of the objective function at each iteration: -```{r, out.width="75%"} -plot(res_learner$objective_values, - xlab = 'Iteration', ylab = 'Objective function', - type = 'l', col = 'blue', lwd = 2) -``` +We can inspect how the objective function changes during optimization with +`plot_objective()`. A decreasing curve shows progress toward a smaller objective; +a flat-looking curve alone does not establish convergence. +```{r objective-history, out.width="75%"} +plot_objective(res_learner) +``` #### D-LEARNER Method We can apply the D-LEARNER method via the `dlearner` function. This method makes stronger assumptions on the heterogeneity across the source and target populations. It consequently does not rely on choosing tuning parameters. The `dlearner` function can be applied as follows: -```{r} -res_dlearner <- dlearner(Y_source = dat_highsim$Y_source, +```{r single-source-direct} +res_dlearner <- dlearner(Y_source = dat_highsim$Y_source, Y_target = dat_highsim$Y_target) res_dlearner$dlearner_estimate[1:5, 1:5] ``` +## Advanced examples +For multiple sources, source weights, and separate row and column penalties, +see the [advanced tutorial](vignettes/advanced-learner.Rmd). diff --git a/README.md b/README.md index a059c2c..e9ef4cf 100644 --- a/README.md +++ b/README.md @@ -36,6 +36,12 @@ You can install the development version of `learner` from devtools::install_github("stmcg/learner") ``` +`plot_cv()` needs the ggplot2 package (version 3.4.0 or later): + +``` r +install.packages("ggplot2") +``` + ## Example We illustrate an example of how `learner` can be used. We first load the @@ -67,10 +73,11 @@ can specify candidate values of 1, 10, and 100 for `lambda_1` and follows: ``` r -res_cv.learner <- cv.learner(Y_source = dat_highsim$Y_source, - Y_target = dat_highsim$Y_target, - lambda_1_all = c(1, 10, 100), - lambda_2_all = c(1, 10, 100), +set.seed(815) +res_cv.learner <- cv.learner(Y_source = dat_highsim$Y_source, + Y_target = dat_highsim$Y_target, + lambda_1_all = c(1, 10, 100), + lambda_2_all = c(1, 10, 100), step_size = 0.003) res_cv.learner$lambda_1_min #> [1] 100 @@ -78,13 +85,27 @@ res_cv.learner$lambda_2_min #> [1] 1 ``` +The heatmap shows the cross-validation error for each candidate +combination. Lighter cells indicate lower error; the black box marks the +selected combination. This plot requires the optional package `ggplot2` +(version 3.4.0 or later). + +``` r +plot_cv(res_cv.learner, lambda_1_all = c(1, 10, 100), + lambda_2_all = c(1, 10, 100), + n_observed = sum(!is.na(dat_highsim$Y_target))) +``` + + + Next, we apply the `learner` function with these values of `lambda_1` and `lambda_2`: ``` r -res_learner <- learner(Y_source = dat_highsim$Y_source, +res_learner <- learner(Y_source = dat_highsim$Y_source, Y_target = dat_highsim$Y_target, - lambda_1 = 100, lambda_2 = 1, + lambda_1 = res_cv.learner$lambda_1_min, + lambda_2 = res_cv.learner$lambda_2_min, step_size = 0.003) ``` @@ -101,16 +122,16 @@ res_learner$learner_estimate[1:5, 1:5] #> [5,] 0.5046133 0.01845138 -1.4798209 -0.765601712 -0.2230227 ``` -We can check the convergence of the numerical optimization algorithm by -plotting the values of the objective function at each iteration: +We can inspect how the objective function changes during optimization +with `plot_objective()`. A decreasing curve shows progress toward a +smaller objective; a flat-looking curve alone does not establish +convergence. ``` r -plot(res_learner$objective_values, - xlab = 'Iteration', ylab = 'Objective function', - type = 'l', col = 'blue', lwd = 2) +plot_objective(res_learner) ``` - + #### D-LEARNER Method @@ -120,7 +141,7 @@ and target populations. It consequently does not rely on choosing tuning parameters. The `dlearner` function can be applied as follows: ``` r -res_dlearner <- dlearner(Y_source = dat_highsim$Y_source, +res_dlearner <- dlearner(Y_source = dat_highsim$Y_source, Y_target = dat_highsim$Y_target) res_dlearner$dlearner_estimate[1:5, 1:5] #> [,1] [,2] [,3] [,4] [,5] @@ -130,3 +151,8 @@ res_dlearner$dlearner_estimate[1:5, 1:5] #> [4,] 1.6347696 -2.91985925 0.04876288 -2.357504089 0.8127403 #> [5,] 0.4757450 0.05665543 -1.50373470 -0.744076401 -0.2876033 ``` + +## Advanced examples + +For multiple sources, source weights, and separate row and column +penalties, see the [advanced tutorial](vignettes/advanced-learner.Rmd). diff --git a/README_files/figure-gfm/cv-heatmap-1.png b/README_files/figure-gfm/cv-heatmap-1.png new file mode 100644 index 0000000..45931dc Binary files /dev/null and b/README_files/figure-gfm/cv-heatmap-1.png differ diff --git a/README_files/figure-gfm/objective-history-1.png b/README_files/figure-gfm/objective-history-1.png new file mode 100644 index 0000000..113c9e5 Binary files /dev/null and b/README_files/figure-gfm/objective-history-1.png differ diff --git a/README_files/figure-gfm/objective-near-optimum.png b/README_files/figure-gfm/objective-near-optimum.png new file mode 100644 index 0000000..71c8919 Binary files /dev/null and b/README_files/figure-gfm/objective-near-optimum.png differ diff --git a/README_files/figure-gfm/unnamed-chunk-6-1.png b/README_files/figure-gfm/unnamed-chunk-6-1.png deleted file mode 100644 index 360e568..0000000 Binary files a/README_files/figure-gfm/unnamed-chunk-6-1.png and /dev/null differ diff --git a/README_files/figure-gfm/unnamed-chunk-7-1.png b/README_files/figure-gfm/unnamed-chunk-7-1.png deleted file mode 100644 index 360e568..0000000 Binary files a/README_files/figure-gfm/unnamed-chunk-7-1.png and /dev/null differ diff --git a/data-raw/gen-asymmetric.R b/data-raw/gen-asymmetric.R new file mode 100644 index 0000000..ff2a86e --- /dev/null +++ b/data-raw/gen-asymmetric.R @@ -0,0 +1,21 @@ +# Adapted from simulations/helper-uhigh-vlow.R, set_theta_Ushared_Vdiff(), +# in stmcg/learner-paper, commit cb092c7e239b9a7b8eb8fcea80bce26f8017be86. +# Uses dif_factor_v = 2 from runsim-uhigh-vlow.R. Smaller dimensions and fixed +# rank make this a package example, not a reproduction of the simulation study. +local({ + set.seed(20261022) + p <- 100L; q <- 50L; r <- 3L + basis <- svd(matrix(rnorm(p * q), p, q), nu = r, nv = r) + strengths <- basis$d[seq_len(r)] + Theta_target <- basis$u %*% diag(strengths) %*% t(basis$v) + threshold_v <- 1 / (2 * sqrt(q)) + delta_v <- matrix(runif(q * r, -threshold_v, threshold_v), q, r) + source_v <- qr.Q(qr(basis$v + delta_v)) + Theta_source <- basis$u %*% diag(rev(strengths)) %*% t(source_v) + # Independent population noise; arguments to rnorm() are standard deviations. + Y_target <- Theta_target + matrix(rnorm(p * q, sd = sqrt(0.10)), p, q) + Y_source <- Theta_source + matrix(rnorm(p * q, sd = sqrt(0.01)), p, q) + dat_asymmetric <- list(Y_source = Y_source, Y_target = Y_target, + Theta_source = Theta_source, Theta_target = Theta_target) + usethis::use_data(dat_asymmetric, overwrite = TRUE, version = 2) +}) diff --git a/data-raw/gen-data.R b/data-raw/gen-data.R index 53a7863..9e47feb 100644 --- a/data-raw/gen-data.R +++ b/data-raw/gen-data.R @@ -90,3 +90,6 @@ local({ ) usethis::use_data(dat_multisource, overwrite = TRUE, version = 2) }) + +# To generate the asymmetric example independently, run: +# source("data-raw/gen-asymmetric.R") diff --git a/data/dat_asymmetric.rda b/data/dat_asymmetric.rda new file mode 100644 index 0000000..99f235f Binary files /dev/null and b/data/dat_asymmetric.rda differ diff --git a/inst/examples/plot-diagnostics.R b/inst/examples/plot-diagnostics.R new file mode 100644 index 0000000..9dd32c1 --- /dev/null +++ b/inst/examples/plot-diagnostics.R @@ -0,0 +1,43 @@ +# This heatmap demonstration requires the optional ggplot2 package. +if (!requireNamespace("ggplot2", quietly = TRUE) || + utils::packageVersion("ggplot2") < "3.4.0") { + stop('Install ggplot2 >= 3.4.0 to run this example: install.packages("ggplot2")') +} +# Synthetic data demonstrate the interface, not statistical performance. +# Use a small aligned subset to keep this demonstration quick. +sources <- lapply(learner::dat_multisource$Y_source, + function(x) x[1:20, 1:10]) +target <- learner::dat_multisource$Y_target[1:20, 1:10] +weights <- c(0.7, 0.3) +row_grid <- c(0, 1, 10) +col_grid <- c(0, 1, 3) +balance_grid <- c(0.1, 1) + +set.seed(815) +cv <- learner::cv.learner( + sources, target, r = 3, source_weights = weights, + lambda_1_row_all = row_grid, lambda_1_col_all = col_grid, + lambda_2_all = balance_grid, step_size = 0.0003, n_folds = 3, + control = list(max_iter = 500)) +learner::plot_cv( + cv, lambda_1_row_all = row_grid, lambda_1_col_all = col_grid, + lambda_2_all = balance_grid, n_observed = sum(!is.na(target))) + +fit <- learner::learner( + sources, target, r = cv$r, source_weights = weights, + lambda_1_row = cv$lambda_1_row_min, lambda_1_col = cv$lambda_1_col_min, + lambda_2 = cv$lambda_2_min, step_size = 0.0003, + control = list(max_iter = 2000)) +learner::plot_objective(fit, iterations = "near_optimum") + +# Common-penalty results are supported as well. +set.seed(815) +common_grid <- c(0, 1, 10) +common_cv <- learner::cv.learner( + sources, target, r = 3, source_weights = weights, + lambda_1_all = common_grid, lambda_2_all = balance_grid, + step_size = 0.0003, n_folds = 3, control = list(max_iter = 500)) +learner::plot_cv(common_cv, lambda_1_all = common_grid, + lambda_2_all = balance_grid, n_observed = sum(!is.na(target))) + +# Inspect stopping codes as well as curves. Plotting does not prove convergence. diff --git a/inst/examples/plot-paper-displays.R b/inst/examples/plot-paper-displays.R new file mode 100644 index 0000000..59b9b9d --- /dev/null +++ b/inst/examples/plot-paper-displays.R @@ -0,0 +1,7 @@ +# Population spaces and contributions using simulated multiple-source data. +space_target <- learner::dat_multisource$Y_target +space_sources <- learner::dat_multisource$Y_source +learner::plot_source_spaces(space_target, space_sources, r=3, + target_name='Target (simulated)') +space_fit <- learner::dlearner(space_sources,space_target,r=3,source_weights=c(0.7,0.3)) +learner::plot_contributions(space_fit, main='D-LEARNER contributions (simulated data)') diff --git a/man/dat_asymmetric.Rd b/man/dat_asymmetric.Rd new file mode 100644 index 0000000..fea468b --- /dev/null +++ b/man/dat_asymmetric.Rd @@ -0,0 +1,65 @@ +% Generated by roxygen2: do not edit by hand +% Please edit documentation in R/data-asymmetric.R +\docType{data} +\name{dat_asymmetric} +\alias{dat_asymmetric} +\title{Simulated data set: One shared latent space} +\format{ +A list of four aligned matrices, each with 100 rows and 50 columns: +\describe{ + \item{Y_source}{Observed source data.} + \item{Y_target}{Observed target data.} + \item{Theta_source}{True rank-3 source signal.} + \item{Theta_target}{True rank-3 target signal.} +} +} +\usage{ +dat_asymmetric +} +\description{ +One source and one target with the same true U-space but different true +V-spaces. This example illustrates why the two space penalties in LEARNER +can be selected separately. +} +\details{ +The target signal is the rank-3 truncated SVD of a standard normal +matrix. The source keeps its U-space, reverses its three singular values, +and perturbs its V basis with independent uniform entries between +\eqn{-1/(2\sqrt{50})} and \eqn{1/(2\sqrt{50})}, followed by QR orthonormalization. +This adapts the U-shared, V-perturbed "low similarity" scenario in Section +2.4.1 of the Supporting Information of McGrath et al. (2026) to a small +example (the paper uses 500 by 500 matrices of rank 4). It is not a +replication of that study. The Supporting Information calls this setting +"same column spaces but different row spaces": the span of U is the column +space of the signal matrix, and the span of V is its row space. + +Independent mean-zero normal noise is added to each signal, with variance +0.10 for the target and 0.01 for the source. The exact shared-space statement +applies to the true signals; noise also affects estimated spaces. +True signals are supplied to illustrate estimation error, not for fitting +or penalty selection. Generation code is in \file{data-raw/gen-asymmetric.R} +in the source repository. +} +\examples{ +data(dat_asymmetric) +d <- dat_asymmetric +error <- function(fit) norm(fit$learner_estimate - d$Theta_target, "F") +# Equal penalties versus stronger borrowing on the shared U side. The values +# are illustrative and were not selected by cross-validation; see +# vignette("advanced-learner") for selecting them. +common <- learner(d$Y_source, d$Y_target, r = 3, lambda_1 = 10, + lambda_2 = 0.1, step_size = 0.003, + control = list(max_iter = 2000)) +separate <- learner(d$Y_source, d$Y_target, r = 3, lambda_1_row = 100, + lambda_1_col = 10, lambda_2 = 0.1, step_size = 0.003, + control = list(max_iter = 2000)) +c(common = error(common), separate = error(separate)) +} +\references{ +McGrath, S., Zhu, C., O'Dea, R., Guo, M. and Duan, R. (2026). \emph{LEARNER: A transfer learning method for low-rank matrix estimation}. Statistics in Medicine, 45(15-17), e70669. \doi{10.1002/sim.70669}. +} +\seealso{ +\code{\link{dat_highsim}}, \code{\link{dat_multisource}}, + \code{\link{learner}}, \code{\link{cv.learner}} +} +\keyword{datasets} diff --git a/man/plot_contributions.Rd b/man/plot_contributions.Rd new file mode 100644 index 0000000..37683c6 --- /dev/null +++ b/man/plot_contributions.Rd @@ -0,0 +1,66 @@ +% Generated by roxygen2: do not edit by hand +% Please edit documentation in R/plot-spaces.R +\name{plot_contributions} +\alias{plot_contributions} +\title{Plot variable contributions to latent components} +\usage{ +plot_contributions( + x, + r = NULL, + row_index = NULL, + column_index = NULL, + max_display = 200L, + limits = NULL, + row_label = "Row index", + column_label = "Column index", + main = "Estimated target contributions" +) +} +\arguments{ +\item{x}{A complete estimated matrix, or a \code{learner()} or +\code{dlearner()} result. The singular vectors are not rotated.} + +\item{r}{Number of components. Required for a matrix; for a fitting result +the default is its rank.} + +\item{row_index, column_index}{Optional indices of the rows and columns to +display. The singular vectors are always computed from the full matrices.} + +\item{max_display}{Maximum number of rows, and of columns, to display when +no indices are given (default 200). Evenly spaced indices are used.} + +\item{limits}{Optional pair of positive upper limits for the color scales +(row side, then column side). Each scale runs from 0 (white) to its limit +(dark red). Larger values get the darkest color, and their number is returned +as \code{clipped_count}.} + +\item{row_label, column_label}{Axis labels. The defaults are generic, so use +labels that describe your rows and columns.} + +\item{main}{Plot title.} +} +\value{ +Invisibly, a list with the full \code{row_scores} and +\code{column_scores} (variables in rows, components in columns), the displayed +\code{row_index} and \code{column_index}, \code{limits}, and +\code{clipped_count}. The returned scores are not clipped. +} +\description{ +Show which rows and columns contribute to each latent component of a fitted +target matrix, using squared singular-vector entries. Higher scores are darker red. +} +\details{ +Darker red indicates a larger contribution to a component. +Columns are shown on the left and rows on the right, with separate color +scales. Scores are squared singular-vector entries from the estimated matrix; +they sum to one over all variables on each side for each component. + +Scores describe contributions, not effect sizes or correlations. Selecting +variables changes only the display, without rescaling scores. Components are +not matched across populations, and components with tied singular values are +not unique. +} +\examples{ +fit <- dlearner(dat_multisource$Y_source, dat_multisource$Y_target, r = 3) +plot_contributions(fit,max_display=50) +} diff --git a/man/plot_cv.Rd b/man/plot_cv.Rd new file mode 100644 index 0000000..70e9d20 --- /dev/null +++ b/man/plot_cv.Rd @@ -0,0 +1,97 @@ +% Generated by roxygen2: do not edit by hand +% Please edit documentation in R/plotting.R +\name{plot_cv} +\alias{plot_cv} +\title{Plot the LEARNER cross-validation scores} +\usage{ +plot_cv( + x, + lambda_1_all = NULL, + lambda_2_all = NULL, + lambda_1_row_all = NULL, + lambda_1_col_all = NULL, + n_observed = NULL, + lambda_2_index = NULL, + log_labels = FALSE, + relative_limit = 0.3, + show_values = TRUE, + main = "Cross-validation error" +) +} +\arguments{ +\item{x}{A result returned by \code{cv.learner()}.} + +\item{lambda_1_all}{Candidate common penalties, in the same order as in +\code{cv.learner()}. Leave as \code{NULL} when separate penalties were used.} + +\item{lambda_2_all}{Candidate balance penalties, in the same order as in +\code{cv.learner()}.} + +\item{lambda_1_row_all, lambda_1_col_all}{Candidate row and column penalties, +in the same order as in \code{cv.learner()}. Both are needed when separate +penalties were used.} + +\item{n_observed}{Number of observed target entries used in CV. If given, +the scores are shown as mean squared errors; otherwise as sums of squared +errors. Do not count missing entries.} + +\item{lambda_2_index}{Which balance-penalty panels to draw when separate +penalties were used, as positions in \code{lambda_2_all}. The default draws +all of them.} + +\item{log_labels}{If TRUE, label the axes with base-10 logarithms of the +candidates, which must then be positive. The default is FALSE.} + +\item{relative_limit}{Cells whose error is more than this fraction above the +smallest error are drawn gray and labeled as above the cutoff (default 0.3, +or 30 percent). Use \code{NULL} to color every finite value.} + +\item{show_values}{Whether to print scores inside cells. Defaults to TRUE.} + +\item{main}{Plot title.} +} +\value{ +Invisibly, a list describing the plot. It includes the \code{scores}, +the selected combination (\code{best_index}), flags for selections on a grid +boundary (\code{boundary}), and the ggplot pages (\code{plot} is the first +page and \code{plots} holds all of them). +} +\description{ +Compare cross-validation errors across candidate penalties. Lighter cells +indicate lower error, and a black box marks the selected combination. +} +\details{ +Supply the same penalty grids, in the same order, used in +\code{cv.learner()}. With a common space penalty, the axes show that penalty +and the balance penalty. With separate penalties, the axes show row and +column penalties, with one panel per balance penalty and shared colors. + +Colors and cell labels show the increase above the smallest CV error. +If the minimum is zero, absolute errors are shown instead. Gray cells mark +unavailable scores (NA) or errors above \code{relative_limit}; the cutoff +affects only the display. When several combinations tie, the first is marked. +A selected value at a grid boundary may suggest trying a wider grid. + +This plot does not check whether the fits converged. +Requires ggplot2 >= 3.4.0; install it with \code{install.packages("ggplot2")}. +Save a returned page with \code{ggplot2::ggsave(plot = result$plot, ...)}. +} +\examples{ +if (requireNamespace("ggplot2", quietly = TRUE) && + utils::packageVersion("ggplot2") >= "3.4.0") { +row_grid <- c(0.1, 1) +col_grid <- c(0.1, 2) +balance_grid <- c(0.1, 1) +target <- dat_highsim$Y_target[1:12, 1:8] +set.seed(42) +cv <- cv.learner(dat_highsim$Y_source[1:12, 1:8], target, + r = 2, lambda_1_row_all = row_grid, + lambda_1_col_all = col_grid, lambda_2_all = balance_grid, + step_size = 0.0003, n_folds = 2, + control = list(max_iter = 20)) +plot_cv(cv, lambda_1_row_all = row_grid, lambda_1_col_all = col_grid, + lambda_2_all = balance_grid, n_observed = sum(!is.na(target))) + +} + +} diff --git a/man/plot_objective.Rd b/man/plot_objective.Rd new file mode 100644 index 0000000..1b07be6 --- /dev/null +++ b/man/plot_objective.Rd @@ -0,0 +1,45 @@ +% Generated by roxygen2: do not edit by hand +% Please edit documentation in R/plotting.R +\name{plot_objective} +\alias{plot_objective} +\title{Plot the objective function values of a LEARNER fit} +\usage{ +plot_objective(x, iterations = "all", main = NULL) +} +\arguments{ +\item{x}{A result returned by \code{learner}.} + +\item{iterations}{Which iterations to show: \code{"all"} (the default), +\code{"near_optimum"} for up to 41 iterations around the smallest recorded +value, or a vector of iteration numbers such as \code{1:1000}. Use iteration +numbers if you want a different width around the minimum.} + +\item{main}{Title of the plot. The default is \code{"Objective function"}, +with \code{"(near optimum)"} added for the near-optimum view.} +} +\value{ +Invisibly, a data frame of the plotted iterations and objective +values. +} +\description{ +Plots the values of the objective function at each iteration of the +numerical optimization algorithm in \code{learner}. +} +\details{ +The line shows the recorded objective values, and each iteration is +also marked with a dot when 100 or fewer iterations are shown. The red point +marks the smallest recorded value when it is in the displayed range. +Initialization is not included in this history. A curve that zigzags between +two levels means that the objective alternates from one iteration to the next +instead of settling; a flat-looking curve alone does not establish +convergence. +} +\examples{ +fit <- learner(dat_highsim$Y_source, dat_highsim$Y_target, + r = 3, lambda_1 = 1, lambda_2 = 1, step_size = 0.003, + control = list(max_iter = 1000)) +plot_objective(fit) +plot_objective(fit, iterations = "near_optimum") +plot_objective(fit, iterations = 1:100) + +} diff --git a/man/plot_source_spaces.Rd b/man/plot_source_spaces.Rd new file mode 100644 index 0000000..97d5410 --- /dev/null +++ b/man/plot_source_spaces.Rd @@ -0,0 +1,69 @@ +% Generated by roxygen2: do not edit by hand +% Please edit documentation in R/plot-spaces.R +\name{plot_source_spaces} +\alias{plot_source_spaces} +\title{Compare target and source latent spaces} +\usage{ +plot_source_spaces( + Y_target, + Y_source, + r, + row_index = NULL, + column_index = NULL, + max_display = 200L, + limits = NULL, + row_label = "Row index", + column_label = "Column index", + target_name = "Target population" +) +} +\arguments{ +\item{Y_target}{Complete target matrix, or a completed estimate. Missing +values are not allowed.} + +\item{Y_source}{Source matrix, or a list of source matrices, with the same +rows and columns as the target.} + +\item{r}{Rank used for every matrix.} + +\item{row_index, column_index}{Optional indices of the rows and columns to +display. The singular vectors are always computed from the full matrices.} + +\item{max_display}{Maximum number of rows, and of columns, to display when +no indices are given (default 200). Evenly spaced indices are used.} + +\item{limits}{Optional pair of positive color limits (row side, then column +side). Each scale is symmetric around zero and shared across populations. +Values beyond a limit get the extreme color, and their number is returned as +\code{clipped_count}.} + +\item{row_label, column_label}{Axis labels. The defaults are generic, so use +labels that describe your rows and columns.} + +\item{target_name}{Label for the target panel.} +} +\value{ +Invisibly, a list with the displayed projection matrices +(\code{row_projectors} and \code{column_projectors}, named by population), the +displayed \code{row_index} and \code{column_index}, \code{limits}, and +\code{clipped_count}. +} +\description{ +Compare patterns in the target and source latent spaces. Similar patterns +suggest that the populations share similar spaces at the chosen rank. +} +\details{ +Each heatmap shows a projection matrix computed from the leading +singular vectors. Column-side patterns appear above row-side patterns. +Red indicates negative values and blue positive values; populations share +a color scale within each side so their patterns can be compared. + +Rows and columns must already be aligned across populations. Selecting +indices changes only what is displayed; spaces are estimated from the full +matrices. These plots show each population separately, without source weights. +Similar colors are a visual comparison, not a statistical test of shared spaces. +} +\examples{ +plot_source_spaces(dat_multisource$Y_target, dat_multisource$Y_source, + r = 3, max_display = 20) +} diff --git a/tests/testthat/helper-plotting.R b/tests/testthat/helper-plotting.R new file mode 100644 index 0000000..2e9ca4f --- /dev/null +++ b/tests/testthat/helper-plotting.R @@ -0,0 +1,15 @@ +# Devices are temporary and closed even when an assertion fails. +plot_device <- function() { + path <- tempfile(fileext = '.pdf') + grDevices::pdf(path, width = 10, height = 5) + path +} + +cv_plot_example <- function(separate = TRUE) { + if (separate) { + scores <- array(c(8, 7, 6, 5, 4, 3, 2, 1), c(2, 2, 2)) + list(mse_all = scores, lambda_1_min = NA_real_, lambda_1_row_min = 0, + lambda_1_col_min = 3, lambda_2_min = 1, r = 2) + } else list(mse_all = matrix(c(8, 7, 6, 5), 2, 2), + lambda_1_min = 0, lambda_2_min = 1, r = 2) +} diff --git a/tests/testthat/test-asymmetric-data.R b/tests/testthat/test-asymmetric-data.R new file mode 100644 index 0000000..ae0d48c --- /dev/null +++ b/tests/testthat/test-asymmetric-data.R @@ -0,0 +1,51 @@ +test_that('asymmetric example shares only the true U-space', { + d <- dat_asymmetric + expect_named(d, c('Y_source', 'Y_target', 'Theta_source', 'Theta_target')) + for (m in d) { + expect_identical(dim(m), c(100L, 50L)) + expect_true(all(is.finite(m))) + } + target <- svd(d$Theta_target) + source <- svd(d$Theta_source) + expect_equal(sum(target$d > 1e-8), 3) + expect_equal(sum(source$d > 1e-8), 3) + expect_equal(tcrossprod(target$u[, 1:3]), tcrossprod(source$u[, 1:3]), + tolerance = 1e-10) + expect_gt(norm(tcrossprod(target$v[, 1:3]) - tcrossprod(source$v[, 1:3]), 'F'), 0.1) + noise <- cbind(target = as.vector(d$Y_target - d$Theta_target), + source = as.vector(d$Y_source - d$Theta_source)) + ratios <- apply(noise, 2, var) / c(0.10, 0.01) + expect_true(all(ratios > 0.8 & ratios < 1.2)) + expect_lt(abs(cor(noise)[1, 2]), 0.05) +}) + +test_that('source reverses the pairing of target singular values at the low-similarity V scale', { + d <- dat_asymmetric + target <- svd(d$Theta_target, nu = 3, nv = 3) + source <- svd(d$Theta_source, nu = 3, nv = 3) + # Reversing the pairing leaves the set of singular values unchanged; a dropped + # QR step or a rescaled signal would not. + expect_equal(source$d[1:3], target$d[1:3], tolerance = 1e-8) + # The top three singular values of a 100 x 50 standard normal matrix lie in + # about 14.5 to 18 (2000 simulated draws). + expect_true(all(target$d[1:3] > 14 & target$d[1:3] < 19)) + # The strength the source carries on each shared left direction is the + # reversed target strength. + strength_on_u <- sqrt(rowSums(crossprod(target$u[, 1:3], d$Theta_source)^2)) + expect_equal(strength_on_u, rev(target$d[1:3]), tolerance = 1e-8) + # The V perturbation is at the low-similarity scale: 1/(4 sqrt(q)) gives about + # 0.34 and 1/sqrt(q) about 1.1, against about 0.65 for 1/(2 sqrt(q)). + d_v <- norm(tcrossprod(target$v[, 1:3]) - tcrossprod(source$v[, 1:3]), 'F') + expect_gt(d_v, 0.5) + expect_lt(d_v, 0.8) +}) + +test_that('noise is mean-zero with the documented scales', { + d <- dat_asymmetric + noise <- cbind(target = as.vector(d$Y_target - d$Theta_target), + source = as.vector(d$Y_source - d$Theta_source)) + se <- sqrt(c(0.10, 0.01) / nrow(noise)) + expect_true(all(abs(colMeans(noise)) < 5 * se)) + ratios <- apply(noise, 2, var) / c(0.10, 0.01) + expect_true(all(abs(ratios - 1) < 5 * sqrt(2 / nrow(noise)))) +}) diff --git a/tests/testthat/test-cv-heatmap-style.R b/tests/testthat/test-cv-heatmap-style.R new file mode 100644 index 0000000..1c9e0e8 --- /dev/null +++ b/tests/testthat/test-cv-heatmap-style.R @@ -0,0 +1,75 @@ +test_that('relative heatmaps preserve score alignment and distinguish high error from missing', { + testthat::skip_if_not_installed("ggplot2", minimum_version = "3.4.0") + path <- plot_device() + on.exit({grDevices::dev.off(); unlink(path)}, add = TRUE) + x <- list(mse_all = matrix(c(100, 110, 140, NA_real_), 2), lambda_1_min = 1, lambda_2_min = 0.1) + args <- list(x = x, lambda_1_all = c(1, 2), lambda_2_all = c(.1, 1)) + z <- do.call(plot_cv, args) + expect_s3_class(z$plot, 'ggplot') + expect_equal(z$display_data$score, c(100, 110, 140, NA_real_)) + expect_equal(z$display_data$label, c('0.0%', '10.0%', '>30%', 'NA')) + expect_equal(z$display_data$selected, c(TRUE, FALSE, FALSE, FALSE)) + expect_equal(z$above_limit_count, 1) + expect_equal(z$unavailable_count, 1) + expect_equal(z$display_limits, c(0, .1)) + expect_match(z$plot$labels$caption, 'not a divergence diagnosis', fixed = TRUE) + mse <- do.call(plot_cv, c(args, list(n_observed = 10))) + expect_equal(mse$display_data$display, z$display_data$display) + expect_equal(mse$minimum, 10) + full <- do.call(plot_cv, c(args, list(relative_limit = NULL, show_values = FALSE))) + expect_equal(full$above_limit_count, 0) + expect_equal(full$display_limits, c(0, .4)) + expect_equal(length(full$plot$layers), 2) + expect_error(do.call(plot_cv, c(args, list(relative_limit = 0))), 'relative_limit') + expect_error(do.call(plot_cv, c(args, list(show_values = NA))), 'show_values') +}) + +test_that('zero minima use absolute scores and exact ties remain first in CV order', { + testthat::skip_if_not_installed("ggplot2", minimum_version = "3.4.0") + path <- plot_device() + on.exit({grDevices::dev.off(); unlink(path)}, add = TRUE) + x <- list(mse_all = matrix(c(0, 0, 2, 4), 2), lambda_1_min = 1, lambda_2_min = .1) + z <- plot_cv(x, lambda_1_all = c(1, 2), lambda_2_all = c(.1, 1)) + expect_false(z$relative) + expect_equal(z$above_limit_count, 0) + expect_equal(z$display_data$display, c(0, 0, 2, 4)) + expect_equal(z$exact_minima, 2) + expect_equal(which(z$display_data$selected), 1L) + expect_match(z$plot$labels$caption, 'absolute SSE', fixed = TRUE) + x$mse_all[] <- 0 + expect_gt(diff(plot_cv(x, lambda_1_all = c(1, 2), lambda_2_all = c(.1, 1))$display_limits), 0) +}) + +test_that('separate heatmaps share scales and return printable pages', { + testthat::skip_if_not_installed("ggplot2", minimum_version = "3.4.0") + path <- plot_device() + on.exit({grDevices::dev.off(); unlink(path)}, add = TRUE) + x <- list(mse_all = array(7:1, c(1, 1, 7)), lambda_1_row_min = 1, + lambda_1_col_min = 2, lambda_2_min = 7) + args <- list(x = x, lambda_1_row_all = 1, lambda_1_col_all = 2, lambda_2_all = 1:7, + relative_limit = NULL) + full <- do.call(plot_cv, args) + subset <- do.call(plot_cv, c(args, list(lambda_2_index = c(7, 2)))) + expect_length(full$plots, 2) + expect_equal(full$display_limits, subset$display_limits) + expect_equal(as.character(subset$plot$data$panel), c('2', '7')) + expect_equal(levels(subset$plot$data$panel), c('7', '2')) + expect_equal(sum(subset$plot$data$selected), 1) +}) + + +test_that('close or repeated balance candidates remain distinct heatmap panels', { + testthat::skip_if_not_installed("ggplot2", minimum_version = "3.4.0") + path <- plot_device() + on.exit({grDevices::dev.off(); unlink(path)}, add = TRUE) + for (balance in list(c(1, 1.00001), c(1, 1))) { + x <- list(mse_all = array(1:8, c(2, 2, 2)), lambda_1_row_min = 1, + lambda_1_col_min = 1, lambda_2_min = 1) + result <- plot_cv(x, lambda_1_row_all = c(1, 2), + lambda_1_col_all = c(1, 2), lambda_2_all = balance) + built <- ggplot2::ggplot_build(result$plot) + expect_equal(nrow(built$layout$layout), 2L) + expect_equal(as.integer(table(built$data[[1]]$PANEL)), c(4L, 4L)) + expect_equal(as.integer(built$data[[3]]$PANEL), 1L) + } +}) diff --git a/tests/testthat/test-cv-plot-errors.R b/tests/testthat/test-cv-plot-errors.R new file mode 100644 index 0000000..ec0239f --- /dev/null +++ b/tests/testthat/test-cv-plot-errors.R @@ -0,0 +1,48 @@ +test_that('CV heatmaps reject invalid titles and separate arguments in common mode', { + args <- list(x = cv_plot_example(FALSE), lambda_1_all = c(10, 0), + lambda_2_all = c(0, 1)) + for (title in list(NA_character_, character(), c('a', 'b'), 1)) { + expect_error(do.call(plot_cv, c(args, list(main = title))), + 'main must be one nonmissing character string', fixed = TRUE) + } + for (extra in list(list(lambda_1_row_all = c(1, 2)), + list(lambda_1_col_all = c(1, 2)), + list(lambda_2_index = 1))) { + expect_error(do.call(plot_cv, c(args, extra)), + 'Separate-penalty arguments require a 3D score array', fixed = TRUE) + } +}) + +test_that('CV heatmaps explain how to install the optional plotting dependency', { + # Older supported testthat versions do not provide local_mocked_bindings(). + skip_if_not_installed('testthat', minimum_version = '3.2.0') + original_require <- base::requireNamespace + testthat::local_mocked_bindings( + requireNamespace = function(package, ...) { + if (identical(package, 'ggplot2')) return(FALSE) + original_require(package, ...) + }, .package = 'base') + expect_error(plot_cv(cv_plot_example(FALSE), lambda_1_all = c(10, 0), + lambda_2_all = c(0, 1)), + 'Install it with install.packages("ggplot2")', fixed = TRUE) +}) + +test_that('CV heatmaps explain when the plotting dependency needs updating', { + skip_if_not_installed('testthat', minimum_version = '3.2.0') + # Simulate availability/version checks without changing installed packages. + original_require <- base::requireNamespace + original_version <- utils::packageVersion + testthat::local_mocked_bindings( + requireNamespace = function(package, ...) { + if (identical(package, 'ggplot2')) return(TRUE) + original_require(package, ...) + }, .package = 'base') + testthat::local_mocked_bindings( + packageVersion = function(pkg, ...) { + if (identical(pkg, 'ggplot2')) return(package_version('3.3.6')) + original_version(pkg, ...) + }, .package = 'utils') + expect_error(plot_cv(cv_plot_example(FALSE), lambda_1_all = c(10, 0), + lambda_2_all = c(0, 1)), + 'requires ggplot2 >= 3.4.0. Update it', fixed = TRUE) +}) diff --git a/tests/testthat/test-paper-plots.R b/tests/testthat/test-paper-plots.R new file mode 100644 index 0000000..c4c6a53 --- /dev/null +++ b/tests/testthat/test-paper-plots.R @@ -0,0 +1,56 @@ +test_that('CV annotations distinguish small differences from exact ties and numeric boundaries', { + testthat::skip_if_not_installed("ggplot2", minimum_version = "3.4.0") + path<-plot_device();on.exit({grDevices::dev.off();unlink(path)},add=TRUE) + x<-list(mse_all=matrix(c(1.06,1.051,1.07,1.06,1.052,1.07),3), + lambda_1_min=50,lambda_2_min=0.01) + d<-plot_cv(x,lambda_1_all=c(10,50,100),lambda_2_all=c(0.01,1)) + expect_equal(d$exact_minima,1) + expect_equal(d$score_range,c(1.051,1.07)) + expect_equal(d$lambda_2_slice_range,c(1.051,1.052)) + expect_identical(d$boundary,c(common=FALSE,balance=TRUE)) + x$mse_all[2,2]<-1.051 + expect_equal(plot_cv(x,lambda_1_all=c(10,50,100),lambda_2_all=c(0.01,1))$exact_minima,2) + x$mse_all<-matrix(c(3,1,2),3);x$lambda_1_min<-1;x$lambda_2_min<-1 + expect_identical(plot_cv(x,lambda_1_all=c(3,1,2),lambda_2_all=1)$boundary, + c(common=TRUE,balance=FALSE)) +}) + +test_that('space plots compute full-matrix spaces before taking common submatrices', { + path<-plot_device();on.exit({grDevices::dev.off();unlink(path)},add=TRUE) + # This rank-one matrix has analytically known normalized left/right vectors. + u<-c(1,2,3);v<-c(2,1);y<-outer(u,v) + set.seed(1);before<-.Random.seed + old<-graphics::par(c('mfrow','mar')) + d<-plot_source_spaces(y,list(A=y,B=2*y),r=1,row_index=c(3,1),column_index=2:1) + expect_equal(d$row_projectors[[1]],outer(u,u)[c(3,1),c(3,1)]/sum(u^2)) + expect_equal(d$column_projectors[[1]],outer(v,v)[2:1,2:1]/sum(v^2)) + expect_equal(d$row_projectors[[2]],d$row_projectors[[3]]) + expect_identical(.Random.seed,before) + expect_equal(graphics::par(c('mfrow','mar')),old) + expect_equal(d$limits,c(row=9/14,column=4/5)) + clipped<-plot_source_spaces(y,y,r=1,limits=c(.1,.1)) + expect_gt(sum(unlist(clipped$clipped_count)),0) + expect_equal(clipped$row_projectors[[1]],outer(u,u)/sum(u^2)) + expect_error(plot_source_spaces(y,y,r=2),'zero singular') + expect_error(plot_source_spaces(y,y,r=1,row_index=c(1,1)),'distinct') + expect_error(plot_source_spaces(y,y,r=1,limits=c(0,.1)),'positive') + y[1]<-NA + expect_error(plot_source_spaces(y,y,r=1),'complete finite') +}) + +test_that('contributions use squared normalized singular vectors, not raw factors', { + path<-plot_device();on.exit({grDevices::dev.off();unlink(path)},add=TRUE) + u<-c(1,2,3);v<-c(2,1);y<-outer(u,v) + d<-plot_contributions(y,r=1,row_index=c(3,1),column_index=2) + expect_equal(drop(d$row_scores),u^2/sum(u^2)) + expect_equal(drop(d$column_scores),v^2/sum(v^2)) + expect_equal(colSums(d$row_scores),1) + expect_equal(colSums(d$column_scores),1) + expect_equal(d$row_index,c(3L,1L)) + expect_equal(plot_contributions(-y,r=1)$row_scores,d$row_scores) + expect_equal(plot_contributions(list(learner_estimate=y,r=1))$column_scores,d$column_scores) + expect_equal(plot_contributions(list(dlearner_estimate=y,r=1))$row_scores,d$row_scores) + expect_error(plot_contributions(y),'Supply r') + expect_error(plot_contributions(y,r=2),'zero singular') + expect_error(plot_contributions(y,r=1,column_index=3),'valid integer') +}) diff --git a/tests/testthat/test-plotting.R b/tests/testthat/test-plotting.R new file mode 100644 index 0000000..191f27b --- /dev/null +++ b/tests/testthat/test-plotting.R @@ -0,0 +1,228 @@ +test_that('objective plots show the requested iterations', { + path <- plot_device() + on.exit({grDevices::dev.off(); unlink(path)}, add = TRUE) + # Decrease to the minimum at iteration 31, then values that stay above it. + fit <- list(objective_values = c(seq(100, 10, length.out = 30), rep(c(9, 9.5), 50)), + convergence_criterion = 2L) + original <- fit + result <- withVisible(plot_objective(fit)) + expect_false(result$visible) + expect_equal(result$value$iteration, 1:130) + expect_equal(result$value$objective, fit$objective_values) + expect_equal(plot_objective(fit, iterations = 2:4)$iteration, 2:4) + expect_equal(plot_objective(fit, iterations = c(5, 2, 2))$iteration, c(2, 5)) + expect_equal(plot_objective(fit, iterations = 1:1000)$iteration, 1:130) + expect_equal(plot_objective(fit, iterations = c(1000, 2, 1))$iteration, 1:2) + # Twenty records on each side of the first minimum (iteration 31). + expect_equal(plot_objective(fit, iterations = 'near_optimum')$iteration, 11:51) + expect_identical(fit, original) +}) + +test_that('near-optimum windows center on the minimum and zoom both axes', { + path <- plot_device() + on.exit({grDevices::dev.off(); unlink(path)}, add = TRUE) + # Steep decrease to an interior minimum, then a slow rise. + fit <- list(objective_values = 10 + 100 * exp(-0.05 * (1:2000)) + 0.0005 * (1:2000)) + minimum <- which.min(fit$objective_values) + plot_objective(fit) + full_x <- graphics::par('usr')[1:2] + full_y <- graphics::par('usr')[3:4] + result <- plot_objective(fit, iterations = 'near_optimum') + expect_equal(result$iteration, (minimum - 20L):(minimum + 20L)) + expect_lt(diff(graphics::par('usr')[1:2]), 0.05 * diff(full_x)) + expect_lt(diff(graphics::par('usr')[3:4]), 0.01 * diff(full_y)) + # Appending worse records does not change the neighborhood of the minimum. + extended <- list(objective_values = c(fit$objective_values, rep(200, 3000))) + expect_equal(plot_objective(extended, iterations = 'near_optimum')$iteration, + result$iteration) + # Reproduce the early minimum in a 2000-record history. + early <- list(objective_values = (seq_len(2000) - 99)^2) + expect_equal(plot_objective(early, iterations = 'near_optimum')$iteration, 79:119) + # At the start, shift the window right; ties use the first minimum. + fit$objective_values <- seq_len(100) + expect_equal(plot_objective(fit, iterations = 'near_optimum')$iteration, 1:41) + fit$objective_values <- rep(1, 100) + expect_equal(plot_objective(fit, iterations = 'near_optimum')$iteration, 1:41) +}) + +test_that('near-optimum plots show the final stretch when the minimum is at the end', { + path <- plot_device() + on.exit({grDevices::dev.off(); unlink(path)}, add = TRUE) + # A history that is still decreasing at the last iteration. + fit <- list(objective_values = seq(100, 10, length.out = 100), convergence_criterion = 1L) + plot_objective(fit) + full_x <- graphics::par('usr')[1:2] + expect_equal(plot_objective(fit, iterations = 'near_optimum')$iteration, 60:100) + # The final stretch has its own horizontal axis. + expect_gt(graphics::par('usr')[1], full_x[1] + 50) + # Short histories are shown in full. + fit <- list(objective_values = c(9, 5, 4), convergence_criterion = 1L) + expect_equal(plot_objective(fit, iterations = 'near_optimum')$iteration, 1:3) +}) + +test_that('near-optimum plots handle alternating and missing values and keep graphics settings', { + path <- plot_device() + on.exit({grDevices::dev.off(); unlink(path)}, add = TRUE) + # Alternating values after a decrease; near_optimum shows the raw values. + values <- c(seq(50, 11, length.out = 40), 10 + rep(c(0, 0.5), 200)) + values[50] <- NA + fit <- list(objective_values = values, convergence_criterion = 2L) + old <- graphics::par('mar') + result <- plot_objective(fit) + expect_equal(result$iteration, seq_along(values)) + expect_equal(plot_objective(fit, iterations = 'near_optimum')$iteration, 21:61) + expect_equal(graphics::par('mar'), old) +}) + +test_that('objective plots handle flat, singleton, and non-finite histories', { + path <- plot_device() + on.exit({grDevices::dev.off(); unlink(path)}, add = TRUE) + for (values in list(4, c(4, 4, 4), c(4, Inf, NaN), c(Inf, 3, 1, NA))) { + fit <- list(objective_values = values, convergence_criterion = 2L) + expect_equal(plot_objective(fit)$objective, values) + expect_true(length(plot_objective(fit, iterations = 'near_optimum')$iteration) >= 1L) + } + fit <- list(objective_values = c(2, 1, 3), convergence_criterion = 2L) + for (bad in list(0, 4, 1.5, NA, numeric(), 'near', TRUE)) { + expect_error(plot_objective(fit, iterations = bad), 'iterations') + } + expect_error(plot_objective(list(objective_values = c(Inf, NA))), 'learner') + expect_error(plot_objective(list()), 'learner') +}) + +test_that('near-optimum plots handle short, alternating, and incomplete histories', { + path <- plot_device() + on.exit({grDevices::dev.off(); unlink(path)}, add = TRUE) + for (history in list(rep(c(10, 12), 5), c(1, 3), c(1, 3, NA, 2, 4), c(1, NA, 3))) { + expect_equal(plot_objective(list(objective_values = history), + iterations = 'near_optimum')$objective, history) + } +}) + +test_that('CV plots retain candidate order, mark the global minimum, and scale SSE', { + testthat::skip_if_not_installed("ggplot2", minimum_version = "3.4.0") + path <- plot_device() + on.exit({grDevices::dev.off(); unlink(path)}, add = TRUE) + x <- cv_plot_example() + original <- x + old <- graphics::par(c('mfrow','mar')) + args <- list(x = x, lambda_1_row_all = c(10, 0), + lambda_1_col_all = c(0.2, 3), lambda_2_all = c(0, 1)) + sse <- do.call(plot_cv, args) + mse <- do.call(plot_cv, c(args, list(n_observed = 40))) + expect_identical(sse$grids$row, c(10, 0)) + expect_identical(sse$best_index, c(2L, 2L, 2L)) + expect_equal(sse$scores, x$mse_all) + expect_equal(mse$scores, x$mse_all / 40) + expect_equal(sse$scale, 'Validation SSE') + expect_equal(mse$scale, 'Validation MSE') + expect_equal(mse$color_limits, range(x$mse_all) / 40) + expect_identical(x, original) + expect_equal(graphics::par(c('mfrow','mar')), old) + subset <- do.call(plot_cv, c(args, list(lambda_2_index = 1))) + expect_equal(subset$color_limits, sse$color_limits) + expect_equal(subset$shown_lambda_2_index, 1) + expect_equal(subset$best_index, sse$best_index) + common <- plot_cv(cv_plot_example(FALSE), lambda_1_all = c(10, 0), lambda_2_all = c(0, 1)) + expect_identical(common$best_index, c(2L, 2L)) +}) + +test_that('CV plots handle singleton axes, duplicate candidates, ties and unavailable scores', { + testthat::skip_if_not_installed("ggplot2", minimum_version = "3.4.0") + path <- plot_device() + on.exit({grDevices::dev.off(); unlink(path)}, add = TRUE) + x <- list(mse_all = array(c(5, 5, Inf, NA), c(1, 2, 2)), + lambda_1_row_min = 0, lambda_1_col_min = 2, lambda_2_min = 1) + z <- plot_cv(x, lambda_1_row_all = 0, lambda_1_col_all = c(2, 2), + lambda_2_all = c(1, 10)) + expect_identical(z$best_index, c(1L, 1L, 1L)) + expect_equal(z$unavailable_count, 2) + expect_gt(diff(z$color_limits), 0) + x <- list(mse_all = matrix(0, 1, 1), lambda_1_min = 0, lambda_2_min = 0) + z <- plot_cv(x, lambda_1_all = 0, lambda_2_all = 0) + expect_identical(z$best_index, c(1L, 1L)) + expect_gt(diff(z$color_limits), 0) + # Seven balance slices exercise pagination without inventing new candidates. + x <- list(mse_all = array(7:1, c(1, 1, 7)), lambda_1_row_min = 0, + lambda_1_col_min = 2, lambda_2_min = 7) + z <- plot_cv(x, lambda_1_row_all = 0, lambda_1_col_all = 2, lambda_2_all = 1:7) + expect_equal(z$shown_lambda_2_index, 1:7) + expect_identical(z$best_index, c(1L, 1L, 7L)) +}) + +test_that('CV plots reject missing, mismatched, and ambiguous grid information', { + testthat::skip_if_not_installed("ggplot2", minimum_version = "3.4.0") + x <- cv_plot_example() + args <- list(x = x, lambda_1_row_all = c(10, 0), + lambda_1_col_all = c(0.2, 3), lambda_2_all = c(0, 1)) + expect_error(plot_cv(x), 'lambda_1_row_all') + bad <- args; bad$lambda_1_row_all <- c(0,10) + expect_error(do.call(plot_cv,bad), 'selected penalties') + bad <- args; bad$lambda_1_row_all <- 0 + expect_error(do.call(plot_cv,bad), 'score-array dimension') + expect_error(do.call(plot_cv,c(args,list(lambda_1_all=1))), 'Do not supply') + expect_error(do.call(plot_cv,c(args,list(n_observed=0))), 'n_observed') + expect_error(do.call(plot_cv,c(args,list(lambda_2_index=c(1,1)))), 'distinct') + expect_error(do.call(plot_cv,c(args,list(lambda_2_index=3))), 'distinct') + expect_error(do.call(plot_cv,c(args,list(lambda_2_index=NA_real_))), 'distinct') + x$mse_all[] <- Inf + expect_error(plot_cv(x), 'No finite') + x$mse_all[] <- -1 + expect_error(plot_cv(x), 'cannot be negative') + expect_error(plot_cv(list(mse_all=1:3)), 'two- or three-dimensional') +}) + +test_that('plot functions accept actual fits and both CV modes without changing outputs', { + testthat::skip_if_not_installed("ggplot2", minimum_version = "3.4.0") + path <- plot_device() + on.exit({grDevices::dev.off(); unlink(path)}, add = TRUE) + set.seed(623) + s <- matrix(rnorm(48), 8, 6) + t <- matrix(rnorm(48), 8, 6); t[c(1,3)] <- NA_real_ + fit <- learner(s,t,r=2,lambda_1=1,lambda_2=1,step_size=0.001, + control=list(max_iter=6)) + expect_equal(plot_objective(fit)$objective,fit$objective_values) + for (separate in c(FALSE,TRUE)) { + args <- list(Y_source=list(s,s*0.9),Y_target=t,r=2,step_size=0.001, + lambda_2_all=c(0,1),n_folds=2,control=list(max_iter=6)) + grid <- if(separate) list(lambda_1_row_all=c(0,1),lambda_1_col_all=c(0.5,2)) else list(lambda_1_all=c(0,1)) + cv <- do.call(cv.learner,c(args,grid)) + original <- cv + result <- do.call(plot_cv,c(list(x=cv,lambda_2_all=c(0,1),n_observed=sum(!is.na(t))),grid)) + expect_equal(result$scores,cv$mse_all/46) + expect_identical(cv,original) + } +}) + + +test_that('paper-style log labels preserve scores and candidate selection', { + testthat::skip_if_not_installed("ggplot2", minimum_version = "3.4.0") + path <- plot_device() + on.exit({grDevices::dev.off(); unlink(path)}, add = TRUE) + x <- list(mse_all = matrix(c(4, 3, 2, 1), 2, 2), + lambda_1_min = 100, lambda_2_min = 1) + raw <- plot_cv(x, lambda_1_all = c(10, 100), lambda_2_all = c(0.01, 1)) + log <- plot_cv(x, lambda_1_all = c(10, 100), lambda_2_all = c(0.01, 1), log_labels = TRUE) + expect_equal(log$scores, raw$scores) + expect_equal(log$best_index, raw$best_index) + expect_equal(log$grids, raw$grids) + expect_true(log$log_labels) + expect_error(plot_cv(x, lambda_1_all = c(0, 100), lambda_2_all = c(0.01, 1), + log_labels = TRUE), 'strictly positive') + expect_error(plot_cv(x, log_labels = NA), 'TRUE or FALSE') + expect_error(plot_cv(x, log_labels = 'yes'), 'TRUE or FALSE') +}) + +test_that('objective plots reject ranges with no finite values before drawing', { + path <- plot_device() + on.exit({grDevices::dev.off(); unlink(path)}, add = TRUE) + fit <- list(objective_values = c(3, 1, NA_real_, Inf, NaN)) + old <- graphics::par(c('mar', 'usr')) + expect_error( + withCallingHandlers(plot_objective(fit, iterations = 3:5), + warning = function(w) stop('Unexpected warning: ', conditionMessage(w))), + 'selected iterations contain no finite objective values') + expect_equal(graphics::par(c('mar', 'usr')), old) + expect_equal(plot_objective(fit, iterations = 2:5)$objective, + fit$objective_values[2:5]) +}) diff --git a/vignettes/advanced-learner.Rmd b/vignettes/advanced-learner.Rmd new file mode 100644 index 0000000..ee875d0 --- /dev/null +++ b/vignettes/advanced-learner.Rmd @@ -0,0 +1,516 @@ +--- +title: "LEARNER and D-LEARNER: Multiple Sources, Source Weights, and Separate Penalties" +output: + rmarkdown::html_vignette: + toc: true + toc_depth: 2 +vignette: > + %\VignetteIndexEntry{LEARNER and D-LEARNER: Multiple Sources, Source Weights, and Separate Penalties} + %\VignetteEngine{knitr::rmarkdown} + %\VignetteEncoding{UTF-8} +--- + +```{r setup, include=FALSE} +knitr::opts_chunk$set(collapse = TRUE, comment = "#>", fig.width = 7, fig.height = 5) +``` + +The README introduces estimation with one source population. This tutorial +explains two extensions, one at a time: using multiple sources, and allowing +different penalties for the two latent spaces. We first introduce source +weights shared by D-LEARNER and LEARNER, then explain how each method uses them. +Finally, we return to one source to study separate penalties in LEARNER. + +Weights determine how source-space information is combined. Penalties control +LEARNER's fit to the target and its departures from those spaces. D-LEARNER +uses source weights but has no penalty parameters to tune. + +```{r load-package, message=FALSE} +library(learner) +``` + +## 1. Starting Point: Two Methods and a Single Source + +Write the target estimate as $UV^\top$, with $U$ of size $p\times r$ and $V$ +of size $q\times r$. Let $P_U$ and $P_V$ project onto the leading source +singular-vector spaces. The columns of $U$ span the signal matrix's column +space, and the columns of $V$ span its row space. We use **U-space** and +**V-space** throughout. In the package interface, `lambda_1_row` acts on $U$ +and `lambda_1_col` acts on $V$. + +D-LEARNER estimates the target by direct projection, $P_UY_0P_V$, without +iterative optimization or penalty tuning. LEARNER instead fits target factors +while penalizing departures from the source spaces. With complete target data +$Y_0$, its single-source, common-penalty objective is + +$$ +L(U,V)=\|UV^\top-Y_0\|_F^2 ++\lambda_1\|(I_p-P_U)U\|_F^2 ++\lambda_1\|(I_q-P_V)V\|_F^2 ++\lambda_2\|U^\top U-V^\top V\|_F^2. +$$ + +The first term fits the target data. The next two encourage the target factors +to follow the source spaces, with common strength `lambda_1`. The last term, +controlled by `lambda_2`, balances the scales of the two factors. + +For missing target entries, the fit term uses only observed entries and is +divided by the observed fraction. Penalty terms stay the same. All examples +below have complete observed matrices and fix the rank at 3 to focus on the +extensions. See `?learner` for the optimization settings and stopping codes. + +## 2. Multiple Sources and Source Weights + +### Aligned populations and their precision + +Multiple related populations can provide different information about the target. +Their usefulness depends on how well their spaces match the target and how +precisely those spaces can be estimated. + +`dat_multisource` contains one target and two aligned sources, each with 100 +rows and 50 columns. Their rank-3 true signals share latent spaces. Independent +normal noise has variance 0.10 for the target, 0.01 for A, and 0.50 for B. +Thus A is observed more precisely than B. + +```{r multiple-data} +sources <- dat_multisource$Y_source +target <- dat_multisource$Y_target +weights <- c(0.7, 0.3) +``` + +These weights are illustrative, not estimated. They correspond to sources in +list order, are normalized to sum to one, and remain fixed during CV. Omitting +weights gives equal weights. Sources must already have matching rows and +columns in the same order. + +### Combining source-space information + +For source $k$, let $\hat U_k$ and $\hat V_k$ contain its leading orthonormal +singular vectors at rank 3. The weights combine their projection matrices: + +$$ +\bar P_U=\sum_k w_k\hat U_k\hat U_k^\top,\qquad +\bar P_V=\sum_k w_k\hat V_k\hat V_k^\top, +\quad w_k\geq 0,\quad \sum_k w_k=1. +$$ + +A larger weight gives a source more influence in these combinations; it does +not guarantee a more accurate estimate. The same weights are used on both sides. +The combined matrices need not themselves be orthogonal projectors. + +These weights do not directly average the observed source matrices or the +separately fitted estimates. Both methods below use the same combined spaces, +but turn that information into a target estimate differently. Weight selection +by cross-validation is outside this tutorial: all CV calls hold the weights fixed. + +## 3. D-LEARNER with Multiple Sources + +### Direct estimation without penalties + +D-LEARNER uses the same source list and weights without penalty tuning: + +$$\widehat\Theta_{\rm D}=\bar P_U Y_0\bar P_V.$$ + +In general this is not the weighted average of the source-specific D-LEARNER +estimates: multiplying the two weighted matrices also produces cross-source +terms. The formula above is the operation implemented by `dlearner()`. + +### Compare source choices + +We compare sources using D-LEARNER at the same rank, so penalty tuning and +iterative optimization do not introduce additional differences. + +```{r multiple-direct} +direct <- dlearner(sources, target, r = 3, source_weights = weights) +estimates <- list( + Target_only = dlearner(target, target, r = 3)$dlearner_estimate, + A_only = dlearner(sources$A, target, r = 3)$dlearner_estimate, + B_only = dlearner(sources$B, target, r = 3)$dlearner_estimate, + Weighted_A_B = direct$dlearner_estimate) +source_errors <- vapply(estimates, function(estimate) { + norm(estimate - dat_multisource$Theta_target, "F") +}, numeric(1)) +round(source_errors, 3) +``` + +Using the target as its own source gives its rank-3 truncated SVD. True signals +are known only because these data are simulated; they are not used for fitting +or CV. Smaller Frobenius error means a closer estimate of the true matrix. +In this example A alone is best; the weighted fit improves on B alone but does +not improve on A. These are single-dataset errors, not CV scores or a guarantee +that averaging sources helps. + +## 4. LEARNER with Multiple Sources + +### From weights to penalties + +We now use the same data and fixed weights with LEARNER. Unlike D-LEARNER, +LEARNER can move away from the combined source spaces to fit the target data. +The penalties control that trade-off. We keep a common space penalty in this +section so that multiple sources are the only extension to the basic method. + +The objective becomes + +$$ +L_{\rm multi}(U,V)=\|UV^\top-Y_0\|_F^2 ++\lambda_1\|(I_p-\bar P_U)U\|_F^2 ++\lambda_1\|(I_q-\bar P_V)V\|_F^2 ++\lambda_2\|U^\top U-V^\top V\|_F^2. +$$ + +Weights determine the combined source-space matrices; `lambda_1` controls the +penalty on departures from them. `lambda_2` balances the factor scales. A +source weight is therefore not interchangeable with a penalty value. LEARNER uses +the first source for initialization and, when rank is omitted, rank selection. + +### Select penalties and fit LEARNER + +Keep the two space penalties equal in this first example. + +```{r multiple-cv} +space_grid <- c(0, 1, 10) +balance_grid <- c(0.1, 1) +set.seed(815) +cv_multi <- cv.learner( + sources, target, r = 3, source_weights = weights, + lambda_1_all = space_grid, lambda_2_all = balance_grid, + n_folds = 3, step_size = 0.0003, control = list(max_iter = 2000)) +c(space = cv_multi$lambda_1_min, balance = cv_multi$lambda_2_min) +``` + +The heatmap compares the candidate penalties. Pale cells have lower error; +the black box marks the selected combination. `n_observed` converts held-out +sums of squared errors to mean squared errors. `plot_cv()` needs the ggplot2 +package (version 3.4.0 or later); if it is not installed, the two heatmap chunks +in this tutorial are skipped. + +```{r multiple-cv-plot, eval = requireNamespace("ggplot2", quietly = TRUE) && utils::packageVersion("ggplot2") >= "3.4.0"} +plot_cv(cv_multi, lambda_1_all = space_grid, lambda_2_all = balance_grid, + n_observed = sum(!is.na(target))) +``` + +Refit using all observed target entries: + +```{r multiple-refit} +fit_multi <- learner( + sources, target, r = 3, source_weights = weights, + lambda_1 = cv_multi$lambda_1_min, lambda_2 = cv_multi$lambda_2_min, + step_size = 0.0003, control = list(max_iter = 2000)) +fit_multi$convergence_criterion +``` + +The result reports stopping code `r fit_multi$convergence_criterion` after +`r length(fit_multi$objective_values)` updates. Code 1 means that the +objective-change threshold was met; code 2 means the iteration limit was +reached; code 3 indicates excessive growth or a non-finite update. + +In this run, CV selected space penalty `r cv_multi$lambda_1_min` and balance +penalty `r cv_multi$lambda_2_min`, both at the upper grid boundary. These are +choices within the supplied grid, not established optimal penalties. Section 6 +explains how to investigate boundaries and stopping behavior. + +## 5. LEARNER with Separate U- and V-Space Penalties + +We now return to a single source. Only the penalty structure changes in this +example; source weighting is not involved. D-LEARNER has no corresponding +penalty parameters. + +### An asymmetric example + +`dat_asymmetric` has one source and one target. Their true U-space is shared, +whereas their true V-spaces differ. It adapts the U-shared/V-perturbed +construction in the paper's simulation code, using 100 rows, 50 columns, +rank 3, and independent noise with variances 0.10 (target) and 0.01 (source). +See `?dat_asymmetric` for generation details. + +The simulation is adapted from Section 2.4.1 of the Supporting Information of +McGrath, Zhu, O'Dea, Guo and Duan (2026, *Statistics in Medicine* 45(15-17), +e70669), and separate row and column penalties are described in its Section 1.1. +That section calls this setting "same column spaces but different row spaces": +the span of U is the column space of the signal matrix, and the span of V is its +row space. The settings below (step size, penalty grids, iteration budget) were chosen for +this small example rather than taken from the paper, which used 500 by 500 +matrices of rank 4, a step size of 0.07 and 75 iterations. + +We might therefore want to borrow more strongly on the U side than on the V +side. This is a reason to try separate penalties, rather than imposing that +preference as the selected result. + +### What changes in the objective? + +With a single source, replace the common penalty by two values: + +$$ +L_{\rm separate}(U,V)=\|UV^\top-Y_0\|_F^2 ++\lambda_{1,\rm row}\|(I_p-P_U)U\|_F^2 ++\lambda_{1,\rm col}\|(I_q-P_V)V\|_F^2 ++\lambda_2\|U^\top U-V^\top V\|_F^2. +$$ + +Equal row and column penalties recover the common-penalty objective. Select +both penalties from the data rather than using the known simulation truth. + +### Compare common and separate penalties + +Use the same rank, folds, step size, and iteration budget. The common grid is +the diagonal of the separate grid, so the separate search includes every common +candidate as well as unequal pairs. Resetting the seed makes the folds match. + +```{r asymmetric-comparison} +# One source: the true U-space is shared, whereas the true V-spaces differ. +dat <- learner::dat_asymmetric +space_grid <- c(0, 10, 100) +balance_grid <- c(0.001, 0.01, 0.1, 1) +settings <- list(max_iter = 2000) + +# Reset the seed so both searches use the same validation folds. The common +# grid is the diagonal of the separate grid; rank and optimizer settings match. +set.seed(815) +cv_common <- learner::cv.learner( + dat$Y_source, dat$Y_target, r = 3, + lambda_1_all = space_grid, lambda_2_all = balance_grid, + n_folds = 3, step_size = 0.003, control = settings) +set.seed(815) +cv_separate <- learner::cv.learner( + dat$Y_source, dat$Y_target, r = 3, + lambda_1_row_all = space_grid, lambda_1_col_all = space_grid, + lambda_2_all = balance_grid, n_folds = 3, + step_size = 0.003, control = settings) + +fit_common <- learner::learner( + dat$Y_source, dat$Y_target, r = 3, + lambda_1 = cv_common$lambda_1_min, lambda_2 = cv_common$lambda_2_min, + step_size = 0.003, control = settings) +fit_separate <- learner::learner( + dat$Y_source, dat$Y_target, r = 3, + lambda_1_row = cv_separate$lambda_1_row_min, + lambda_1_col = cv_separate$lambda_1_col_min, + lambda_2 = cv_separate$lambda_2_min, step_size = 0.003, control = settings) + +comparison <- data.frame( + method = c('Common penalties', 'Separate penalties'), + lambda_row = c(cv_common$lambda_1_min, cv_separate$lambda_1_row_min), + lambda_col = c(cv_common$lambda_1_min, cv_separate$lambda_1_col_min), + lambda_balance = c(cv_common$lambda_2_min, cv_separate$lambda_2_min), + cv_mse = c(min(cv_common$mse_all), min(cv_separate$mse_all)) / length(dat$Y_target), + frobenius_error = c(norm(fit_common$learner_estimate - dat$Theta_target, 'F'), + norm(fit_separate$learner_estimate - dat$Theta_target, 'F')), + stop_code = c(fit_common$convergence_criterion, fit_separate$convergence_criterion)) +print(comparison, row.names = FALSE) +# This is one simulation, not a general performance claim. The minimum CV score +# was used for selection, so it is not an independent test error. Check stopping +# codes and grid boundaries before interpreting a selected fit. +``` + +A lower minimum CV score for the larger search is not independent evidence of +better prediction: that score was used to select parameters. Frobenius errors +compare refitted estimates with simulation truth. Interpret both alongside +stopping codes, and use repeated simulations or an independent test set for a +performance study. This single example does not guarantee unequal selected +penalties or improved estimation. + +In this run, the separate search selected a U-space penalty (`lambda_1_row`) of +`r cv_separate$lambda_1_row_min` and a V-space penalty (`lambda_1_col`) of +`r cv_separate$lambda_1_col_min`, placing a stronger penalty on +departures from the shared U-space, as the construction suggests. A penalty of +100 versus 10 does not mean that ten times as much information is transferred: +the penalty terms also depend on the factors and their departures from the +source spaces. The common search selected +`r cv_common$lambda_1_min` for both. The selected U penalty is the largest value +in the grid (`r max(space_grid)`), so a larger grid might change the choice; +the grid is kept small so that the example runs quickly. + +In this asymmetric-space example, separate penalties allow different borrowing +strength for the shared U space and the different V space. In this simulation +the separate fit had a +`r if (comparison$frobenius_error[2] < comparison$frobenius_error[1]) "lower" else "higher"` +estimation error than the common fit (Frobenius error +`r round(comparison$frobenius_error[2], 2)` compared with +`r round(comparison$frobenius_error[1], 2)`). The size of the difference depends +on the balance grid and on the cross-validation split, and this is one simulated +data set, so it should not be read as a general result. + +## 6. Diagnostics and Sensitivity Checks + +### Grid boundaries and balance-penalty sensitivity + +For the multiple-source example, both selected penalties are at their upper +grid endpoints. In exploratory runs, extending the space grid to 100 and the +balance grid to 10, then extending the space grid to 1000, moved the selection +to the upper endpoints again. This illustrates why a selected value should be +interpreted relative to the grid supplied, rather than as a best penalty. + +For the asymmetric example, the selected U penalty is also at the upper endpoint, +while the common balance penalty is at the lower endpoint. Extending a grid can +change both the selected parameters and the estimation-error comparison. + +The balance penalty is less well determined than the space penalties. The common +search selected `r cv_common$lambda_2_min`, which is +`r if (cv_common$lambda_2_min == min(balance_grid)) "the smallest value in the grid" else if (cv_common$lambda_2_min == max(balance_grid)) "the largest value in the grid" else "inside the grid"`, +and the separate search selected `r cv_separate$lambda_2_min`. At the selected +space penalties the CV scores for the `r length(balance_grid)` balance values are +close to one another (see the plot below, if ggplot2 is available), so the +selected balance penalty can change with the fold split or with `step_size`. A +grid that continues below `r min(balance_grid)` can also change the common +selection and the size of the difference between the two fits. + +```{r asymmetric-cv-plot, eval = requireNamespace("ggplot2", quietly = TRUE) && utils::packageVersion("ggplot2") >= "3.4.0"} +plot_cv(cv_separate, lambda_1_row_all = space_grid, + lambda_1_col_all = space_grid, lambda_2_all = balance_grid, + n_observed = sum(!is.na(dat$Y_target))) +``` + +### Final refits and fits inside cross-validation + +The stop codes in the table (`r fit_common$convergence_criterion` for the common +refit, `r fit_separate$convergence_criterion` for the separate refit) describe +the two refits only. Code 1 means that the change in the objective fell below +the threshold, and code 2 that the iteration limit was reached. +`cv.learner()` does not report the stop codes of the fits it runs inside +cross-validation. When this example was prepared, about a third of those fits in +each search reached the iteration limit. Meeting the threshold does not by itself +show that a fit is close to its minimum, and a fit that reaches the limit can +still give a usable held-out score, so it is more informative to check whether +the selection is stable. Doubling `max_iter` or changing `step_size` to 0.002, +0.005, 0.001 or 0.0005 (with larger budgets for the smaller steps) left the +penalties selected by both searches unchanged. A larger step size of 0.01 (with +2000 updates) changed only the balance penalty chosen by the separate search. + + + +### Reproduce the sensitivity checks + +The following optional code uses the asymmetric data, rank, grids and fold seed +from Section 5. It repeats both searches and reports the selected penalties, +refit errors and stopping codes. It is not run when the vignette is built. +The smaller steps are paired with larger iteration budgets. + +```{r sensitivity-checks, eval=FALSE} +check_setting <- function(step, budget) { + rows <- lapply(c(FALSE, TRUE), function(separate) { + grids <- if (separate) { + list(lambda_1_row_all = space_grid, lambda_1_col_all = space_grid) + } else { + list(lambda_1_all = space_grid) + } + set.seed(815) + cv <- do.call(cv.learner, c(list( + Y_source = dat$Y_source, Y_target = dat$Y_target, r = 3, + lambda_2_all = balance_grid, n_folds = 3, step_size = step, + control = list(max_iter = budget)), grids)) + u <- if (separate) cv$lambda_1_row_min else cv$lambda_1_min + v <- if (separate) cv$lambda_1_col_min else cv$lambda_1_min + fit <- learner(dat$Y_source, dat$Y_target, r = 3, + lambda_1_row = u, lambda_1_col = v, + lambda_2 = cv$lambda_2_min, step_size = step, + control = list(max_iter = budget)) + data.frame(method = if (separate) "Separate" else "Common", + step_size = step, max_iter = budget, + U_penalty = u, V_penalty = v, balance = cv$lambda_2_min, + error = norm(fit$learner_estimate - dat$Theta_target, "F"), + stop_code = fit$convergence_criterion) + }) + do.call(rbind, rows) +} +configurations <- data.frame( + step = c(0.003, 0.003, 0.002, 0.005, 0.001, 0.0005, 0.01), + budget = c(2000, 4000, 6000, 4000, 10000, 20000, 2000)) +sensitivity <- do.call(rbind, lapply(seq_len(nrow(configurations)), function(i) { + check_setting(configurations$step[i], configurations$budget[i]) +})) +print(sensitivity, row.names = FALSE) +``` + +To inspect the fits *inside* CV, the next optional block reconstructs the three +folds and fits each candidate. Common-penalty candidates are the diagonal of +the separate grid, so they can be inspected from the same results. This uses +complete target data; with missing data, sample only observed entries. + +```{r inner-cv-stopping, eval=FALSE} +set.seed(815) +indices <- sample(which(!is.na(dat$Y_target))) +n <- length(indices) +folds <- lapply(1:3, function(k) { + indices[seq.int(floor((k - 1) * n / 3) + 1, floor(k * n / 3))] +}) +grid <- expand.grid(U = space_grid, V = space_grid, balance = balance_grid) +inner_fits <- do.call(rbind, lapply(seq_len(nrow(grid)), function(i) { + do.call(rbind, lapply(1:3, function(k) { + training <- dat$Y_target + training[folds[[k]]] <- NA + fit <- learner(dat$Y_source, training, r = 3, + lambda_1_row = grid$U[i], lambda_1_col = grid$V[i], + lambda_2 = grid$balance[i], step_size = 0.003, + control = list(max_iter = 2000)) + data.frame(U = grid$U[i], V = grid$V[i], balance = grid$balance[i], + fold = k, stop_code = fit$convergence_criterion) + })) +})) +table(inner_fits$stop_code) +with(subset(inner_fits, U == V), table(stop_code)) +``` + +Changing the penalty grids above also allows grid sensitivity to be explored. +For the multiple-source example, rerun Section 4 with expanded grids and keep +its data, weights and fold seed fixed. In either workflow, simulation truth is +used only to describe estimation error after selection, not to choose penalties. + +### Inspect the objective history + +```{r asymmetric-objective} +plot_objective(fit_separate) +``` + +The curve shows the recorded objective values of the separate-penalty fit, +`r length(fit_separate$objective_values)` updates in all (stop code +`r fit_separate$convergence_criterion`). A curve that ends at the iteration limit +cannot show whether more iterations would still change the fit. A larger +`max_iter` helps if the objective is still decreasing, and can help if an +alternation between two values is shrinking, though slowly. Changing `step_size` +can help as well. + +To inspect a specific range, use `iterations = 1:100`. The +`iterations = "near_optimum"` view shows up to 41 records around the smallest +recorded value. It draws the raw values, so a curve that zigzags between two +levels shows that the objective alternates from one iteration to the next +instead of settling. + +### Additional estimates and diagnostic views + +D-LEARNER still projects onto both source spaces directly, without separate +penalties. It provides another estimate to inspect, not a separate-penalty fit. +The target as its own source gives the target-only truncated SVD, shown as a +reference. In this example D-LEARNER has the larger error of the two, which can +happen when one source space differs from the target's, because D-LEARNER relies +on both. + +```{r asymmetric-direct} +direct_asymmetric <- dlearner(dat$Y_source, dat$Y_target, r = 3) +target_only <- dlearner(dat$Y_target, dat$Y_target, r = 3) +c(Target_only = norm(target_only$dlearner_estimate - dat$Theta_target, "F"), + D_LEARNER = norm(direct_asymmetric$dlearner_estimate - dat$Theta_target, "F")) +``` + +Additional diagnostic views are available: + +```{r additional-plots, eval=FALSE} +plot_source_spaces(dat$Y_target, dat$Y_source, r = 3) +plot_contributions(fit_separate) +``` + +## 7. Practical Takeaways + +Use source weights to specify how multiple populations contribute to the +combined source-space information. D-LEARNER applies that information directly; +LEARNER also uses penalties to control departures from the spaces while fitting +the target. Separate U- and V-space penalties allow those departures to be +penalized differently when a common strength is too restrictive. + +Start with the single-source workflow in the README. Introduce multiple sources +when relevant aligned populations are available, and consider separate penalties +when the two spaces may differ in similarity. These extensions can be combined, +but are presented separately here to make their roles clear. Check grid +boundaries and stopping behavior before interpreting a selected fit. Simulation +truth helps explain these examples; it is not available for choosing parameters +in an application, and one example does not establish a general advantage. + +See [`?learner`](../help/learner), [`?cv.learner`](../help/cv.learner), [`?dlearner`](../help/dlearner), [`?dat_multisource`](../help/dat_multisource), and [`?dat_asymmetric`](../help/dat_asymmetric) for details.