Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
b0068bf
Add objective, CV, and latent-space plotting diagnostics
Jodielidada Oct 16, 2026
6211428
Merge branch 'multisource-penalties-clean' into plotting-diagnostics-…
Jodielidada Oct 16, 2026
ee5ca49
Merge branch 'multisource-penalties-clean' into plotting-diagnostics-…
Jodielidada Oct 16, 2026
3e076b6
Fix objective pair endpoints and clarify plotting usage
Jodielidada Oct 16, 2026
e743ffa
Place objective-plot window around the minimum and keep the legend of…
Jodielidada Oct 16, 2026
65553cc
Show the objective plot for the single-source example in the README
Jodielidada Oct 16, 2026
42aca12
Clarify README installation, grid boundaries, stopping code, and sour…
Jodielidada Oct 16, 2026
5cae4ca
Add a second source comparison with noisier populations
Jodielidada Oct 16, 2026
27a93a3
Merge core examples into plotting branch
Jodielidada Oct 16, 2026
279c45e
Add plot previews for the pull request description
Jodielidada Oct 16, 2026
7de8372
Merge pull request #6 from stmcg/plotting-diagnostics-clean
stmcg Oct 19, 2026
799d7fd
Bring the plotting diagnostics into main
Jodielidada Oct 23, 2026
59fb1d0
Remove PR preview images and the plotting guide
Jodielidada Oct 23, 2026
6064f47
Simplify the plotting help pages
Jodielidada Oct 23, 2026
6ad1d2a
Restore the original README and add the CV and objective plots
Jodielidada Oct 23, 2026
c6674ce
Show the raw objective values in the near-optimum view
Jodielidada Oct 23, 2026
9928bff
Add dat_asymmetric: one source whose U space matches the target
Jodielidada Oct 23, 2026
083dc57
Add an advanced tutorial for multiple sources and separate penalties
Jodielidada Oct 23, 2026
440757c
Simplify the argument and value text in the plotting help pages
Jodielidada Oct 23, 2026
cddc518
Add near-optimum figure for PR review
Jodielidada Oct 23, 2026
d3965ec
Clarify vignette flow for source weights and method-specific penalties
Jodielidada Oct 24, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 9 additions & 2 deletions DESCRIPTION
Original file line number Diff line number Diff line change
Expand Up @@ -30,11 +30,18 @@ URL: https://github.com/stmcg/learner
BugReports: https://github.com/stmcg/learner/issues
Imports: ScreeNOT,
Rcpp (>= 1.0.11),
RcppEigen
RcppEigen,
graphics,
grDevices,
stats
LinkingTo: Rcpp, RcppEigen
Suggests:
testthat (>= 3.0.0)
testthat (>= 3.0.0),
ggplot2 (>= 3.4.0),
knitr,
rmarkdown
Config/testthat/edition: 3
Depends:
R (>= 2.10)
VignetteBuilder: knitr
RoxygenNote: 7.3.2
4 changes: 4 additions & 0 deletions NAMESPACE
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,10 @@
export(cv.learner)
export(dlearner)
export(learner)
export(plot_contributions)
export(plot_cv)
export(plot_objective)
export(plot_source_spaces)
import(RcppEigen)
importFrom(Rcpp,sourceCpp)
useDynLib(learner, .registration = TRUE)
4 changes: 4 additions & 0 deletions NEWS.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,10 @@
populations with independently generated noise.
* Add separate row- and column-space penalties in `learner()` and
cross-validation over these penalties in `cv.learner()`.
* Add `dat_asymmetric` and an advanced tutorial illustrating multiple sources
and separate space penalties in two distinct workflows.
* Add plotting functions for objective function values, cross-validation
results, population latent spaces, and variable contributions.

### learner version 1.0.0 (2025-03-02)

Expand Down
53 changes: 53 additions & 0 deletions R/data-asymmetric.R
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
#' Simulated data set: One shared latent space
#'
#' One source and one target with the same true U-space but different true
#' V-spaces. This example illustrates why the two space penalties in LEARNER
#' can be selected separately.
#'
#' @docType data
#' @format A list of four aligned matrices, each with 100 rows and 50 columns:
#' \describe{
#' \item{Y_source}{Observed source data.}
#' \item{Y_target}{Observed target data.}
#' \item{Theta_source}{True rank-3 source signal.}
#' \item{Theta_target}{True rank-3 target signal.}
#' }
#' @details The target signal is the rank-3 truncated SVD of a standard normal
#' matrix. The source keeps its U-space, reverses its three singular values,
#' and perturbs its V basis with independent uniform entries between
#' \eqn{-1/(2\sqrt{50})} and \eqn{1/(2\sqrt{50})}, followed by QR orthonormalization.
#' This adapts the U-shared, V-perturbed "low similarity" scenario in Section
#' 2.4.1 of the Supporting Information of McGrath et al. (2026) to a small
#' example (the paper uses 500 by 500 matrices of rank 4). It is not a
#' replication of that study. The Supporting Information calls this setting
#' "same column spaces but different row spaces": the span of U is the column
#' space of the signal matrix, and the span of V is its row space.
#'
#' Independent mean-zero normal noise is added to each signal, with variance
#' 0.10 for the target and 0.01 for the source. The exact shared-space statement
#' applies to the true signals; noise also affects estimated spaces.
#' True signals are supplied to illustrate estimation error, not for fitting
#' or penalty selection. Generation code is in \file{data-raw/gen-asymmetric.R}
#' in the source repository.
#'
#' @references
#' McGrath, S., Zhu, C., O'Dea, R., Guo, M. and Duan, R. (2026). \emph{LEARNER: A transfer learning method for low-rank matrix estimation}. Statistics in Medicine, 45(15-17), e70669. \doi{10.1002/sim.70669}.
#'
#' @examples
#' data(dat_asymmetric)
#' d <- dat_asymmetric
#' error <- function(fit) norm(fit$learner_estimate - d$Theta_target, "F")
#' # Equal penalties versus stronger borrowing on the shared U side. The values
#' # are illustrative and were not selected by cross-validation; see
#' # vignette("advanced-learner") for selecting them.
#' common <- learner(d$Y_source, d$Y_target, r = 3, lambda_1 = 10,
#' lambda_2 = 0.1, step_size = 0.003,
#' control = list(max_iter = 2000))
#' separate <- learner(d$Y_source, d$Y_target, r = 3, lambda_1_row = 100,
#' lambda_1_col = 10, lambda_2 = 0.1, step_size = 0.003,
#' control = list(max_iter = 2000))
#' c(common = error(common), separate = error(separate))
#' @seealso \code{\link{dat_highsim}}, \code{\link{dat_multisource}},
#' \code{\link{learner}}, \code{\link{cv.learner}}
#' @keywords datasets
"dat_asymmetric"
189 changes: 189 additions & 0 deletions R/plot-spaces.R
Original file line number Diff line number Diff line change
@@ -0,0 +1,189 @@
# Internal helpers shared by the space and contribution plots.
check_plot_matrix <- function(x, name) {
if (!is.matrix(x) || !is.numeric(x) || is.complex(x) || any(dim(x)==0) || any(!is.finite(x))) {
stop(name, ' must be a complete finite real numeric matrix.')
}
}

plot_indices <- function(index, n, max_display, name) {
if(is.null(index)) return(unique(as.integer(round(seq(1,n,length.out=min(n,max_display))))))
if(!is.numeric(index)||is.complex(index)||length(index)==0||any(!is.finite(index))||
any(index!=floor(index))||any(index<1|index>n)||anyDuplicated(index)) {
stop(name, ' must contain distinct valid integer indices.')
}
as.integer(index)
}

plot_svd <- function(x, r) {
s<-svd(x,nu=r,nv=r)
if(s$d[r] <= max(dim(x))*s$d[1]*.Machine$double.eps) {
stop('r includes numerically zero singular directions; use a smaller rank.')
}
s
}

plot_limits_pair <- function(limits, automatic) {
if(is.null(limits))return(automatic)
if(!is.numeric(limits)||is.complex(limits)||length(limits)!=2L||
any(!is.finite(limits))||any(limits<=0))stop('limits must contain two finite positive values: row then column.')
stats::setNames(as.numeric(limits),c('row','column'))
}

# Use a raster to avoid drawing hundreds of thousands of individual rectangles.
draw_matrix_panel <- function(values, x_index, y_index, limits, colors, title, xlab, ylab) {
nx<-nrow(values);ny<-ncol(values)
clipped<-pmax(limits[1],pmin(limits[2],values))
bins<-1L+floor((clipped-limits[1])/diff(limits)*(length(colors)-1L))
pixels<-matrix(colors[pmax(1L,pmin(length(colors),bins))],nx,ny)
graphics::plot.new()
# Reserve a constant fraction of the panel width for the color key.
graphics::plot.window(xlim=c(0.5,nx+0.5+nx*0.15),ylim=c(0.5,ny+0.5),xaxs='i',yaxs='i')
graphics::rasterImage(grDevices::as.raster(t(pixels[,ny:1,drop=FALSE])),
0.5,0.5,nx+0.5,ny+0.5,interpolate=FALSE)
graphics::rect(0.5,0.5,nx+0.5,ny+0.5,border='gray40',lwd=0.7)
ix<-unique(as.integer(round(seq(1,nx,length.out=min(5,nx)))))
iy<-unique(as.integer(round(seq(1,ny,length.out=min(8,ny)))))
graphics::axis(1,at=ix,labels=x_index[ix],cex.axis=0.85)
graphics::axis(2,at=iy,labels=y_index[iy],las=1,cex.axis=0.85)
graphics::title(main=title,xlab=xlab,ylab=ylab,cex.main=1.05,cex.lab=0.9)
draw_color_key(nx+0.5+nx*0.05,nx+0.5+nx*0.075,0.5,ny+0.5,limits,colors)
}

#' Compare target and source latent spaces
#'
#' Compare patterns in the target and source latent spaces. Similar patterns
#' suggest that the populations share similar spaces at the chosen rank.
#'
#' @param Y_target Complete target matrix, or a completed estimate. Missing
#' values are not allowed.
#' @param Y_source Source matrix, or a list of source matrices, with the same
#' rows and columns as the target.
#' @param r Rank used for every matrix.
#' @param row_index,column_index Optional indices of the rows and columns to
#' display. The singular vectors are always computed from the full matrices.
#' @param max_display Maximum number of rows, and of columns, to display when
#' no indices are given (default 200). Evenly spaced indices are used.
#' @param limits Optional pair of positive color limits (row side, then column
#' side). Each scale is symmetric around zero and shared across populations.
#' Values beyond a limit get the extreme color, and their number is returned as
#' \code{clipped_count}.
#' @param row_label,column_label Axis labels. The defaults are generic, so use
#' labels that describe your rows and columns.
#' @param target_name Label for the target panel.
#' @return Invisibly, a list with the displayed projection matrices
#' (\code{row_projectors} and \code{column_projectors}, named by population), the
#' displayed \code{row_index} and \code{column_index}, \code{limits}, and
#' \code{clipped_count}.
#' @details Each heatmap shows a projection matrix computed from the leading
#' singular vectors. Column-side patterns appear above row-side patterns.
#' Red indicates negative values and blue positive values; populations share
#' a color scale within each side so their patterns can be compared.
#'
#' Rows and columns must already be aligned across populations. Selecting
#' indices changes only what is displayed; spaces are estimated from the full
#' matrices. These plots show each population separately, without source weights.
#' Similar colors are a visual comparison, not a statistical test of shared spaces.
#'
#' @examples
#' plot_source_spaces(dat_multisource$Y_target, dat_multisource$Y_source,
#' r = 3, max_display = 20)
#' @export
plot_source_spaces <- function(Y_target, Y_source, r, row_index=NULL, column_index=NULL,
max_display=200L, limits=NULL, row_label='Row index',
column_label='Column index', target_name='Target population') {
check_plot_matrix(Y_target,'Y_target')
sources<-prepare_sources(Y_source,Y_target,allow_missing_target=FALSE)$matrices
for(s in sources)check_plot_matrix(s,'Y_source')
r<-resolve_rank(r,Y_target);validate_count(max_display,'max_display')
ri<-plot_indices(row_index,nrow(Y_target),max_display,'row_index')
ci<-plot_indices(column_index,ncol(Y_target),max_display,'column_index')
labels<-names(sources)
if(is.null(labels))labels<-rep('',length(sources))
labels[is.na(labels)]<-''
labels<-ifelse(nzchar(labels),paste('Source',labels),paste('Source',seq_along(sources)))
populations<-c(list(Y_target),sources);names(populations)<-make.unique(c(target_name,labels))
bases<-lapply(populations,plot_svd,r=r)
up<-lapply(bases,function(s)tcrossprod(s$u[ri,,drop=FALSE]))
vp<-lapply(bases,function(s)tcrossprod(s$v[ci,,drop=FALSE]))
auto<-c(row=max(vapply(up,function(p)max(abs(p)),numeric(1))),
column=max(vapply(vp,function(p)max(abs(p)),numeric(1))))
auto[auto==0]<-1e-8
cap<-plot_limits_pair(limits,auto)
counts<-list(row=vapply(up,function(p)sum(abs(p)>cap[1]),integer(1)),
column=vapply(vp,function(p)sum(abs(p)>cap[2]),integer(1)))
colors<-grDevices::colorRampPalette(c('firebrick3','white','dodgerblue3'))(101)
old<-graphics::par(no.readonly=TRUE);on.exit(graphics::par(old),add=TRUE)
for(page in split(seq_along(populations),ceiling(seq_along(populations)/3))) {
graphics::par(mfrow=c(2,length(page)),mar=c(3.5,3.8,2.5,4),oma=c(2,0,0,0),mgp=c(2.2,0.6,0))
for(i in page)draw_matrix_panel(vp[[i]],ci,ci,c(-cap[2],cap[2]),colors,names(populations)[i],column_label,column_label)
for(i in page)draw_matrix_panel(up[[i]],ri,ri,c(-cap[1],cap[1]),colors,names(populations)[i],row_label,row_label)
caption<-paste('Red: negative; blue: positive. Shared scales within each row.',
if(sum(unlist(counts))>0)paste(sum(unlist(counts)),'displayed entries clipped across all panels.')else '')
graphics::mtext(caption,outer=TRUE,side=1,line=0.5,cex=0.8)
}
invisible(list(row_projectors=up,column_projectors=vp,row_index=ri,column_index=ci,
limits=cap,clipped_count=counts))
}

#' Plot variable contributions to latent components
#'
#' Show which rows and columns contribute to each latent component of a fitted
#' target matrix, using squared singular-vector entries. Higher scores are darker red.
#'
#' @param x A complete estimated matrix, or a \code{learner()} or
#' \code{dlearner()} result. The singular vectors are not rotated.
#' @param r Number of components. Required for a matrix; for a fitting result
#' the default is its rank.
#' @inheritParams plot_source_spaces
#' @param limits Optional pair of positive upper limits for the color scales
#' (row side, then column side). Each scale runs from 0 (white) to its limit
#' (dark red). Larger values get the darkest color, and their number is returned
#' as \code{clipped_count}.
#' @param main Plot title.
#' @return Invisibly, a list with the full \code{row_scores} and
#' \code{column_scores} (variables in rows, components in columns), the displayed
#' \code{row_index} and \code{column_index}, \code{limits}, and
#' \code{clipped_count}. The returned scores are not clipped.
#' @details Darker red indicates a larger contribution to a component.
#' Columns are shown on the left and rows on the right, with separate color
#' scales. Scores are squared singular-vector entries from the estimated matrix;
#' they sum to one over all variables on each side for each component.
#'
#' Scores describe contributions, not effect sizes or correlations. Selecting
#' variables changes only the display, without rescaling scores. Components are
#' not matched across populations, and components with tied singular values are
#' not unique.
#'
#' @examples
#' fit <- dlearner(dat_multisource$Y_source, dat_multisource$Y_target, r = 3)
#' plot_contributions(fit,max_display=50)
#' @export
plot_contributions <- function(x, r=NULL, row_index=NULL, column_index=NULL,
max_display=200L, limits=NULL, row_label='Row index',
column_label='Column index', main='Estimated target contributions') {
if(is.list(x)) {
estimate<-if(!is.null(x$learner_estimate))x$learner_estimate else x$dlearner_estimate
if(is.null(r))r<-x$r
} else estimate<-x
check_plot_matrix(estimate,'x')
if(is.null(r))stop('Supply r for a matrix without a stored rank.')
r<-resolve_rank(r,estimate);validate_count(max_display,'max_display')
ri<-plot_indices(row_index,nrow(estimate),max_display,'row_index')
ci<-plot_indices(column_index,ncol(estimate),max_display,'column_index')
s<-plot_svd(estimate,r)
row_scores<-s$u^2;col_scores<-s$v^2
cap<-plot_limits_pair(limits,c(row=max(row_scores[ri,,drop=FALSE]),column=max(col_scores[ci,,drop=FALSE])))
cap[cap==0]<-1e-8
counts<-c(row=sum(row_scores[ri,,drop=FALSE]>cap[1]),column=sum(col_scores[ci,,drop=FALSE]>cap[2]))
colors<-grDevices::colorRampPalette(c('white','firebrick3'))(100)
old<-graphics::par(no.readonly=TRUE);on.exit(graphics::par(old),add=TRUE)
graphics::par(mfrow=c(1,2),mar=c(3.8,3.8,2.2,4),oma=c(2,0,2,0),mgp=c(2.2,0.6,0))
draw_matrix_panel(col_scores[ci,,drop=FALSE],ci,seq_len(r),c(0,cap[2]),colors,'Column contributions',column_label,'Component')
draw_matrix_panel(row_scores[ri,,drop=FALSE],ri,seq_len(r),c(0,cap[1]),colors,'Row contributions',row_label,'Component')
graphics::mtext(main,outer=TRUE,side=3,line=0.4,font=2,cex=1.1)
caption<-paste('Squared singular-vector entries; separate color scales.',
if(sum(counts)>0)paste(sum(counts),'displayed entries clipped.')else '')
graphics::mtext(caption,outer=TRUE,side=1,line=0.4,cex=0.8)
invisible(list(row_scores=row_scores,column_scores=col_scores,row_index=ri,column_index=ci,
limits=cap,clipped_count=counts))
}
Loading
Loading