Parallel Model Fitting in R: Workers, Cores, and Threads

Summary

A parallel forecasting run that fits many independent statistical models stalled at 99% complete, with only 7 of a machine’s 20-plus logical processors busy. The cause was not faulty core detection. Two separate settings were at work: one that halved the number of parallel workers in exchange for multithreaded model fits, and a default in future.apply that assigned all work in fixed batches before the run began.

This note explains the concepts involved (cores, logical processors, workers, and threads), the trade-off between running more workers and giving each model fit more threads, and four changes that resolved the stall. With those changes in place, the full run completed without stalling in a couple of hours.

Key concepts

Physical cores. A physical core is an independent processing unit on the CPU. Each one can run one stream of heavy numerical work at full speed.

Logical processors. With hyperthreading, the operating system presents each physical core as two logical processors. Windows Task Manager shows logical processors, so a CPU with 16 physical cores may appear to have 24 or 32 “processors.” For compute-heavy work like model fitting, the physical core count is the more meaningful limit. In R, parallel::detectCores(logical = FALSE) returns physical cores, and detectCores() returns logical processors. One caveat: R’s documentation states that the logical = FALSE option is honored only on macOS, Solaris, and Windows. On Linux, it may return the logical count instead, so code that relies on it there can end up starting more workers than intended.

Workers. In the future framework, a worker is a separate background R session. With plan(multisession, workers = n), R launches n sessions, each receiving tasks and the data those tasks need. Each worker holds its own copy of that data, so memory use grows with the number of workers.

Threads. A thread is a stream of work inside a single process. Many R modeling functions can multithread a single fit, such as mgcv::bam() (via nthreads), xgboost (via nthread), or ranger (via num.threads). How much a fit gains from extra threads varies by library and model. In some, only certain steps of the fit run in parallel, so two threads make a fit faster but not twice as fast, and a worker running a two-threaded fit may spend much of its time on one core.

Putting them together. The load on the machine is roughly workers × threads per worker. When the job consists of many independent model fits, the most efficient arrangement is usually one single-threaded worker per physical core, since every core then does useful work for the whole run.

Workers vs. threads: the trade-off

Our pipeline supports two configurations, chosen by a single setting. Both hold one physical core back for the main R session and the operating system. Appendix A shows the logic in R.

On a machine with 16 physical cores, the two configurations compare as follows:

The multithreaded configuration explains the stalled run: about 7 busy cores, even though the machine showed more than 20 logical processors.

Why single-threaded fits are deterministic. Random number generation is a separate issue, handled by future.seed = TRUE, which gives every task its own reproducible random stream. The remaining source of run-to-run variation is floating-point arithmetic. When a fit is split across threads, partial results can be summed in a different order each time, and floating-point addition is not perfectly associative. The differences are tiny, but results will not match exactly. A single-threaded fit always sums in the same order and gives identical results every run.

Why exact replication matters. In quantitative research, being able to reproduce a result exactly is more than a convenience. When a forecast changes, you need to know whether the change came from new data, a code change, or nothing at all. If results drift slightly from run to run, that question becomes much harder to answer, and debugging turns into guesswork. Exact replication also lets you compare model versions fairly, verify that a refactor didn’t change any outputs, and recreate a past forecast precisely if it’s ever questioned. Tiny floating-point differences rarely change a conclusion, but they make it impossible to confirm that nothing else changed.

The counterintuitive result. The deterministic configuration is not a slower, cautious mode. For a job made of many independent fits, it is also likely the faster one, because it uses roughly twice as many workers and keeps every core busy. The multithreaded configuration earns its keep mainly when memory is tight or when a few very large fits dominate the job.

Case study: the run that stalled at 99%

The stall came from how tasks were assigned, not from how many workers existed. The pipeline ran smoothly on one dataset, then stalled near completion on a larger dataset whose tasks varied much more in size.

The cause: static chunking. By default, future.apply::future_lapply() splits its input into one chunk per worker before the run begins. Each worker processes only its own chunk. A worker that finishes early sits idle, even while another worker still has a long queue. When task sizes are uneven, the run’s length is set by the unluckiest chunk, and the last few percent can take a very long time with most cores idle.

We made four changes. None of them alters how any individual model is fit. Appendix B shows a simplified version of the revised pattern.

1. Dynamic scheduling. Setting future.scheduling = Inf makes each task its own unit of work, handed to whichever worker is free next.
2. Largest tasks first. Sorting tasks by size, largest first, means the longest fits start early and the small ones fill in around them. Without this, a large task that happens to come last can still run alone at the end.
3. Send each worker only the data it needs. Passing the full dataset along with every task multiplies memory use and transfer time. Splitting the data first and pairing each task with its own slice keeps each worker’s footprint small.
4. Define the worker function at the top level. In R, a function defined inside another function carries that function’s environment with it. When future sends such a function to the workers, any large objects in that environment can travel too. Defining the worker function at the top level avoids this hidden copy.

One side effect to note: with future.seed = TRUE, random streams are assigned by position in the input list. Reordering tasks therefore changes which stream each task receives. Results remain reproducible from run to run, but may differ slightly from those produced before the change.

Results

The changes worked. The engine no longer stalls near completion. The full production run, covering 72 separate model groups, thousands of companies, and decades of history, now completes in a couple of hours.

Practical checklist

For anyone fitting many models in parallel in R, these checks catch most of the problems described above:

1. Print the worker count at the start of every run, for example with print(future::plan()). Task Manager alone cannot tell you how many workers were requested.
2. Compare physical cores with logical processors before setting worker counts. In Windows, Task Manager’s Performance tab lists both.
3. Treat worker count and task scheduling as separate settings. Fixing one does not fix the other.
4. Use future.scheduling = Inf (or a small future.chunk.size) whenever task sizes vary.
5. Order tasks largest first.
6. Send each worker only the data its task needs, and define worker functions at the top level.
7. Use future.seed = TRUE for reproducible random streams, and single-threaded fits when results must match exactly.
8. Watch memory on the first run after increasing the worker count.

References

1. Bengtsson, H. future (R package). Documents workers, plan(multisession), and how globals are identified and exported to workers. cran.r-project.org/package=future
2. Bengtsson, H. future.apply (R package), reference page for future_lapply(). Documents the default chunking (future.scheduling = 1, one chunk per worker), future.chunk.size, and future.seed. search.r-project.org: future_lapply
3. R Core Team. detectCores(), package parallel. Explains the logical argument for physical cores versus logical CPUs. search.r-project.org: detectCores
4. Wood, S. N. mgcv (R package), including bam() and its nthreads argument. cran.r-project.org/package=mgcv
5. Chen, T., et al. xgboost (R package), including the nthread parameter. cran.r-project.org/package=xgboost
6. Wright, M. N. ranger (R package), including the num.threads argument. cran.r-project.org/package=ranger
7. Goldberg, D. (1991). What every computer scientist should know about floating-point arithmetic. ACM Computing Surveys, 23(1), 5–48. doi.org/10.1145/103162.103163

Appendix A: Setting workers and threads

physical_cores <- parallel::detectCores(logical = FALSE)
usable_cores   <- max(1L, physical_cores - 1L)

if (deterministic) {
  num_workers <- usable_cores          # one worker per core
  fit_threads <- 1L                    # single-threaded fits
} else {
  num_workers <- max(1L, floor(usable_cores / 2))
  fit_threads <- max(1L, floor(usable_cores / num_workers))
}

Appendix B: The revised scheduling pattern

# Top-level worker function: carries no hidden environment
fit_one <- function(task, p) {
  res <- fit_model(task$data)   # your model-fitting function
  p()
  res
}

run_all <- function(dt, groups, group_col) {
  slices <- split(dt, by = group_col)          # data.table split
  tasks  <- lapply(groups, function(g) list(id = g, data = slices[[as.character(g)]]))

  sizes  <- vapply(tasks, function(t) NROW(t$data), integer(1))
  ord    <- order(-sizes)                      # largest first

  res <- progressr::with_progress({
    p <- progressr::progressor(steps = length(tasks))
    future.apply::future_lapply(
      tasks[ord], fit_one, p = p,
      future.seed       = TRUE,                # reproducible random streams
      future.scheduling = Inf                  # dynamic load balancing
    )
  })
  res[order(ord)]                              # restore original order
}