Introducing rcicrdiagnostics: an R package that runs data quality checks on your reverse correlation data

R
Reverse Correlation
Psychology
Package
Author

M. Oliveira

Published

April 24, 2026

Image generated with Gemini 3 Flash Image (Nano banana 2).

If you’ve ever run a reverse correlation (RC) experiment, you’ve probably lived this moment before: you feed your data into the classification image (CI) generation pipeline, look at the output, and something just feels off (but apparently nothing technically broke). You see no errors or warnings, just a CI that looks plausible and yet doesn’t quite add up (if you’re critical enough to even notice, which I’m hoping we all are ;) ).

More often than not, the issue is not with your theory, but with your data pipeline. AS a Super Mario fan, I was waiting for the moment I could write this in a blog post about research stuff: Time for some plumbing, let’sa gooo! (yes, that’s how bad my sense of humor can be ;) )

RC pipelines are unusually fragile to small data mistakes. Some examples: a response column coded as {0, 1} instead of {-1, 1}, stimulus IDs that are off by one, a noise matrix whose dimensions don’t match what your response data assumes, an .RData parameter file from a different stimulus batch than the one your participants actually saw. None of these throw an error. All of them produce a CI that looks convincing but may carry some unwanted mathematical distortions behind the scenes. Worse, because CIs are visually interpretable, a broken pipeline can still look “like a face” and trick us into believing the result.

After running into these kinds of issues more times than I’d like to admit — both in my own projects and when helping others — I decided to put together a toolkit that checks for them systematically. It’s called rcicrdiagnostics and I’m releasing v0.1.0 today.

What it does

rcicrdiagnostics is an R package that runs a battery of data-quality diagnostics on your response data before you compute any CIs or information values. Think of it as a pre-flight checklist: you load your response CSV and the auxiliary file that describes your stimuli, call run_diagnostics(), and get back a clear pass / warn / fail report per check.

It supports both of the pipelines that currently dominate the field:

  • Two-image forced choice (2IFC) — the classical reverse correlation paradigm implemented in the rcicr package (Dotsch, 2016, 2023).
  • Brief-RC — the more recent variant introduced by Schmitz, Rougier, and Yzerbyt (2024) that shows 12 or 20 noisy faces per trial instead of two.

A single method argument decides which pipeline to validate; the rest of the API is shared.

What the checks actually catch

Here’s a flavour of what comes out of run_diagnostics(). Every finding lands in one of four buckets (pass, warn, fail, skip), and each check names the participants or trials involved so you can jump straight to fixing them.

  • Response coding. Are responses {-1, +1} as the pipeline expects, or accidentally {0, 1} from your data-collection tool? This is easy to miss and it silently halves your CI.
  • Trial counts. Does every participant have the number of trials you expected? Participants who terminated early or whose data got partially lost skew your group-level CIs.
  • Stimulus alignment. Do the stimulus IDs in your response data actually exist in your stimulus pool, and are they within range? Off-by-one errors here are surprisingly common and hard to spot visually.
  • Response bias. Are some participants just clicking the same side over and over? These produce near-constant “CIs” that carry no signal.
  • Reaction times. Any participants with an unusually high rate of extremely fast responses? Likely rushed or inattentive trials that contaminate the CI.
  • Version compatibility. Is your .RData parameter file from the same rcicr version you’re now trying to analyse with?
  • Response inversion. Does a participant’s CI look like its own mirror image? A small but real fraction of participants get confused about which face they’re supposed to be picking, and their “inverted” CIs cancel out everyone else’s.

If everything passes, you can move on to generating CIs and computing infoVal with confidence. If something fails, the report tells you exactly which participants or rows to investigate before you continue.

Quick look

The package ships with a runnable demo that you can try without any real study data. It generates synthetic 2IFC and Brief-RC datasets using an AI-generated base face (thispersondoesnotexist.com), runs the full diagnostic battery on each, and prints the reports:

# install.packages("remotes")
remotes::install_github("olivethree/rcicrdiagnostics")

library(rcicrdiagnostics)

# Source the runnable examples in order (about a minute total).
source(system.file("examples", "01_generate_stimuli.R",          package = "rcicrdiagnostics"))
source(system.file("examples", "02a_bogus_responses_2ifc.R",     package = "rcicrdiagnostics"))
source(system.file("examples", "02b_bogus_responses_briefrc12.R", package = "rcicrdiagnostics"))
source(system.file("examples", "02c_bogus_responses_briefrc20.R", package = "rcicrdiagnostics"))
source(system.file("examples", "03_run_diagnostics_demo.R",       package = "rcicrdiagnostics"))

For your own data, the minimum you need is a response CSV with participant_id, stimulus, and response columns (optionally rt), plus either an rcicr .RData for 2IFC or a noise matrix for Brief-RC:

responses <- read.csv("path/to/your/responses.csv")

report <- run_diagnostics(
  responses,
  method = "2ifc",
  rdata  = "path/to/your/rcicr_stimuli.RData",
  col_rt = "rt"
)
print(report)

That’s it.

A note on CI scaling decisions

The package ships with a user’s guide which, besides covering every function with runnable examples, includes a chapter I hope will save some headaches: Choosing a CI scaling option.

It answers three questions that come up in pretty much every RC project:

  1. What does the scaling argument in rcicr actually do?
  2. Which scaling option should I use for which downstream analysis — infoVal, pixel-wise correlations, rating-task stimuli, figures?
  3. When does scaling affect my results, and when does it not?

There is a lot of confusion in the literature about this, and it shows up in how people interpret their CIs, how they present them to judges, and how they compare CIs across conditions or studies. The chapter lays out the distinction between the raw CI (the data you do statistics on) and the scaled CI (the display-ready image you show to eyes), and provides a decision table for picking the right scaling per analytical goal.

Status and what’s next

v0.1.0 is the first tagged release. The API is not yet frozen, I expect to refine function signatures and defaults before a potenial new future release, so please indicate the current version if you use it in any of your papers. Distribution is GitHub-only for now; there is no CRAN plan at this stage as working on the RC methods toipic is mainly a “hobby” beyond my daily job research focus.

Feedback, bug reports, and feature requests are very welcome. Please open an issue on GitHub or reach out directly (m.j.barbosa.de.oliveira@tue.nl).

How to cite

Oliveira, M. (2026). rcicrdiagnostics: Data quality diagnostics for reverse correlation experiments in social psychology using the rcicr package (Version 0.1.0) [R package]. Zenodo. https://doi.org/10.5281/zenodo.19734757

Or from within R: citation("rcicrdiagnostics").

Thanks for reading!

References

  • Brinkman, L., Goffin, S., van de Schoot, R., van Haren, N. E. M., Dotsch, R., & Aarts, H. (2019). Quantifying the informational value of classification images. Behavior Research Methods, 51(5), 2059–2073. https://doi.org/10.3758/s13428-019-01232-2

  • Dotsch, R. (2016). rcicr: Reverse-correlation image-classification toolbox [R package, development versions]. https://github.com/rdotsch/rcicr

  • Dotsch, R. (2023). rcicr: Reverse correlation image classification toolbox (Version 1.0.1) [R package]. https://github.com/rdotsch/rcicr

  • Schmitz, M., Rougier, M., & Yzerbyt, V. (2024). Introducing the brief reverse correlation: An improved tool to assess visual representations. European Journal of Social Psychology. https://doi.org/10.1002/ejsp.3100