where your rules come alive.

A conformance rule engine for standardised clinical data, written in Java: SDTM, ADaM and SEND rules from the rule sheets of CDISC, FDA and PMDA, and rules written straight from the CDISC implementation guides — checks written in a language you can read, understand and extend. That makes it easy to add custom checks, for example against a company-extended implementation guide.

Use it as a Java library, on the command line, through a REST API with its web client, or inside Cumba Data Browser.

Open source · AGPL-3.0 SDTM · ADaM · SEND CDISC · FDA · PMDA Library · CLI · REST API Inside Cumba Data Browser
CoreJ - ein Herz, dessen rechte Haelfte in ein j auslaeuft

What CoreJ is

A fast engine

A runtime engine that checks clinical data, written in Java and open source. Datasets are checked in parallel, and the rules within each dataset can run on as many worker threads as you have cores. Every check is compiled once when it is loaded, then run a whole column at a time rather than row by row.

Complete rule sets

SDTM, ADaM and SEND rules from the authorities' rule sheets:

  • CDISC Conformance Rules: SDTMIG 3.2–3.4, ADaMIG 1.0–1.3, SENDIG 3.0–3.1.1, SENDIG-DART 1.1–1.2, SENDIG-GENETOX 1.0
  • FDA Validator and Business Rules: SDTMIG 3.1.2–3.4, SENDIG 3.0–3.1.1, SENDIG-AR 1.0, SENDIG-DART 1.1
  • PMDA Validation Rules: SDTMIG 3.1.2–3.4, ADaMIG 1.0–1.3
  • Define-XML 2.0 and 2.1, for CDISC and PMDA

Plus our own rules, written directly from the CDISC implementation guides: SDTMIG 3.2–3.4, ADaMIG 1.2–1.3, SENDIG 3.0–3.1.1. Every rule points to the passage it comes from, and none asks for more than the guide does.

A language for checks

The rule sets that ship with CoreJ and the checks you add yourself are written in the same language: plain expressions over the dataset's variables, with functions for each kind of check:

  • normal checks — values, text, patterns
  • clinical dates and durations
  • keys and relationships
  • across rows and datasets
  • metadata, Define-XML, controlled terminology and dictionaries

Your own rules alongside

Company-specific checks in that same language, running in the same pass as the standard ones.

Grouped findings

The report sums up each rule per dataset with a count — not a wall of one line per row. The affected rows, variables and values are listed underneath, up to a limit you can set. As JSON or Excel.

Three comparisons

Conformance is checked in three directions — structure, labels, lengths, codelists. Each one optional:

  • data against your Define-XML
  • data against the CDISC definitions
  • your Define-XML against the CDISC definitions

Define-XML 2.0 and 2.1.

Tested rule by rule

Every rule ships with its own test data (cases that must be flagged and cases that must pass), and every build runs them. Each rule from CDISC, FDA or PMDA carries the id its authority publishes it under.

Your data as it is

Reads SAS XPT and SAS7BDAT, Dataset-JSON (also NDJSON and compressed), CSV, Excel and Parquet. CDISC Library content comes from a local metadata store, seeded once without an API key; controlled terminology and the external dictionaries (MedDRA, WHODrug, LOINC and more; you supply the licensed ones) are bound per run.

One language for every check

The rule sets that ship with CoreJ and the checks you add yourself are written in the same language.

Example check
Core: { Id: "P300-AE0001" }
Description: "AESTDTC is earlier than RFICDTC in DM."
Scope: { Domains: { Include: ["AE"] } }
Match_Datasets: [ { Name: "DM", Keys: ["USUBJID"] } ]
Check:
  expression: >-
    not empty(AESTDTC) and date(AESTDTC) < date(DM.RFICDTC)
Outcome:
  Message: "AESTDTC is before RFICDTC."
  Output_Variables: ["AESTDTC", "DM.RFICDTC"]

Add checks of your own for what your organisation requires — same language, same run. Write your own: the rule guide →

Example finding
Core Id:  P300-AE0001
Dataset:  ae.dsjc      Row: 412
Variable: AESTDTC
Value:    "2021-03-04"
Message:  AESTDTC is before RFICDTC.

The finding carries the check's id and its message, so the whole rule can be read from it.

How can I use CoreJ?

One engine, four ways to run it, from the one that needs no setup to the one inside your own code — plus a rule editor.

1

In Cumba Data Browser

One dialog runs the checks on the library you have open. When you open the library from its Define-XML, the rule packages matching the standard it declares are pre-selected. The findings are highlighted at the rows and values they belong to. The validation report opens as a view of its own, with overview, rules, datasets and findings, ready to filter, sort and export. The Define-XML itself can be validated too. No setup beyond the browser itself.

cumba.net →   The manual →

2

On the command line

Run a study from a script, a pipeline step or a nightly job. The data stays on the machine that runs the check.

Download the release, unzip it and run it — the rule sets come bundled, and Java 25 is all it needs: ./run.sh -rp cdisc-sdtmig-3-4 -d ./datasets -o report.json (run.bat on Windows). The findings come out as JSON, as Excel (-of xlsx), or both (-of json,xlsx). Seed the CDISC metadata store once with ./run.sh --seed-cache, no API key needed; Define-XML, controlled terminology and dictionaries are options of the same command.

Releases on GitHub →

3

As a service: REST API and web client

Run CoreJ as a server in your own network. The REST API takes the study files into a session, runs the check in the background and hands back the findings, page by page, or the whole report as JSON or Excel. Its OpenAPI description and Swagger UI come with it.

The web client is the browser front end for it: drop in the study files, choose the rule packages and the Define-XML, follow the run and browse the findings and each rule's definition. Both come as release downloads; if you build the REST service yourself, it can bundle the web client and serve it from the same address. There is no login built in, so run it where only your team can reach it.

REST API on GitHub →   Web client on GitHub →

4

As a Java library, in your own tool

Call the engine from your own Java code: an in-house review tool, a data pipeline, your own front end. You hand it the datasets and the rule sets to run, and get the findings back. The command line and the REST API are built on it, and so is Cumba Data Browser's own engine.

The engine is on Maven Central as net.cumba:cumba-oss-corej-core, with the Define-XML checks cumba-oss-corej-define-conformance and the report writers cumba-oss-corej-report-json and cumba-oss-corej-report-xlsx beside it. You describe the run — rule packages, datasets, Define-XML, controlled terminology — start the validation, and get the findings back as an object. Java 25; give large studies enough memory, for example -Xmx8g.

Maven Central →   Source on GitHub →

5

Write and test rules: the rule editor

The CoreJ rule editor is a browser tool for reviewing and testing the rules of a rule corpus kept in git. It checks a rule while you type, completes functions, variables and controlled terminology, runs the rule against its test cases with the same code as the corpus build, and commits and pushes your change to a branch. The editor and its service come as release downloads, and the source is on GitHub.

Editor on GitHub →   Its service on GitHub →

The engine, the rule corpus and the tools are open source, on GitHub

CoreJ was developed for Cumba Data Browser, and we open-sourced the whole engine under AGPL-3.0. Anyone may use it, in their own company and in their own study. If you distribute CoreJ, or let others use a modified version of it over a network, the AGPL asks you to offer the corresponding source under the same licence; the licence text is what governs.

Tell us how it goes

A rule that fired when it should not have. One that stayed quiet when it should have spoken up. A check you needed and did not find. Any of those is worth a message, and none of them is too small.

Found a bug? Send the rule, the input that triggered it and what you expected instead — that is what we need to reproduce it.

Need the documentation? Each repository's README on GitHub, and the rule guide for writing checks of your own. A manual is on its way.

Want to write to us? Pick what fits, the email opens ready to go:

Want to stay posted? Releases go out in the P300 newsletter, one for all our tools.