Underwriting Models in R and Python on One Platform: MLflow and Databricks

Last updated: October 2026

In many insurers the actuarial team builds sound pricing models in R, and the hard part is getting them into production. When deployment means sending a file to IT and waiting for a release cycle, models reach underwriters months after the data they were trained on.

This article describes the platform we built for a property and casualty insurer, covered in our insurance underwriting case study. Commercial policies took five to seven business days to underwrite, each needed more than 40 data points from six systems, and deploying an actuarial model took about six weeks. Project details come from that engagement. The code is a simplified illustration written for this article, using the Databricks and MLflow features current in October 2026.

One platform for actuaries and engineers

The biggest improvement did not come from the models. It came from putting actuaries and data engineers on the same platform. Before the project, the actuarial team worked in R on their laptops and the engineering team worked in Python on separate infrastructure. In Databricks notebooks with a shared MLflow registry, the actuaries kept writing R, the engineers kept writing Python, and both could work on the same data and the same model lifecycle.

The pricing models had not been recalibrated since 2021, and the loss ratio had been rising by 2.3% a year.

Bringing six underwriting sources together

Each commercial submission depended on data from six places, and each source needed its own ingestion pattern into Delta Lake:

For the inspection reports, Databricks now has a generally available SQL function for this job. ai_parse_document, GA since 16 April 2026, takes binary file content and returns text, tables (as HTML), figures and layout elements such as titles and headers, with bounding boxes. It accepts PDF, image, Word and PowerPoint files up to 500 pages and 100 MB, requires Databricks Runtime 17.3 or later, and is available in some regions only.

CREATE OR REPLACE TABLE underwriting.bronze.inspection_reports_parsed AS
SELECT
    path,
    ai_parse_document(content) AS parsed
FROM READ_FILES(
    '/Volumes/underwriting/raw/inspection_reports/',
    format => 'binaryFile'
);

Put data quality in front of the models

One thing we would do differently is start with the data quality layer. The third-party property feed had a 12% null rate on building square footage, a critical feature for property risk scoring. We did not catch it until model validation showed unexpectedly high error rates on commercial property policies, and checks in front of the models would have saved about a week of debugging.

On Databricks, Lakeflow pipeline expectations are the native way to add those checks. An expectation can warn and keep the row, drop the row, or fail the update, and warnings are recorded in the pipeline's metrics, which is how a rising null rate becomes visible before training. Databricks now recommends the pyspark.pipelines module over the older dlt module.

from pyspark import pipelines as dp

@dp.materialized_view
@dp.expect("building_sqft_present", "building_sqft IS NOT NULL")          # warn and keep
@dp.expect("building_sqft_plausible", "building_sqft BETWEEN 100 AND 5000000")
@dp.expect_or_drop("has_property_id", "property_id IS NOT NULL")          # drop the row
def property_features():
    return spark.read.table("underwriting.bronze.property_feed")

Great Expectations, which we would have used at the time, is still a good choice outside pipelines; its GX Core library is at version 1.24, built around expectation suites, validation definitions and checkpoints.

Registering R and Python models together

The actuarial R models were versioned through MLflow, ending the practice of emailing files to IT. Actuaries could push a new model version from their R notebooks, run automated validation against holdout data, and promote it in one step. Alongside them, the engineers built Python XGBoost models that score risk in real time from the unified features.

Two things have changed in MLflow since then. Models registered in Unity Catalog do not support stages such as Production; promotion is done by moving an alias, such as champion, to the approved version. And new model versions in Unity Catalog must have a model signature. Databricks documents Unity Catalog registration through the Python MLflow client, so for a mixed team we recommend doing the registration step from Python.

import mlflow
import xgboost as xgb
from mlflow.models import infer_signature

mlflow.set_registry_uri("databricks-uc")
MODEL = "underwriting.models.commercial_property_risk"

model = xgb.XGBRegressor(n_estimators=400, max_depth=6).fit(X_train, y_train)

with mlflow.start_run():
    info = mlflow.xgboost.log_model(
        model, name="model",
        signature=infer_signature(X_train, model.predict(X_train)))

version = mlflow.register_model(info.model_uri, MODEL).version

# Move the alias only after the holdout checks pass.
mlflow.MlflowClient().set_registered_model_alias(MODEL, "champion", version)

The mlflow package for R is still maintained, at version 3.10.1 in March 2026 against 3.17.1 for Python, and it can log R models with the crate flavour. If your team also uses Spark from R, note that SparkR is deprecated from Databricks Runtime 16.0 and Databricks recommends sparklyr.

Serving both behind one API

In the project, both the R and XGBoost models sat behind a REST API. When a submission arrives in Guidewire, an event trigger calls the Databricks endpoint, which gathers the 40 data points, runs both models and returns a risk score with a feature-importance explanation of which factors drove it. The underwriter sees a pre-populated screen with the data and a recommended price range, and reviews and approves instead of researching from scratch.

If you build this today, plan the R side deliberately. Databricks documents Model Serving for Python models logged with MLflow's built-in flavours or as pyfunc, and does not mention R. The options are to wrap the R model in a Python pyfunc that calls R through rpy2, which a Databricks engineer has described as involving several complex adjustments, to host R models on a separate platform such as Posit Connect, or to score R models in batch and serve the results. For pricing models that change a few times a year, batch scoring of a rating table can be enough, with the real-time model reserved for the Python side.

Integrating with Guidewire

The Guidewire integration was harder than expected. Its Cloud API rate limits were not well documented, and the event webhook payload schema changed between Guidewire versions without clear deprecation warnings. We built a retry queue with exponential backoff and a dead-letter mechanism for calls that kept failing, so that failed calls were retried and, if they kept failing, set aside for review instead of being dropped.

If you are integrating with Guidewire, we recommend budgeting an extra two to three weeks for API edge cases, and validating every webhook payload against a schema before processing it, so that a version change fails loudly at the edge instead of producing a wrong score.

Underwriting time fell from five to seven days to about four hours, and the loss ratio improved by 3.2% in the first year. Actuarial model deployment went from about six weeks to a matter of hours, and the 40 data points that underwriters used to gather by hand are now filled in automatically.