Quickstart: Spline Derivatives

In many cases, useful information can be extracted from the slopes of backscatter, DO and pH curves. The problem however is that a naiive implementation of point-to-point differences (e.g. numpy.diff) does not account for measurement noise and is therefore very noisy itself.

With bletl.splines, we provide a robust implementation for approximating curves by smoothing splines, giving access to a robust estimation of derivatives.

import ipywidgets
import numpy
import pandas
import pathlib
from matplotlib import cm, pyplot

import bletl

Parse the raw data file

This quickstart uses a raw data file of a process with robotically triggered base and carbon source addition.

filepath = pathlib.Path(r"..\tests\data\BL1\NT_1200rpm_30C_DO-GFP75-pH-BS10_12min_20171221_121339.csv")
bldata = bletl.parse(filepath, lot_number=1515, temp=30)

Plot the data of interest

This notebook contains an interactive plot, so you can play around with different well/filterset/method combinations.

It also means you may need to execute the notebook to see the result.

Make sure to interactively change the last_cycle setting and observe how it affects the prediction of the spline. This is important if you want to use the method for at-line analysis and automated triggering.

def plot_well(method:str, well:str, filterset:str, last_cycle:int=None):
    x, y = bldata[filterset].get_timeseries(well, last_cycle=last_cycle)

    spline = bletl.get_crossvalidated_spline(x, y, method=method, bounds=(0,1.2), k_folds=10)
    derivative = spline.derivative(1)

    # plotting
    fig, (up, down) = pyplot.subplots(dpi=140, nrows=2, sharex='col')
    up.scatter(x, y, s=.4, label="data")
    up.plot(x, spline(x), label="spline", linewidth=0.5)

    down.axhline(0, color="gray", linewidth=1, linestyle=":")
    d = numpy.linspace(0, x[-1], 1000)
    down.plot(d, derivative(d), label="1st derivative")

    # formatting
    #down.set_xlim(0, 23)
    down.set_xlabel("time   [h]")
    down.set_ylabel(f"d{filterset}/dt")
    up.set_ylabel(filterset)
    up.legend()

    pyplot.show()
    return

ipywidgets.interact(
    plot_well,
    method=["us", "ucss"],
    well=list(bldata["pH"].time.columns),
    filterset=list(bldata.keys()),
    last_cycle=range(10, len(bldata["pH"].time)+1)
);

<function __main__.plot_well(method: str, well: str, filterset: str, last_cycle: int = None)>

%load_ext watermark
%watermark -n -u -v -iv -w

Last updated: Fri Jul 02 2021

Python implementation: CPython
Python version       : 3.7.9
IPython version      : 7.19.0

bletl     : 1.0.0
numpy     : 1.19.2
matplotlib: 3.3.2
pandas    : 1.2.1
ipywidgets: 7.6.3

Watermark: 2.1.0