Signal processing
Digital signal processing on sampled signals: convolution, correlation, windows, spectra and power spectral density, peaks, smoothing, detrending and filters. Typical uses: analyse sensor data, find the frequencies of a vibration.
dsp convolvedsp convolvedsp correlatedsp correlatedsp autocorrdsp autocorrdsp windowdsp spectrumdsp periodogramdsp welchdsp peaksdsp peaksdsp moving_averagedsp savgoldsp detrenddsp detrenddsp filterdsp lomb_scargledsp lomb_scargledsp coherencedsp spectra_prep

dsp convolve <json_x> <json_h>


Description

    Linear convolution of two arrays (mode full: n+m-1 values), as numpy.convolve. Direct computation for the small arrays, FFT for the big ones.

Parameters

    json_x:   The signal (JSON array of numbers) - String - required
    json_h:   The kernel - String - required
admin
dsp convolve "[1, 2, 3]" "[0, 1, 0.5]"
mentdb
[0,1,2.5,4,1.5]
Moving sum of 2 points
admin
dsp convolve "[1, 2, 3, 4]" "[1, 1]"
mentdb
[1,3,5,7,4]
Product of two polynomials (1 + 2x)(3 + x)
admin
dsp convolve "[1, 2]" "[3, 1]"
mentdb
[3,7,2]
Impulse: the filter itself
admin
dsp convolve "[1, 0, 0]" "[0.25, 0.5, 0.25]"
mentdb
[0.25,0.5,0.25,0,0]

dsp convolve <json_x> <json_h> <mode>


Description

    Linear convolution with a mode: full (n+m-1 values), same (size of the biggest array, centered), valid (only where the arrays fully overlap), as numpy.convolve.

Parameters

    json_x:   The signal (JSON array of numbers) - String - required
    json_h:   The kernel - String - required
    mode:   full, same or valid - String - required
admin
dsp convolve "[1, 2, 3, 4, 5]" "[1, 1, 1]" "valid"
mentdb
[6,9,12]
Same size as the signal
admin
dsp convolve "[1, 2, 3, 4, 5]" "[1, 1, 1]" "same"
mentdb
[3,6,9,12,9]
Full mode (the default)
admin
dsp convolve "[1, 2, 3, 4, 5]" "[1, 1, 1]" "full"
mentdb
[1,3,6,9,12,9,5]
Smoothing with a triangular kernel
admin
dsp convolve "[0, 0, 4, 0, 0, 2, 0]" "[0.25, 0.5, 0.25]" "same"
mentdb
[0,1,2,1,0.5,1,0.5]
Unknown mode: error
admin
dsp convolve "[1, 2]" "[1]" "foo"
mentdb
Sorry, an error appears (unknown mode 'foo' (full, same, valid)).
Used in use cases (2 recipes)

    Pharmacist & pharmacokinetics › Multiple oral dosing by convolution
    Epidemiologist › Hospital admissions: delay from cases and short-term forecast
Often used with

    cas integrate  ·  dsp correlate  ·  dsp detrend  ·  dsp peaks  ·  json load select  ·  matrix col  ·  num table  ·  plot signal

dsp correlate <json_x> <json_y>


Description

    Cross-correlation c[k] = sum x[n+k]*y[n] (mode full: lags from -(len(y)-1) to len(x)-1), as scipy.signal.correlate. The lag of the maximum gives the delay between two signals.

Parameters

    json_x:   The signal (JSON array of numbers) - String - required
    json_y:   The second signal - String - required
admin
dsp correlate "[1, 2, 3]" "[0, 1, 0.5]"
mentdb
[0.5,2,3.5,3,0]
Delay between two signals
admin
dsp correlate "[0, 0, 1, 2, 1, 0]" "[1, 2, 1]"
mentdb
[0,0,1,4,6,4,1,0]
Find a pattern in a signal: the maximum gives the position
admin
dsp correlate "[0, 1, 0, 0, 2, 4, 2, 0]" "[1, 2, 1]"
mentdb
[0,1,2,1,2,8,12,8,2,0]

dsp correlate <json_x> <json_y> <mode>


Description

    Cross-correlation with a mode: full, same (size of the first array, centered), valid, as scipy.signal.correlate.

Parameters

    json_x:   The signal (JSON array of numbers) - String - required
    json_y:   The second signal - String - required
    mode:   full, same or valid - String - required
admin
dsp correlate "[1, 2, 3, 4, 5]" "[1, 0, -1]" "same"
mentdb
[-2,-2,-2,-2,4]
Valid mode: only the full overlaps
admin
dsp correlate "[1, 2, 3, 4, 5]" "[1, 0, -1]" "valid"
mentdb
[-2,-2,-2]
Full mode
admin
dsp correlate "[1, 2, 3]" "[1, 1]" "full"
mentdb
[1,3,5,3]
Used in use cases (3 recipes)

    Epidemiologist › Hospital admissions: delay from cases and short-term forecast
    Geologist & seismologist › Arrival delay between two seismic stations
    Electronics & signal engineer › Pulse compression with a Barker 13 code
Often used with

    dsp peaks  ·  stat max  ·  dsp convolve  ·  dsp detrend  ·  json doc  ·  json load select  ·  matrix col  ·  num table

dsp autocorr <json_x>


Description

    Autocorrelation function r[k] for the lags 0 to n-1 (normalized: r[0] = 1, mean removed), as statsmodels acf. A periodic signal gives peaks at the multiples of its period.

Parameters

    json_x:   The signal (JSON array of numbers) - String - required
admin
dsp autocorr "[1, 3, 2, 5, 4, 6]"
mentdb
[1,0.1,0.34285714285714286,-0.44285714285714284,-0.14285714285714285,-0.35714285714285715]
Periodic signal
admin
dsp autocorr "[0, 0.951, 0.588, -0.588, -0.951, 0, 0.951, 0.588, -0.588, -0.951]"
mentdb
[1,0.3090169540333321,-0.6281577337028904,-0.5853676973471077,0.15450847701666606,0.5,0.15450847701666606,-0.2236492566862244,-0.18085922033044166,0]
Alternating signal: the sign changes at each lag
admin
dsp autocorr "[1, 2, 1, 2, 1, 2]"
mentdb
[1,-0.8333333333333334,0.6666666666666666,-0.5,0.3333333333333333,-0.16666666666666666]
Random noise: close to 0 except at lag 0
admin
dsp autocorr "[0.5, -1.2, 0.3, 0.8, -0.4, 1.1, -0.9, 0.2]"
mentdb
[1,-0.5708874458874459,0.09415584415584419,0.0827922077922078,-0.354978354978355,0.3674242424242425,-0.1331168831168831,0.014610389610389612]

dsp autocorr <json_x> <nlags>


Description

    Autocorrelation function for the lags 0 to nlags.

Parameters

    json_x:   The signal (JSON array of numbers) - String - required
    nlags:   The last lag - Number - required
admin
dsp autocorr "[1, 3, 2, 5, 4, 6, 5, 8, 7, 9]" 3
mentdb
[1,0.45,0.5,-0.03333333333333333]
Sine with a period of 5 samples: a new maximum at lag 5
admin
dsp autocorr "[0, 0.951, 0.588, -0.588, -0.951, 0, 0.951, 0.588, -0.588, -0.951]" 6
mentdb
[1,0.3090169540333321,-0.6281577337028904,-0.5853676973471077,0.15450847701666606,0.5,0.15450847701666606]
Lag 1 only
admin
dsp autocorr "[1, 3, 2, 5, 4, 6]" 1
mentdb
[1,0.1]
Used in use cases (2 recipes)

    Biologist & microbiologist › Predator-prey cycles of Lotka-Volterra
    Epidemiologist › Weekly incidence report with exact Poisson limits
Often used with

    bigdec compare  ·  bigdec div  ·  bigdec sum  ·  dsp detrend  ·  dsp moving_average  ·  dsp peaks  ·  dsp spectrum  ·  json load select

dsp window <name> <n>


Description

    Window of n points (symmetric, as numpy.hanning, numpy.hamming...): boxcar, hann, hamming, blackman, bartlett, flattop. Multiply a signal by a window before a spectrum to reduce the leakage.

Parameters

    name:   boxcar, hann, hamming, blackman, bartlett or flattop - String - required
    n:   The number of points - Number - required
admin
dsp window "hann" 5
mentdb
[0,0.49999999999999994,1,0.5000000000000001,0]
Hamming window
admin
dsp window "hamming" 5
mentdb
[0.08000000000000002,0.54,1,0.5400000000000001,0.08000000000000002]
Blackman window
admin
dsp window "blackman" 5
mentdb
[-1.3877787807814457E-17,0.3399999999999999,0.9999999999999999,0.3400000000000001,-1.3877787807814457E-17]
Bartlett (triangular)
admin
dsp window "bartlett" 5
mentdb
[0,0.5,1,0.5,0]
Flat top (amplitude measures)
admin
dsp window "flattop" 5
mentdb
[-4.210510000000013E-4,-0.05473684000000003,1.000000003,-0.05473683999999998,-4.210510000000013E-4]
Rectangular (no window)
admin
dsp window "boxcar" 4
mentdb
[1,1,1,1]
Unknown window: error
admin
dsp window "foo" 5
mentdb
Sorry, an error appears (unknown window 'foo' (boxcar, hann, hamming, blackman, bartlett, flattop)).
Used in use cases (1 recipe)

    Acoustics engineer › Fan noise PSD: periodogram against Welch
Often used with

    dsp periodogram  ·  dsp welch  ·  matrix col  ·  num table  ·  plot chart  ·  stat dist_random  ·  stat mean  ·  stat sum

dsp spectrum <json_x> <fs>


Description

    One-sided amplitude spectrum of a real signal: a sine of amplitude A at the frequency f gives A at f.
Exact discrete Fourier transform of the n values (any n, no zero padding): frequencies k*fs/n for k = 0 to n/2.
Returns {f, amplitude, phase (radians)}.

Parameters

    json_x:   The signal (JSON array of numbers) - String - required
    fs:   The sampling frequency (Hz) - Number - required
admin
dsp spectrum "[0, 1.0607, 0.5, 1.0607, 0, -1.0607, -0.5, -1.0607, 0, 1.0607, 0.5, 1.0607, 0, -1.0607, -0.5, -1.0607]" 8
mentdb
{"f": [0,0.5,1,1.5,2,2.5,3,3.5,4], "amplitude": [0,0,1.000028162804571,0,0,0,0.5000281628045709,0,0], "phase": [0,0,-1.5707963267948966,0,0,0,-1.570796326794897,0,0]}
Sine of 2 Hz sampled at 10 Hz
admin
dsp spectrum "[0, 0.951, 0.588, -0.588, -0.951, 0, 0.951, 0.588, -0.588, -0.951]" 10
mentdb
{"f": [0,1,2,3,4,5], "amplitude": [4.742874840267547E-17,1.8552687127769032E-16,1.0000579802757321,1.953222381142193E-16,1.8996532112663283E-4,4.6007864835756186E-17], "phase": [-2.782821983319221,2.1480195828229656,-1.5707963267948966,-3.095865835156779,1.5707963267949914,-1.281413005760718]}
Constant signal: everything at 0 Hz
admin
dsp spectrum "[2, 2, 2, 2]" 4
mentdb
{"f": [0,1,2], "amplitude": [2,0,0], "phase": [0,0,0]}
Sum of two sines of 1 Hz and 2 Hz
admin
dsp spectrum "[0, 1.707, 1, -0.293, 0, 0.293, -1, -1.707]" 8
mentdb
{"f": [0,1,2,3,4], "amplitude": [0,0.9999244942988892,1,7.55057011108673E-5,0], "phase": [0,-1.5707963267948966,-1.5707963267948963,1.5707963267938316,0]}
Used in use cases (4 recipes)

    Acoustics engineer › Spectrum and THD of a tone
    Epidemiologist › Weekly incidence report with exact Poisson limits
    Hydrologist & oceanographer › Tidal constituents from a sea level record
    Electronics & signal engineer › Dominant frequencies of a signal
Often used with

    dsp peaks  ·  matrix col  ·  num table  ·  bigdec compare  ·  bigdec div  ·  bigdec sum  ·  dsp autocorr  ·  dsp detrend

dsp periodogram <json_x> <fs>


Description

    Power spectral density (one-sided, density in unit^2/Hz), as scipy.signal.periodogram: mean removed, no window.
Returns {f, psd}.

Parameters

    json_x:   The signal (JSON array of numbers) - String - required
    fs:   The sampling frequency (Hz) - Number - required
admin
dsp periodogram "[1, 2, 1, 0, 1, 2, 1, 0]" 4
mentdb
{"f": [0,0.5,1,1.5,2], "psd": [0,0,1,0,0]}
Sine of 2 Hz sampled at 10 Hz
admin
dsp periodogram "[0, 0.951, 0.588, -0.588, -0.951, 0, 0.951, 0.588, -0.588, -0.951]" 10
mentdb
{"f": [0,1,2,3,4,5], "psd": [2.2494861750442914E-33,1.7210109983044337E-32,0.5000579819565882,1.9075388350973887E-32,1.8043411615372368E-8,2.11672362674521E-33]}
Periodogram of a noisy signal
admin
dsp periodogram "[0.5, -1.2, 0.3, 0.8, -0.4, 1.1, -0.9, 0.2]" 8
mentdb
{"f": [0,1,2,3,4], "psd": [0,0.04137209171765961,0.053125,0.45237790828234037,0.030625000000000006]}
Used in use cases (2 recipes)

    Acoustics engineer › Fan noise PSD: periodogram against Welch
    Climate & environment scientist › Warming trend and a hidden cycle
Often used with

    matrix col  ·  num table  ·  dsp detrend  ·  dsp peaks  ·  dsp welch  ·  dsp window  ·  pa rl close  ·  pa rl load_from_json

dsp welch <json_x> <fs> <nperseg>


Description

    Power spectral density by the Welch method, as scipy.signal.welch: segments of nperseg values, hann window, 50% overlap, mean removed in each segment, mean of the segments. Less noisy than the periodogram.
Returns {f, psd, segments}.

Parameters

    json_x:   The signal (JSON array of numbers) - String - required
    fs:   The sampling frequency (Hz) - Number - required
    nperseg:   The number of values per segment - Number - required
admin
dsp welch "[1, 2, 1, 0, 1, 2, 1, 0, 1, 2, 1, 0, 1, 2, 1, 0]" 4 8
mentdb
{"f": [0,0.5,1,1.5,2], "psd": [0,0.16666666666666666,0.6666666666666666,0.16666666666666666,0], "segments": 3}
Sine of 1 Hz sampled at 10 Hz, segments of 10
admin
dsp welch "[0, 0.588, 0.951, 0.951, 0.588, 0, -0.588, -0.951, -0.951, -0.588, 0, 0.588, 0.951, 0.951, 0.588, 0, -0.588, -0.951, -0.951, -0.588]" 10 10
mentdb
{"f": [0,1,2,3,4,5], "psd": [3.0404014055393164E-33,0.3333719879710588,0.08337466272256133,1.2028941076908978E-8,3.0072352692446657E-9,2.988362781710737E-33], "segments": 3}
Segment larger than the signal: one segment (the whole signal)
admin
dsp welch "[1, 2, 3]" 4 8
mentdb
{"f": [0,1.3333333333333333], "psd": [0.12500000000000006,0.25000000000000006], "segments": 1}
Used in use cases (1 recipe)

    Acoustics engineer › Fan noise PSD: periodogram against Welch
Often used with

    dsp periodogram  ·  dsp window  ·  matrix col  ·  num table  ·  plot chart  ·  stat dist_random  ·  stat mean  ·  stat sum

dsp peaks <json_x>


Description

    Find the peaks (local maxima, the middle of a plateau) as scipy.signal.find_peaks.
Returns {indices, values, prominences}: the prominence is the height of the peak above the highest of its two bases.

Parameters

    json_x:   The signal (JSON array of numbers) - String - required
admin
dsp peaks "[0, 2, 1, 3, 1, 2.5, 0, 1, 0]"
mentdb
{"indices": [1,3,5,7], "values": [2,3,2.5,1], "prominences": [1,3,1.5,1]}
Peaks of a noisy signal
admin
dsp peaks "[1, 3, 2, 4, 1, 5, 2, 2, 6, 1]"
mentdb
{"indices": [1,3,5,8], "values": [3,4,5,6], "prominences": [1,3,3,5]}
Plateau: the middle
admin
dsp peaks "[0, 1, 2, 2, 2, 1, 0]"
mentdb
{"indices": [3], "values": [2], "prominences": [2]}
No peak in a monotone signal
admin
dsp peaks "[1, 2, 3, 4]"
mentdb
{"indices": [], "values": [], "prominences": []}
All the local maxima, even the small ones
admin
dsp peaks "[0, 0.1, 1.2, 0.2, 0, 0.1, 0, 0.1, 1.1, 0.1, 0, 0.2, 0, 0.1, 1.3, 0.2]"
mentdb
{"indices": [2,5,8,11,14], "values": [1.2,0.1,1.1,0.2,1.3], "prominences": [1.2,0.1,1.1,0.2,1.1]}

dsp peaks <json_x> <json_options>


Description

    Find the peaks with conditions (JSON object, all optional), as scipy.signal.find_peaks:
- height: minimum value
- distance: minimum number of samples between two peaks (the highest peaks are kept)
- prominence: minimum prominence

Parameters

    json_x:   The signal (JSON array of numbers) - String - required
    json_options:   {"height": h, "distance": d, "prominence": p} - String - required
admin
dsp peaks "[0, 2, 1, 3, 1, 2.5, 0, 1, 0]" "{\"height\": 1.5, \"distance\": 3}"
mentdb
{"indices": [3], "values": [3], "prominences": [3]}
Only the prominent peaks
admin
dsp peaks "[0, 0.1, 1.2, 0.2, 0, 0.1, 0, 0.1, 1.1, 0.1, 0, 0.2, 0, 0.1, 1.3, 0.2]" "{\"prominence\": 0.5}"
mentdb
{"indices": [2,8,14], "values": [1.2,1.1,1.3], "prominences": [1.2,1.1,1.1]}
Minimum height
admin
dsp peaks "[1, 3, 2, 4, 1, 5, 2, 2, 6, 1]" "{\"height\": 4}"
mentdb
{"indices": [3,5,8], "values": [4,5,6], "prominences": [3,3,5]}
Minimum distance between peaks
admin
dsp peaks "[1, 3, 2, 4, 1, 5, 2, 2, 6, 1]" "{\"distance\": 4}"
mentdb
{"indices": [3,8], "values": [4,6], "prominences": [3,5]}
Used in use cases (12 recipes)

    Physicist › Logarithmic decrement and Q of a damped oscillator
    Acoustics engineer › Spectrum and THD of a tone
    Chemist › Equivalence point of a weak acid titration
    Biologist & microbiologist › Predator-prey cycles of Lotka-Volterra
    Epidemiologist › Hospital admissions: delay from cases and short-term forecast
    Climate & environment scientist › Warming trend and a hidden cycle
    and 6 more in the Use cases
Often used with

    matrix col  ·  num table  ·  dsp correlate  ·  dsp spectrum  ·  json load select  ·  num ode  ·  stat max  ·  dsp detrend

dsp moving_average <json_x> <w>


Description

    Moving average on w values (mode valid: n-w+1 values, the value i is the mean of x[i] to x[i+w-1]).

Parameters

    json_x:   The signal (JSON array of numbers) - String - required
    w:   The window size - Number - required
admin
dsp moving_average "[1, 2, 3, 4, 10, 6]" 3
mentdb
[2,3,5.666666666666667,6.666666666666667]
Smoothing over 5 points
admin
dsp moving_average "[2, 4, 3, 6, 5, 8, 7, 10, 9]" 5
mentdb
[4,5.2,5.8,7.2,7.8]
Window of 1: the signal
admin
dsp moving_average "[1, 2, 3]" 1
mentdb
[1,2,3]
Smoothing of noisy measures
admin
dsp moving_average "[10.2, 9.8, 10.5, 9.9, 10.1, 10.4, 9.7]" 3
mentdb
[10.166666666666666,10.066666666666668,10.166666666666666,10.133333333333333,10.066666666666665]
Used in use cases (2 recipes)

    Epidemiologist › Weekly incidence report with exact Poisson limits
    IoT & sensor engineer › Drift detection of a humidity sensor
Often used with

    and  ·  bigdec compare  ·  bigdec div  ·  bigdec sum  ·  dsp autocorr  ·  dsp detrend  ·  dsp spectrum  ·  json load select

dsp savgol <json_x> <window_length> <polyorder>


Description

    Savitzky-Golay smoothing, as scipy.signal.savgol_filter (mode interp): each value is replaced by the value of the polynomial fitted on the window around it. Smooths the noise and keeps the shape of the peaks better than a moving average.

Parameters

    json_x:   The signal (JSON array of numbers) - String - required
    window_length:   The window length (odd) - Number - required
    polyorder:   The order of the polynomial (less than the window length) - Number - required
admin
dsp savgol "[2, 4, 3, 6, 5, 8, 7, 10, 9]" 5 2
mentdb
[2.1142857142857143,3.342857142857143,4.285714285714286,4.628571428571431,6.371428571428571,6.62857142857143,8.371428571428572,9.085714285714285,9.228571428571428]
Smoothing that keeps the peaks
admin
dsp savgol "[1, 2, 4, 8, 4, 2, 1, 2, 1]" 5 3
mentdb
[1.1571428571428466,1.3714285714285757,4.942857142857143,6.2857142857142865,4.942857142857143,1.8285714285714298,1.428571428571429,1.7142857142857095,1.0714285714285676]
Keeps a parabola exactly (order 2)
admin
dsp savgol "[0, 1, 4, 9, 16, 25, 36]" 5 2
mentdb
[-6.581115618108337E-16,1.0000000000000002,4.000000000000002,9.000000000000004,16.000000000000004,25,36]
Smoothing of a noisy peak
admin
dsp savgol "[0.1, 0.3, 0.2, 0.9, 1.6, 2.1, 1.5, 1.0, 0.2, 0.3, 0.1]" 5 2
mentdb
[0.15714285714285695,0.13142857142857137,0.3628571428571431,0.8485714285714288,1.6600000000000001,1.9200000000000002,1.637142857142857,0.8628571428571431,0.4057142857142858,0.16285714285714237,0.13428571428571345]
Even window: error
admin
dsp savgol "[1, 2, 3, 4, 5, 6]" 4 2
mentdb
Sorry, an error appears (the window length must be odd).
Used in use cases (1 recipe)

    IoT & sensor engineer › Remove mains hum from a sensor by FFT
Often used with

    json uarray  ·  matrix col  ·  num fft  ·  num ifft  ·  num table  ·  plot spectrum  ·  stat dist_random  ·  vector add

dsp detrend <json_x>


Description

    Remove the linear trend (least squares line), as scipy.signal.detrend.

Parameters

    json_x:   The signal (JSON array of numbers) - String - required
admin
dsp detrend "[1, 2.2, 2.9, 4.1, 5]"
mentdb
[-0.05999999999999983,0.15000000000000036,-0.14000000000000012,0.07000000000000028,-0.019999999999999574]
Remove a linear drift
admin
dsp detrend "[0.1, 1.2, 1.9, 3.1, 4.0, 5.2]"
mentdb
[0.023809523809524197,0.12095238095238092,-0.18190476190476224,0.015238095238094829,-0.08761904761904837,0.10952380952380825]
A straight line becomes 0
admin
dsp detrend "[1, 2, 3, 4]"
mentdb
[-4.440892098500626E-16,0,0,4.440892098500626E-16]
Remove a drift from a sensor
admin
dsp detrend "[0, 1.1, 0.2, 1.3, 0.4, 1.5, 0.6]"
mentdb
[-0.4285714285714285,0.5714285714285716,-0.42857142857142855,0.5714285714285715,-0.4285714285714285,0.5714285714285715,-0.4285714285714285]

dsp detrend <json_x> <type>


Description

    Remove the trend: linear (least squares line) or constant (mean), as scipy.signal.detrend.

Parameters

    json_x:   The signal (JSON array of numbers) - String - required
    type:   linear or constant - String - required
admin
dsp detrend "[1, 2.2, 2.9, 4.1, 5]" "constant"
mentdb
[-2.04,-0.8399999999999999,-0.14000000000000012,1.0599999999999996,1.96]
Linear (the default)
admin
dsp detrend "[1, 2.2, 2.9, 4.1, 5]" "linear"
mentdb
[-0.05999999999999983,0.15000000000000036,-0.14000000000000012,0.07000000000000028,-0.019999999999999574]
Constant: remove the mean
admin
dsp detrend "[10, 12, 11, 13]" "constant"
mentdb
[-1.5,0.5,-0.5,1.5]
Used in use cases (3 recipes)

    Epidemiologist › Weekly incidence report with exact Poisson limits
    Climate & environment scientist › Warming trend and a hidden cycle
    Epidemiologist › Hospital admissions: delay from cases and short-term forecast
Often used with

    dsp peaks  ·  json load select  ·  bigdec compare  ·  bigdec div  ·  bigdec sum  ·  dsp autocorr  ·  dsp convolve  ·  dsp correlate

dsp filter <json_x> <fs> <type> <cutoff> <order>


Description

    Butterworth filter without phase shift (forward and backward), as scipy.signal.butter + scipy.signal.filtfilt (padding odd).
- type: lowpass, highpass (cutoff: a frequency in Hz), bandpass, bandstop (cutoff: "[low, high]")
- the cutoff frequencies must be below fs/2 (Nyquist)
- order 1 to 10 (the effective order is doubled by the forward-backward filtering)

Parameters

    json_x:   The signal (JSON array of numbers) - String - required
    fs:   The sampling frequency (Hz) - Number - required
    type:   lowpass, highpass, bandpass or bandstop - String - required
    cutoff:   The cutoff frequency (Hz) or [low, high] - String - required
    order:   The order of the filter (1 to 10) - Number - required
admin
dsp filter "[0, 1.0607, 0.5, 1.0607, 0, -1.0607, -0.5, -1.0607, 0, 1.0607, 0.5, 1.0607, 0, -1.0607, -0.5, -1.0607]" 8 "lowpass" 2 2
mentdb
[-1.3462326241470146E-5,0.697036533277046,0.9571374563393378,0.6970042532872518,-1.1275709785596444E-5,-0.696962053000557,-0.9570698020806241,-0.69733193528391,-3.6991180647036326E-4,0.6988536647040171,0.959289272919446,0.6863099470598071,-0.012565725710205783,-0.6346133470628598,-0.883894918658211,-1.0607298646826482]
High-pass filter: keeps the 2 Hz sine, removes the offset of 1
admin
dsp filter "[1, 1.951, 1.588, 0.412, 0.049, 1, 1.951, 1.588, 0.412, 0.049]" 10 "highpass" 0.5 2
mentdb
[-0.03988401794025298,0.931387324966643,0.6039863498107468,-0.5157271874000008,-0.7991922818418904,0.2588416854207784,1.3510592025507757,1.167368235216259,0.20435983916485934,0.07509202665678849]
Band-pass between 1.5 and 2.5 Hz
admin
dsp filter "[0, 1.0607, 0.5, 1.0607, 0, -1.0607, -0.5, -1.0607, 0, 1.0607, 0.5, 1.0607, 0, -1.0607, -0.5, -1.0607]" 8 "bandpass" "[1.5, 2.5]" 2
mentdb
[0.001060074785809004,0.036130734774016154,0.008496960452136994,0.025910593883864436,0.008386042285966577,-0.02972738633250585,-0.018331800828019214,-0.020104267239520232,-0.014286822786016092,-9.883450014580332E-4,0.06655612292939293,0.08922638724731279,-0.10453041317523352,-0.10369348333760424,0.12427387910982077,8.507399330917459E-4]
Band-stop around 2 Hz
admin
dsp filter "[0, 1.0607, 0.5, 1.0607, 0, -1.0607, -0.5, -1.0607, 0, 1.0607, 0.5, 1.0607, 0, -1.0607, -0.5, -1.0607]" 8 "bandstop" "[1.5, 2.5]" 2
mentdb
[-0.001130946726046994,1.0244372793668777,0.4915791168428236,1.03501631837094,-0.008388605549900369,-1.0312184608666062,-0.48188918105162737,-1.0405811086383807,0.014919543720290945,1.062384523335179,0.4323172176566519,0.9694606507036329,0.10581892441918667,-0.953401535202933,-0.6245383610208952,-1.065708281130079]
Cutoff above fs/2: error
admin
dsp filter "[1, 2, 3, 4, 5, 6, 7, 8]" 8 "lowpass" 5 2
mentdb
Sorry, an error appears (the cutoff frequency must be between 0 and fs/2 (Nyquist: 4)).
Worked example: Signal processing: a noisy sine (A signal of 5 Hz sampled at 100 Hz, with an offset of 1 and a component of 30 Hz. The spectrum shows the 3 components (0, 5 and 30 Hz); a low-pass filter that keeps the 5 Hz.)
admin
#Worked example: Signal processing: a noisy sine; #The signal: 64 samples (0.64 s); -> "[s]" "[1.0, 1.5943, 1.4114, 1.6327, 2.2364, 2.0, 1.6657, 1.9854, 1.7641, 1.0237, 1.0, 0.9763, 0.2359, 0.0146, 0.3343, 0.0, -0.2364, 0.3673, 0.5886, 0.4057, 1.0, 1.5943, 1.4114, 1.6327, 2.2364, 2.0, 1.6657, 1.9854, 1.7641, 1.0237, 1.0, 0.9763, 0.2359, 0.0146, 0.3343, 0.0, -0.2364, 0.3673, 0.5886, 0.4057, 1.0, 1.5943, 1.4114, 1.6327, 2.2364, 2.0, 1.6657, 1.9854, 1.7641, 1.0237, 1.0, 0.9763, 0.2359, 0.0146, 0.3343, 0.0, -0.2364, 0.3673, 0.5886, 0.4057, 1.0, 1.5943, 1.4114, 1.6327]"; #Spectrum (amplitude per frequency); plot spectrum [s] 100; #Low-pass filter at 10 Hz (Butterworth, order 4); dsp filter [s] 100 "lowpass" 10 4;
mentdb
[0.9999930097118939,1.3092939852234113,1.5875787315079606,1.8078460050884892,1.948836200921165,1.9969912339615492,1.9477492563474101,1.8059680189953273,1.5854636751230018,1.3077338200600306,0.9998934776649827,0.6920155916931277,0.41421168628500377,0.19369547893547154,0.05207638409600408,0.0032311980110093152,0.051966372796807214,0.19352620993240696,0.41404174519165166,0.6919217855420904,0.9999706385554283,1.308023464008542,1.5859175841410789,1.806461904410534,1.9480673818647611,1.9968590593299833,1.9480636332655332,1.8064600686535275,1.58589640368927,1.3079666471359002,0.9998988687621337,0.6918595551867278,0.4140138979694278,0.19358247339567827,0.052137106634054464,0.0034817612575836804,0.05233119462707974,0.19384421283614056,0.4141055652668805,0.691551553604036,0.9991105609572429,1.306831721443412,1.5848049544023937,1.8060319660463255,1.9489305584065049,1.9993377674716053,1.9518691717165808,1.8104828743326926,1.588278164196274,1.3065821499116705,0.9932480192562363,0.6801679760943998,0.40012010784749974,0.18304866731598754,0.05207443269408234,0.019916054809746342,0.086549478242003,0.23891598467614447,0.45337819074940244,0.7009987767716618,0.9546393744045072,1.1959451953764757,1.419740258524501,1.6335468082071016]
Used in use cases (2 recipes)

    Geologist & seismologist › P and S arrivals and epicentral distance
    Electronics & signal engineer › SNR gain of a low-pass filter
Often used with

    dsp peaks  ·  matrix col  ·  num table  ·  stat sum_sq  ·  vector sub

dsp lomb_scargle <json_times> <json_values>


Description

    Lomb-Scargle periodogram of unevenly sampled data (floating mean, as astropy LombScargle): the strongest periods with their power and false alarm probability (Baluev); light curves, field measures with gaps, biological rhythms

Parameters

    json_times:   The times of the measures - String - required
    json_values:   The values - String - required
admin
dsp lomb_scargle "[0.3, 1.1, 2.6, 3.2, 4.9, 5.4, 6.8, 7.7, 9.1, 9.6, 11.2, 12.5, 13.1, 14.8, 15.3, 16.9, 18.2, 19.4, 20.1, 21.7, 22.6, 24.0, 25.3, 26.8]" "[3.24, 3.32, 3.8, 3.71, 2.7, 2.34, 2.23, 2.87, 3.84, 3.41, 3.09, 2.83, 2.11, 1.92, 2.65, 3.57, 3.68, 3.45, 3.17, 2.34, 1.61, 3.07, 3.48, 3.74]"
mentdb
{"n": 24, "baseline": 26.5, "frequencies": 301, "min_frequency": 0.0037735849056603774, "max_frequency": 2.267924528301887, "peaks": [{"frequency": 0.12452830188679245, "period": 8.030303030303031, "power": 0.8660673043737613, "false_alarm_probability": 3.6090653920961557E-7}, {"frequency": 1.528301886792453, "period": 0.6543209876543209, "power": 0.41290676800973186, "false_alarm_probability": 0.48069264449155413}, {"frequency": 2.026415094339623, "period": 0.49348230912476715, "power": 0.40848698227987823, "false_alarm_probability": 0.5046749871642755}]}

dsp lomb_scargle <json_times> <json_values> <json_params>


Description

    Lomb-Scargle with parameters: min_frequency, max_frequency, samples_per_peak (5), nyquist_factor (5), peaks (number returned, 3), return_spectrum (the frequencies and the powers)

Parameters

    json_times:   The times - String - required
    json_values:   The values - String - required
    json_params:   The parameters - String - required
admin
dsp lomb_scargle "[0.3, 1.1, 2.6, 3.2, 4.9, 5.4, 6.8, 7.7, 9.1, 9.6, 11.2, 12.5, 13.1, 14.8, 15.3, 16.9, 18.2, 19.4, 20.1, 21.7, 22.6, 24.0, 25.3, 26.8]" "[3.24, 3.32, 3.8, 3.71, 2.7, 2.34, 2.23, 2.87, 3.84, 3.41, 3.09, 2.83, 2.11, 1.92, 2.65, 3.57, 3.68, 3.45, 3.17, 2.34, 1.61, 3.07, 3.48, 3.74]" "{\"max_frequency\": 0.5, \"peaks\": 1}"
mentdb
{"n": 24, "baseline": 26.5, "frequencies": 67, "min_frequency": 0.0037735849056603774, "max_frequency": 0.5, "peaks": [{"frequency": 0.12452830188679245, "period": 8.030303030303031, "power": 0.8660673043737613, "false_alarm_probability": 8.009743235337652E-8}]}
Used in use cases (1 recipe)

    Astronomer › Period of a variable star from irregular observations (Lomb-Scargle)

dsp coherence <json_x> <json_y> <fs> <nperseg>


Description

    Magnitude squared coherence of two signals (Welch, hann window, 50 % overlap, as scipy.signal.coherence): at which frequencies are they linearly related; phase of the cross spectrum, approximate 5 % significance level

Parameters

    json_x:   The first signal - String - required
    json_y:   The second signal - String - required
    fs:   The sampling frequency - Number - required
    nperseg:   The length of the segments - Number - required
admin
dsp coherence "[0.3, 0.668, 0.71, 0.821, 0.261, -0.888, -1.214, -0.471, 0.154, 0.432, 0.917, 1.004, 0.006, -1.005, -0.929, -0.427, -0.143, 0.464, 1.206, 0.897, -0.255, -0.832, -0.713, -0.656, -0.3, 0.734, 1.293, 0.605, -0.267, -0.536, -0.777, -0.935, -0.164, 0.977, 1.094, 0.413, -0.018, -0.408, -1.059, -0.991, 0.132, 0.957, 0.804, 0.508, 0.248, -0.572, -1.283, -0.77, 0.299, 0.693, 0.705, 0.798, 0.272, -0.868, -1.231, -0.487, 0.174, 0.442, 0.894, 0.999, 0.03, -1.007, -0.953, -0.419, -0.121, 0.45, 1.187, 0.916, -0.241, -0.853, -0.721, -0.633, -0.298, 0.709, 1.297, 0.628, -0.277, -0.557, -0.762, -0.919, -0.184, 0.966, 1.117, 0.418, -0.043, -0.407, -1.035, -0.998, 0.11, 0.97, 0.823, 0.49, 0.233, -0.55, -1.274, -0.793, 0.296, 0.717, 0.701, 0.774, 0.281, -0.846, -1.245, -0.504, 0.193, 0.455, 0.872, 0.993, 0.055, -1.007, -0.977, -0.413, -0.098, 0.439, 1.167, 0.932, -0.225, -0.874, -0.731, -0.609, -0.294, 0.685, 1.3, 0.652, -0.285, -0.579, -0.748, -0.901, -0.202, 0.953, 1.139, 0.426, -0.067, -0.408, -1.01, -1.003, 0.087, 0.981, 0.843, 0.474, 0.217, -0.53, -1.263, -0.817, 0.291, 0.742, 0.7, 0.75, 0.289, -0.824, -1.259, -0.523, 0.211, 0.469, 0.85, 0.984, 0.078, -1.004, -1.002, -0.409]" "[-0.452, 0.371, 0.362, 0.96, 0.519, -0.41, -0.377, -0.901, -0.582, 0.436, 0.406, 0.836, 0.639, -0.447, -0.448, -0.769, -0.687, 0.443, 0.501, 0.703, 0.722, -0.424, -0.562, -0.641, -0.744, 0.392, 0.627, 0.586, 0.752, -0.347, -0.695, -0.541, -0.744, 0.292, 0.76, 0.509, 0.722, -0.23, -0.821, -0.49, -0.686, 0.163, 0.873, 0.487, 0.638, -0.096, -0.915, -0.498, -0.581, 0.031, 0.944, 0.524, 0.517, 0.027, -0.959, -0.564, -0.45, -0.077, 0.958, 0.614, 0.383, 0.116, -0.943, -0.673, -0.32, -0.141, 0.913, 0.738, 0.263, 0.152, -0.871, -0.805, -0.216, -0.148, 0.818, 0.871, 0.18, 0.129, -0.757, -0.933, -0.159, -0.096, 0.692, 0.988, 0.152, 0.051, -0.624, -1.032, -0.16, 0.005, 0.559, 1.064, 0.183, -0.067, -0.498, -1.082, -0.219, 0.133, 0.446, 1.086, 0.267, -0.201, -0.405, -1.074, -0.324, 0.265, 0.376, 1.047, 0.388, -0.324, -0.362, -1.008, -0.455, 0.373, 0.363, 0.957, 0.522, -0.412, -0.378, -0.898, -0.585, 0.437, 0.408, 0.833, 0.641, -0.447, -0.451, -0.766, -0.689, 0.443, 0.504, 0.7, 0.724, -0.423, -0.565, -0.638, -0.745, 0.39, 0.631, 0.583, 0.752, -0.344, -0.698, -0.539, -0.743, 0.289, 0.763, 0.508, 0.72, -0.226, -0.823, -0.49, -0.684, 0.16, 0.875, 0.487, 0.635, -0.093, -0.917, -0.499]" 40 32
mentdb
{"segments": 9, "frequency": [0, 1.25, 2.5, 3.75, 5, 6.25, 7.5, 8.75, 10, 11.25, 12.5, 13.75, 15, 16.25, 17.5, 18.75, 20], "coherence": [0.005734744466193619, 0.004529370416577067, 0.11408705013920196, 0.9999989472618238, 0.9999992816306082, 0.9999849217195268, 0.016852432817657706, 0.004522910214847779, 0.008478868063477425, 0.006128187161700321, 0.006278754510914436, 0.00646391051950297, 0.005751167526314361, 0.006020362238210661, 0.010826504220292759, 0.0071732836971332165, 0.0011451781648464036], "phase": [3.141592653589793, -3.045921480335794, -2.0285710332808105, -0.5999002283569193, -0.6000744768065619, -0.600066320189227, -1.2513142583803218, -0.23993849105103268, 2.281693070107201, -0.6066309116070616, 2.414990251280062, -0.6941127564858603, 2.2965348073825256, -0.9791481049554527, -0.5914907872710473, -2.095229683152493, 2.8801576140382106E-11], "max_coherence": 0.9999992816306082, "max_frequency": 5, "significance_5_percent": 0.31234397806636793}
Used in use cases (1 recipe)

    Electronics & signal engineer › Do two vibration sensors see the same source? (coherence)

dsp spectra_prep <json_spectra> <json_steps>


Description

    Preprocess spectra (one row per spectrum) by steps applied in order: snv (standard normal variate), msc (multiplicative scatter correction on the mean spectrum), detrend (polynomial, degree), sg (Savitzky-Golay: window, order, derivative), als (asymmetric least squares baseline: lambda, p; the baselines are returned)

Parameters

    json_spectra:   The spectra - String - required
    json_steps:   The steps, ex: ["snv", {"step": "sg", "window": 11, "order": 2, "derivative": 1}] - String - required
admin
dsp spectra_prep "[[0.1835, 0.1757, 0.1809, 0.1848, 0.1884, 0.2263, 0.269, 0.35, 0.4809, 0.6332, 0.7964, 0.9521, 1.0542, 1.1036, 1.0966, 1.0491, 0.9784, 0.8854, 0.8045, 0.7151, 0.6184, 0.5257, 0.4344, 0.3587, 0.2995, 0.2505, 0.2238, 0.2127, 0.1921, 0.1809], [0.1097, 0.1171, 0.107, 0.1085, 0.1389, 0.1723, 0.2444, 0.3659, 0.5348, 0.7455, 0.9619, 1.1522, 1.2589, 1.2612, 1.1797, 1.0494, 0.8957, 0.7466, 0.6269, 0.531, 0.439, 0.3614, 0.2964, 0.239, 0.1945, 0.1538, 0.1459, 0.1253, 0.1124, 0.1151]]" "[\"snv\", {\"step\": \"sg\", \"window\": 5, \"order\": 2, \"derivative\": 1}]"
mentdb
{"steps": ["snv", "sg"], "spectra": [[-0.012617453833226322, -0.003505083748766666, 0.005607286335692907, 0.03224931347565191, 0.06458763149631488, 0.12193622666506848, 0.21025840349764977, 0.30430653939260527, 0.39696027074905516, 0.4508673568440501, 0.4347872023787297, 0.35560282550061095, 0.2230750579792336, 0.07013558146866711, -0.06114612242255629, -0.16453973554366663, -0.2218883307124203, -0.24977642148253326, -0.26413582141097447, -0.26864538502486507, -0.2757954168074153, -0.266064253219546, -0.2387695260828397, -0.20331604898679181, -0.15706335376274275, -0.10908990400181434, -0.07494182689926109, -0.05070292247459904, -0.04159055239013948, -0.03247818230567967], [-0.02856142065015793, -0.008084225859902858, 0.012392968930352188, 0.03541203772668916, 0.0842622345344831, 0.15436463107424647, 0.2452215177503828, 0.35755457347650704, 0.451571916084681, 0.49763493915713464, 0.46160076443594983, 0.3305787133951781, 0.13552632288091992, -0.07087384641293787, -0.23347557129430596, -0.3267961204686449, -0.35048709721903715, -0.32490482400537835, -0.28095706671420967, -0.2384775527300506, -0.20669879505121574, -0.18081789608019908, -0.15214982337384217, -0.1286828159414684, -0.09610772290967913, -0.06868392419231345, -0.0479543195357203, -0.0275979970758245, -0.017323848995297098, -0.007049700914769641]]}
Baseline removal of a peak on a sloping baseline
admin
dsp spectra_prep "[[0.0, 0.05, 0.1, 0.15, 0.201, 0.258, 0.357, 0.621, 1.222, 2.051, 2.5, 2.151, 1.422, 0.921, 0.757, 0.758, 0.801, 0.85, 0.9, 0.95, 1.0]]" "[{\"step\": \"als\", \"lambda\": 1000, \"p\": 0.01}]"
mentdb
{"steps": ["als"], "spectra": [[-0.005108578120476129, -0.0061752744658106, -0.007236913318805799, -0.008282323665401325, -0.008293169947351103, -0.0022429171059804576, 0.04590318015563327, 0.2591820876223476, 0.8096303120472187, 1.5882817683624266, 1.9871622751970306, 1.5882817683624069, 0.8096303120471796, 0.25918208762228945, 0.04590318015555661, -0.0022429171060748265, -0.00829316994746232, -0.008282323665528724, -0.007236913318948712, -0.006175274465968883, -0.005108578120649243]], "baselines": [[0.005108578120476129, 0.0561752744658106, 0.1072369133188058, 0.15828232366540132, 0.20929316994735112, 0.26024291710598046, 0.3110968198443667, 0.3618179123776524, 0.4123696879527813, 0.46271823163757353, 0.5128377248029694, 0.562718231637593, 0.6123696879528203, 0.6618179123777106, 0.7110968198444434, 0.7602429171060748, 0.8092931699474624, 0.8582823236655287, 0.9072369133189487, 0.9561752744659688, 1.0051085781206492]]}
Used in use cases (1 recipe)

    Chemist › Near infrared calibration: protein content by PLS
Often used with

    ml metrics rmse  ·  stat pls