ZeroGuess Benchmark (lmfit for wavelet function)

How This Benchmark Works

This benchmark evaluates different parameter estimation and curve fitting approaches using synthetic data with known ground truth values.

Methodology:

  1. Test Data Generation: 50 different parameter sets are randomly generated for the wavelet function. For each parameter set, synthetic data is created with 5% noise added.
  2. Methods Compared:
    • Simple + lmfit: Uses central values from parameter ranges as starting points
    • ZeroGuess + lmfit: Uses ZeroGuess's neural network to estimate starting parameters
    • True + lmfit: Uses the true parameters as starting points (best case scenario)
    Each method is tested with different optimization algorithms (least_squares, dual_annealing)
  3. Metrics:
    • Fit Success: Whether the fitting algorithm converged
    • Parameter Success: Whether all recovered parameters are within 10% of true values
    • Computation Time: Time for parameter estimation and fitting
    • Function Evaluations: Number of function calls required

Summary for wavelet function

Success and Performance Metrics

Method Fit Success Parameter Success Estimation Time (s) Fitting Time (s) Function Evaluations
Simple + lmfit (least_squares) 100.0% 16.0% 0.0004 0.0159 110.4
ZeroGuess + lmfit (least_squares) 100.0% 82.0% 0.0011 0.0082 50.8
True + lmfit (dual_annealing) 100.0% 88.0% 0.0004 0.7916 8264.5
Simple + lmfit (dual_annealing) 100.0% 88.0% 0.0004 0.7882 8261.7
ZeroGuess + lmfit (dual_annealing) 100.0% 90.0% 0.0011 0.7891 8271.8
True + lmfit (least_squares) 100.0% 92.0% 0.0004 0.0117 26.2

Parameter Error Summary

Method frequencyphasepositionwidth
Simple + lmfit (least_squares) 33.0%19.4%23.2%40.2%
ZeroGuess + lmfit (least_squares) 4.4%4.1%1.1%4.1%
True + lmfit (dual_annealing) 4.3%2.5%0.5%2.1%
Simple + lmfit (dual_annealing) 2.7%1.9%0.3%1.8%
ZeroGuess + lmfit (dual_annealing) 5.5%2.5%0.3%1.9%
True + lmfit (least_squares) 3.5%1.3%0.3%1.9%

Visualization

Summary Chart

Sample Fits

Simple + lmfit (least_squares)

Links to all fits: [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18] [19] [20] [21] [22] [23] [24] [25] [26] [27] [28] [29] [30] [31] [32] [33] [34] [35] [36] [37] [38] [39] [40] [41] [42] [43] [44] [45] [46] [47] [48] [49] [50]
Simple + lmfit (least_squares) - Set 1
Simple + lmfit (least_squares) - Set 2
Simple + lmfit (least_squares) - Set 3

ZeroGuess + lmfit (least_squares)

Links to all fits: [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18] [19] [20] [21] [22] [23] [24] [25] [26] [27] [28] [29] [30] [31] [32] [33] [34] [35] [36] [37] [38] [39] [40] [41] [42] [43] [44] [45] [46] [47] [48] [49] [50]
ZeroGuess + lmfit (least_squares) - Set 1
ZeroGuess + lmfit (least_squares) - Set 2
ZeroGuess + lmfit (least_squares) - Set 3

True + lmfit (least_squares)

Links to all fits: [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18] [19] [20] [21] [22] [23] [24] [25] [26] [27] [28] [29] [30] [31] [32] [33] [34] [35] [36] [37] [38] [39] [40] [41] [42] [43] [44] [45] [46] [47] [48] [49] [50]
True + lmfit (least_squares) - Set 1
True + lmfit (least_squares) - Set 2
True + lmfit (least_squares) - Set 3

Simple + lmfit (dual_annealing)

Links to all fits: [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18] [19] [20] [21] [22] [23] [24] [25] [26] [27] [28] [29] [30] [31] [32] [33] [34] [35] [36] [37] [38] [39] [40] [41] [42] [43] [44] [45] [46] [47] [48] [49] [50]
Simple + lmfit (dual_annealing) - Set 1
Simple + lmfit (dual_annealing) - Set 2
Simple + lmfit (dual_annealing) - Set 3

ZeroGuess + lmfit (dual_annealing)

Links to all fits: [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18] [19] [20] [21] [22] [23] [24] [25] [26] [27] [28] [29] [30] [31] [32] [33] [34] [35] [36] [37] [38] [39] [40] [41] [42] [43] [44] [45] [46] [47] [48] [49] [50]
ZeroGuess + lmfit (dual_annealing) - Set 1
ZeroGuess + lmfit (dual_annealing) - Set 2
ZeroGuess + lmfit (dual_annealing) - Set 3

True + lmfit (dual_annealing)

Links to all fits: [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18] [19] [20] [21] [22] [23] [24] [25] [26] [27] [28] [29] [30] [31] [32] [33] [34] [35] [36] [37] [38] [39] [40] [41] [42] [43] [44] [45] [46] [47] [48] [49] [50]
True + lmfit (dual_annealing) - Set 1
True + lmfit (dual_annealing) - Set 2
True + lmfit (dual_annealing) - Set 3