Bayesian geolift with CausalPy#

This notebook covers how to use CausalPy’s Bayesian synthetic control functionality to assess ‘geolift’. Our hypothetical scenario is:

We are a data scientist within a company that operates over Europe. We have been given a historical dataset of sales volumes, in units of 1000’s. The data is broken down by country and was collected at weekly frequency. We have data for the past 4 years.

At the start of 2022, the marketing department decided to refurbish all the stores in Denmark. Now, at the end of 2022, the company wants you to assess whether this refurbishment programme increased sales. If you tell them that the store refurbishment scheme increased sales volumes then they will roll out the scheme to other countries. Nobody said this, but in the back of your mind you worry that if you tell them that refurbishments increase sales but that doesn’t actually happen in the future, then the companies profits will drop, the value of your shares will decrease, and your job security may be at risk.

Your boss is pretty tuned in. She also has these concerns. She knows that while it might be easy to establish an association between the store refurbishments and changes in sales volumes, we really want to know if the store refurbishments caused an increase in sales.

We know that the best way to make causal claims is to run a randomized control trial (sometimes known as an A/B test). If we have randomly assigned stores across Europe (or picked a country) at random, then perhaps an A/B test would do the job. But we did not pick Denmark at random - so we are worried about confounding variables.

But we heard about synthetic control methods and a thing called GeoLift. After some research, we decide this is exactly what we want to do. But we are particularly interested in how certain we are in the level of any uplift we detect, so we want to use Bayesian methods and get easy to interpret Bayesian credible intervals. You find a library called CausalPy that supports exactly that use case and are delighted.

Let’s go!

import arviz as az
import pandas as pd

import causalpy as cp
%load_ext autoreload
%autoreload 2
%config InlineBackend.figure_format = 'retina'
pd.set_option("display.precision", 2)
seed = 42

Load the dataset#

CausalPy includes an example (simulated) dataset suitable to explore ideas around geographical lift testing. All we need to do is to load that, get our observation dates set up appropriately in a pandas dataframe, and define the treatment time.

df = (
    cp.load_data("geolift1")
    .assign(time=lambda x: pd.to_datetime(x["time"]))
    .set_index("time")
)

treatment_time = pd.to_datetime("2022-01-01")
df.head()
Austria Belgium Bulgaria Croatia Cyprus Czech_Republic Estonia Finland Greece Hungary Denmark
time
2019-01-06 1.63 1.75 1.51 1.38 1.97 1.21 2.06 1.69 2.98 4.02 1.71
2019-01-13 1.48 1.65 1.37 1.46 1.86 1.53 1.97 1.82 2.98 4.19 1.77
2019-01-20 1.73 1.66 1.47 1.31 2.30 1.35 1.80 1.70 2.81 4.03 1.93
2019-01-27 1.77 1.57 1.26 1.23 2.01 1.31 1.96 1.59 2.87 4.11 1.80
2019-02-03 1.70 1.73 1.29 1.30 2.07 1.28 1.90 1.86 2.85 4.23 1.64

In our dataset, columns represent the different European countries that we operate in. We also have an index which labels each row with the date - the observations are weekly in frequency. The values in the table represent the sales volumes and are in units of 1000’s. So a value of 2.4 represents 2,400 units sold per week.

So let’s plot that.

untreated = list(set(df.columns).difference({"Denmark"}))
ax = df[untreated].plot(color=[0.7, 0.7, 0.7])
df["Denmark"].plot(color="r", ax=ax)
ax.axvline(treatment_time, color="k", linestyle="--")
ax.set(title="Observed data", ylabel="Sales volume (thousands)");
../_images/718e6785882265d815ac5c8f51715757f07539fe5c456ebedaf909a2260f33cb.png

This looks pretty nice, but also disappointing. It is not immediately obvious from a visual inspection that there is a clear change in sales data in Denmark after the stores were refurbished (shown by the dashed line). This is made worse by the presence of annual seasonality in the data (which is different in each country), and the inherent stochasticisty in the weekly sales data.

Pre-experiment diagnostics: market correlations#

Before fitting a model, it is worth checking how similar the candidate control markets are to each other and to the treated market. Markets that move together in the pre-treatment period are more likely to produce a reliable synthetic control – the weighted combination of controls will more easily replicate the treated unit’s trajectory.

CausalPy provides plot_correlations() to visualise the pairwise Pearson correlations between all locations in the panel data. High positive correlations between the treated market and several controls suggest that the control pool can plausibly reconstruct the treated unit’s counterfactual.

pre_treatment = df.loc[df.index < treatment_time]
corr, ax = cp.plot_correlations(pre_treatment)
ax.set(title="Pairwise correlation of pre-treatment sales");
../_images/f1b9236c6b1a486dc5c4bf8f47b829f517e3d82576ca46e626b36ef22d60b796.png

The heatmap reveals that most countries are strongly positively correlated with Denmark, but two countries – Greece and Hungary – are negatively correlated with the treated unit. This is a sign that those markets are driven by different underlying factors and would be poor donors for the synthetic control.

Donor pool selection#

The synthetic control method works by reconstructing the treated unit as a weighted combination of control units. Including controls that move in the opposite direction to the treated unit can introduce interpolation bias and push the synthetic control outside the convex hull condition of the donor pool [Abadie et al., 2010, Abadie, 2021]. Abadie and L'Hour [2021] further show that dissimilar donors amplify estimation error in disaggregated settings.

Practical guideline

Before fitting a synthetic control model, inspect the pre-treatment correlation heatmap and exclude any control units that are negatively correlated (or only weakly correlated) with the treated unit. This is especially important for Bayesian implementations that use a Dirichlet prior, since all donors receive non-zero weight by construction.

denmark_corr = corr["Denmark"].drop("Denmark")
good_donors = list(denmark_corr[denmark_corr > 0].index)
excluded = list(denmark_corr[denmark_corr <= 0].index)
print(f"Keeping {len(good_donors)} donors: {good_donors}")
print(f"Excluding {len(excluded)} units: {excluded}")
Keeping 9 donors: ['Austria', 'Belgium', 'Bulgaria', 'Croatia', 'Cyprus', 'Czech_Republic', 'Estonia', 'Finland', 'Hungary']
Excluding 1 units: ['Greece']

With the contrarian markets removed, the remaining donor pool consists of countries that co-move with Denmark during the pre-treatment period. We can now proceed to fit the synthetic control model using these curated donors.

Was there a geolift in sales in Denmark?#

In order to calculate what (if any) causal effect there is from the store refurbishment we need to compare the actual sales in Denmark after the intervention and the counterfactual sales in Denmark if the intervention had not taken place. We can see why this is called the counterfactual - we did refurbish the stores in Denmark, so this is a completely hypothetical scenario that runs counter to the facts. But if we could measure (or more realistically estimate) this, that would be our control group.

In this case, we generate a synthetic control, which is the name of the technique we will be using to estimate our counterfactual sales data in Denmark if the refurbishment had not taken place. You can read more about the synthetic control method on the synthetic control wikipedia page, but the basic idea is as follows. For those familiar with traditional (non-Bayesian) modelling methods, the basic synthetic control algorithm can be described like this:

import my_custom_scikit_learn_model as weighted_combination


# fit the model to pre-intervention (training) data
weighted_combination.fit(X_train, y_train)
# estimate the counterfactual given post-intervention (test) data
counterfactual = weighted_combination.predict(X_test)
# estimate the causal impact
causal_impact = y_test - counterfactual
# cumulative causal impact
np.cumsum(causal_impact)

So there is no magic involved - we simply estimate a synthetic Denmark as a weighted sum of the untreated units. We do this based on the ‘training’ data observed before the intervention. We then use that weighted sum model to predict our synthetic Denmark based on ‘test’ data of untreated countries observed after the intervention. We can then simply compare this counterfactual estimate with the observed sales data in Denmark and obtain our estimate of the causal impact. We can then (optionally) calculate the cumulative causal impact to answer the question “How many more sales were caused by the store refurbishment over 2022?”

We can use CausalPy’s API to run this procedure, but using Bayesian inference methods as follows:

Note

The random_seed keyword argument for the PyMC sampler is not necessary. We use it here so that the results are reproducible.

result = cp.SyntheticControl(
    df,
    treatment_time,
    treated_units=["Denmark"],
    control_units=good_donors,
    model=cp.pymc_models.WeightedSumFitter(
        sample_kwargs={"target_accept": 0.95, "random_seed": seed}
    ),
)
Initializing NUTS using jitter+adapt_diag...
Multiprocess sampling (4 chains in 4 jobs)
NUTS: [beta, y_hat_sigma]
?25l                                                                                
                               Step       Grad                                  
  Progress   Draw   Diverge…   size       evals      Speed   Elapsed   Remain…  
 ────────────────────────────────────────────────────────────────────────────── 
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━━━━━   0      0         0.000     0          0.00      0:00:00   -:--:--   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━━━━━   0      0         0.000     0          0.00      0:00:00   -:--:--   
                                                  draws/s                       
  ━━━━━━━   0      0         0.000     0          0.00      0:00:00   -:--:--   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━━━━━   0      0         0.000     0          0.00      0:00:00   -:--:--   
                                                  draws/s                       
  ━━━━━━━   0      0         0.000     0          0.00      0:00:00   -:--:--   
                                                  draws/s                       
  ━━━━━━━   0      0         0.000     0          0.00      0:00:00   -:--:--   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━━━━━   0      0         0.000     0          0.00      0:00:00   -:--:--   
                                                  draws/s                       
  ━━━━━━━   0      0         0.000     0          0.00      0:00:00   -:--:--   
                                                  draws/s                       
  ━━━━━━━   0      0         0.000     0          0.00      0:00:00   -:--:--   
                                                  draws/s                       
  ━━━━━━━   0      0         0.000     0          0.00      0:00:00   -:--:--   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━━━━━   16     0         0.029     127        158.83    0:00:00   0:00:13   
                                                  draws/s                       
  ━━━━━━━   11     0         0.003     255        115.95    0:00:00   0:00:18   
                                                  draws/s                       
  ━━━━━━━   19     0         0.003     255        194.99    0:00:00   0:00:11   
                                                  draws/s                       
  ━━━━━━━   10     0         0.009     255        108.83    0:00:00   0:00:20   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━━━━━   39     0         0.026     63         181.59    0:00:00   0:00:11   
                                                  draws/s                       
  ━━━━━━━   24     0         0.027     127        111.92    0:00:00   0:00:18   
                                                  draws/s                       
  ━━━━━━━   28     0         0.015     255        137.58    0:00:00   0:00:15   
                                                  draws/s                       
  ━━━━━━━   24     0         0.033     63         114.03    0:00:00   0:00:18   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━━━━━   64     0         0.025     127        194.98    0:00:00   0:00:10   
                                                  draws/s                       
  ━━━━━━━   47     0         0.011     255        143.59    0:00:00   0:00:14   
                                                  draws/s                       
  ━━━━━━━   45     0         0.018     127        138.75    0:00:00   0:00:15   
                                                  draws/s                       
  ━━━━━━━   41     0         0.020     63         129.24    0:00:00   0:00:16   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━━━━━   82     0         0.027     127        189.45    0:00:00   0:00:11   
                                                  draws/s                       
  ━━━━━━━   61     0         0.027     127        140.28    0:00:00   0:00:14   
                                                  draws/s                       
  ━━━━━━━   64     0         0.023     127        148.30    0:00:00   0:00:14   
                                                  draws/s                       
  ━━━━━━━   61     0         0.023     127        141.79    0:00:00   0:00:14   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━━━━━   97     0         0.016     127        177.36    0:00:00   0:00:11   
                                                  draws/s                       
  ━━━━━━━   79     0         0.023     127        144.43    0:00:00   0:00:14   
                                                  draws/s                       
  ━━━━━━━   82     0         0.022     127        150.69    0:00:00   0:00:13   
                                                  draws/s                       
  ━━━━━━━   74     0         0.014     255        138.82    0:00:00   0:00:14   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━━━━━   109    0         0.030     127        166.43    0:00:00   0:00:12   
                                                  draws/s                       
  ━━━━━━━   90     0         0.027     127        137.20    0:00:00   0:00:14   
                                                  draws/s                       
  ━━━━━━━   92     0         0.008     255        140.93    0:00:00   0:00:14   
                                                  draws/s                       
  ━━━━━━━   86     0         0.031     63         131.95    0:00:00   0:00:15   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━━━━━   122    0         0.026     127        158.96    0:00:00   0:00:12   
                                                  draws/s                       
  ━━━━━━━   107    0         0.026     255        139.26    0:00:00   0:00:14   
                                                  draws/s                       
  ━━━━━━━   104    0         0.024     127        136.21    0:00:00   0:00:14   
                                                  draws/s                       
  ━━━━━━━   110    0         0.049     127        143.94    0:00:00   0:00:14   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━━━━━   137    0         0.029     127        155.72    0:00:00   0:00:12   
                                                  draws/s                       
  ━━━━━━━   118    0         0.033     127        133.83    0:00:00   0:00:15   
                                                  draws/s                       
  ━━━━━━━   120    0         0.031     127        136.45    0:00:00   0:00:14   
                                                  draws/s                       
  ━━━━━━━   124    0         0.044     127        141.62    0:00:00   0:00:14   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ╸━━━━━━   157    0         0.036     63         158.04    0:00:00   0:00:12   
                                                  draws/s                       
  ━━━━━━━   134    0         0.014     255        135.74    0:00:00   0:00:14   
                                                  draws/s                       
  ━━━━━━━   132    0         0.016     255        133.76    0:00:00   0:00:14   
                                                  draws/s                       
  ━━━━━━━   140    0         0.033     255        141.71    0:00:00   0:00:14   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ╸━━━━━━   179    0         0.015     255        161.99    0:00:01   0:00:12   
                                                  draws/s                       
  ╸━━━━━━   145    0         0.026     63         131.95    0:00:01   0:00:15   
                                                  draws/s                       
  ━━━━━━━   141    0         0.017     191        128.21    0:00:01   0:00:15   
                                                  draws/s                       
  ╸━━━━━━   156    0         0.041     127        142.26    0:00:01   0:00:13   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ╸━━━━━━   191    0         0.031     127        157.78    0:00:01   0:00:12   
                                                  draws/s                       
  ╸━━━━━━   163    0         0.030     127        134.97    0:00:01   0:00:14   
                                                  draws/s                       
  ╸━━━━━━   154    0         0.030     63         128.10    0:00:01   0:00:15   
                                                  draws/s                       
  ╸━━━━━━   177    0         0.053     63         146.97    0:00:01   0:00:13   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ╸━━━━━━   206    0         0.019     255        156.80    0:00:01   0:00:12   
                                                  draws/s                       
  ╸━━━━━━   187    0         0.052     63         141.87    0:00:01   0:00:13   
                                                  draws/s                       
  ╸━━━━━━   171    0         0.028     127        130.00    0:00:01   0:00:15   
                                                  draws/s                       
  ╸━━━━━━   194    0         0.034     95         147.63    0:00:01   0:00:13   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ╸━━━━━━   219    0         0.018     255        153.53    0:00:01   0:00:12   
                                                  draws/s                       
  ╸━━━━━━   204    0         0.018     255        143.84    0:00:01   0:00:13   
                                                  draws/s                       
  ╸━━━━━━   183    0         0.021     127        128.82    0:00:01   0:00:15   
                                                  draws/s                       
  ╸━━━━━━   210    0         0.037     127        148.00    0:00:01   0:00:13   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ╸━━━━━━   235    0         0.038     63         152.81    0:00:01   0:00:12   
                                                  draws/s                       
  ╸━━━━━━   215    0         0.024     127        140.27    0:00:01   0:00:13   
                                                  draws/s                       
  ╸━━━━━━   199    0         0.016     127        130.05    0:00:01   0:00:14   
                                                  draws/s                       
  ╸━━━━━━   227    0         0.037     63         148.31    0:00:01   0:00:12   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ╸━━━━━━   250    0         0.028     127        152.04    0:00:01   0:00:12   
                                                  draws/s                       
  ╸━━━━━━   229    0         0.012     127        139.45    0:00:01   0:00:13   
                                                  draws/s                       
  ╸━━━━━━   210    0         0.029     127        127.81    0:00:01   0:00:15   
                                                  draws/s                       
  ╸━━━━━━   241    0         0.024     255        146.85    0:00:01   0:00:12   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ╸━━━━━━   266    0         0.043     127        151.49    0:00:01   0:00:12   
                                                  draws/s                       
  ╸━━━━━━   236    0         0.019     127        134.99    0:00:01   0:00:14   
                                                  draws/s                       
  ╸━━━━━━   229    0         0.038     127        130.52    0:00:01   0:00:14   
                                                  draws/s                       
  ╸━━━━━━   254    0         0.027     127        145.46    0:00:01   0:00:13   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ╸━━━━━━   283    0         0.030     255        151.59    0:00:01   0:00:12   
                                                  draws/s                       
  ╸━━━━━━   254    0         0.031     63         135.94    0:00:01   0:00:13   
                                                  draws/s                       
  ╸━━━━━━   245    0         0.020     127        131.48    0:00:01   0:00:14   
                                                  draws/s                       
  ╸━━━━━━   278    0         0.031     63         149.15    0:00:01   0:00:12   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━╺━━━━━   293    0         0.014     255        149.34    0:00:01   0:00:12   
                                                  draws/s                       
  ╸━━━━━━   271    0         0.030     127        136.92    0:00:01   0:00:13   
                                                  draws/s                       
  ╸━━━━━━   261    0         0.036     63         132.19    0:00:01   0:00:14   
                                                  draws/s                       
  ━╺━━━━━   292    0         0.020     127        147.99    0:00:01   0:00:12   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━╺━━━━━   302    0         0.023     127        144.99    0:00:02   0:00:12   
                                                  draws/s                       
  ━╺━━━━━   287    0         0.036     127        137.78    0:00:02   0:00:13   
                                                  draws/s                       
  ╸━━━━━━   277    0         0.036     127        133.27    0:00:02   0:00:13   
                                                  draws/s                       
  ━╺━━━━━   302    0         0.026     255        145.49    0:00:02   0:00:12   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━╺━━━━━   314    0         0.025     255        143.46    0:00:02   0:00:12   
                                                  draws/s                       
  ━╺━━━━━   311    0         0.042     15         141.95    0:00:02   0:00:12   
                                                  draws/s                       
  ━╺━━━━━   297    0         0.040     63         135.60    0:00:02   0:00:13   
                                                  draws/s                       
  ━╺━━━━━   315    0         0.025     255        144.35    0:00:02   0:00:12   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━╺━━━━━   332    0         0.030     63         144.26    0:00:02   0:00:12   
                                                  draws/s                       
  ━╺━━━━━   335    0         0.036     127        145.71    0:00:02   0:00:12   
                                                  draws/s                       
  ━╺━━━━━   314    0         0.024     255        137.00    0:00:02   0:00:13   
                                                  draws/s                       
  ━╺━━━━━   334    0         0.024     127        145.59    0:00:02   0:00:12   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━╺━━━━━   347    0         0.032     127        143.75    0:00:02   0:00:12   
                                                  draws/s                       
  ━╺━━━━━   352    0         0.031     127        145.96    0:00:02   0:00:12   
                                                  draws/s                       
  ━╺━━━━━   327    0         0.020     255        136.24    0:00:02   0:00:13   
                                                  draws/s                       
  ━╺━━━━━   346    0         0.021     127        143.95    0:00:02   0:00:12   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━╺━━━━━   366    0         0.031     127        145.02    0:00:02   0:00:12   
                                                  draws/s                       
  ━╺━━━━━   370    0         0.031     63         146.65    0:00:02   0:00:12   
                                                  draws/s                       
  ━╺━━━━━   341    0         0.025     127        135.07    0:00:02   0:00:13   
                                                  draws/s                       
  ━╺━━━━━   359    0         0.025     127        142.67    0:00:02   0:00:12   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━╺━━━━━   382    0         0.024     159        144.87    0:00:02   0:00:12   
                                                  draws/s                       
  ━╺━━━━━   389    0         0.021     127        147.74    0:00:02   0:00:11   
                                                  draws/s                       
  ━╺━━━━━   353    0         0.035     127        134.25    0:00:02   0:00:13   
                                                  draws/s                       
  ━╺━━━━━   376    0         0.030     127        143.05    0:00:02   0:00:12   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━╺━━━━━   397    0         0.033     127        144.20    0:00:02   0:00:12   
                                                  draws/s                       
  ━╺━━━━━   407    0         0.037     127        147.79    0:00:02   0:00:11   
                                                  draws/s                       
  ━╺━━━━━   373    0         0.026     127        135.61    0:00:02   0:00:13   
                                                  draws/s                       
  ━╺━━━━━   397    0         0.034     31         144.32    0:00:02   0:00:12   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━╺━━━━━   420    0         0.035     63         146.58    0:00:02   0:00:11   
                                                  draws/s                       
  ━╺━━━━━   425    0         0.019     127        148.33    0:00:02   0:00:11   
                                                  draws/s                       
  ━╺━━━━━   392    0         0.037     127        136.99    0:00:02   0:00:12   
                                                  draws/s                       
  ━╺━━━━━   412    0         0.029     127        144.02    0:00:02   0:00:12   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━╸━━━━━   438    0         0.023     255        147.35    0:00:02   0:00:11   
                                                  draws/s                       
  ━╸━━━━━   439    0         0.031     127        147.47    0:00:02   0:00:11   
                                                  draws/s                       
  ━╺━━━━━   405    0         0.028     127        136.43    0:00:02   0:00:12   
                                                  draws/s                       
  ━╸━━━━━   432    0         0.029     127        145.30    0:00:02   0:00:11   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━╸━━━━━   452    0         0.035     127        146.21    0:00:03   0:00:11   
                                                  draws/s                       
  ━╸━━━━━   457    0         0.030     127        148.17    0:00:03   0:00:11   
                                                  draws/s                       
  ━╺━━━━━   423    0         0.027     127        136.99    0:00:03   0:00:12   
                                                  draws/s                       
  ━╸━━━━━   447    0         0.024     127        145.01    0:00:03   0:00:11   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━╸━━━━━   468    0         0.015     511        146.34    0:00:03   0:00:11   
                                                  draws/s                       
  ━╸━━━━━   478    0         0.030     127        149.29    0:00:03   0:00:11   
                                                  draws/s                       
  ━╸━━━━━   441    0         0.034     63         137.89    0:00:03   0:00:12   
                                                  draws/s                       
  ━╸━━━━━   459    0         0.018     255        143.70    0:00:03   0:00:11   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━╸━━━━━   473    0         0.019     255        142.92    0:00:03   0:00:11   
                                                  draws/s                       
  ━╸━━━━━   488    0         0.020     127        147.27    0:00:03   0:00:11   
                                                  draws/s                       
  ━╸━━━━━   453    0         0.035     127        136.79    0:00:03   0:00:12   
                                                  draws/s                       
  ━╸━━━━━   467    0         0.025     127        141.07    0:00:03   0:00:11   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━╸━━━━━   485    0         0.030     127        141.52    0:00:03   0:00:11   
                                                  draws/s                       
  ━╸━━━━━   501    0         0.030     63         146.29    0:00:03   0:00:11   
                                                  draws/s                       
  ━╸━━━━━   465    0         0.023     255        135.71    0:00:03   0:00:12   
                                                  draws/s                       
  ━╸━━━━━   486    0         0.040     127        141.90    0:00:03   0:00:11   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━╸━━━━━   501    0         0.032     127        141.64    0:00:03   0:00:11   
                                                  draws/s                       
  ━╸━━━━━   516    0         0.029     127        146.15    0:00:03   0:00:11   
                                                  draws/s                       
  ━╸━━━━━   474    0         0.023     127        134.37    0:00:03   0:00:12   
                                                  draws/s                       
  ━╸━━━━━   503    0         0.029     63         142.46    0:00:03   0:00:11   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━╸━━━━━   519    0         0.036     127        142.36    0:00:03   0:00:11   
                                                  draws/s                       
  ━╸━━━━━   535    0         0.031     127        146.79    0:00:03   0:00:10   
                                                  draws/s                       
  ━╸━━━━━   491    0         0.028     127        134.73    0:00:03   0:00:12   
                                                  draws/s                       
  ━╸━━━━━   523    0         0.035     31         143.63    0:00:03   0:00:11   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━╸━━━━━   537    0         0.036     127        143.10    0:00:03   0:00:11   
                                                  draws/s                       
  ━╸━━━━━   551    0         0.027     127        146.88    0:00:03   0:00:10   
                                                  draws/s                       
  ━╸━━━━━   501    0         0.031     159        133.63    0:00:03   0:00:12   
                                                  draws/s                       
  ━╸━━━━━   540    0         0.038     119        144.29    0:00:03   0:00:11   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━╸━━━━━   557    0         0.033     127        144.33    0:00:03   0:00:10   
                                                  draws/s                       
  ━╸━━━━━   563    0         0.024     127        145.87    0:00:03   0:00:10   
                                                  draws/s                       
  ━╸━━━━━   514    0         0.022     127        133.28    0:00:03   0:00:12   
                                                  draws/s                       
  ━╸━━━━━   558    0         0.038     127        144.69    0:00:03   0:00:10   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━╺━━━━   572    0         0.022     127        143.78    0:00:03   0:00:10   
                                                  draws/s                       
  ━━╺━━━━   580    0         0.022     127        145.97    0:00:03   0:00:10   
                                                  draws/s                       
  ━╸━━━━━   534    0         0.036     63         134.30    0:00:03   0:00:11   
                                                  draws/s                       
  ━━╺━━━━   572    0         0.025     255        144.12    0:00:03   0:00:10   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━╺━━━━   586    0         0.032     127        143.39    0:00:04   0:00:10   
                                                  draws/s                       
  ━━╺━━━━   593    0         0.028     127        145.15    0:00:04   0:00:10   
                                                  draws/s                       
  ━╸━━━━━   552    0         0.036     127        135.23    0:00:04   0:00:11   
                                                  draws/s                       
  ━━╺━━━━   583    0         0.024     255        143.17    0:00:04   0:00:10   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━╺━━━━   605    0         0.021     63         144.31    0:00:04   0:00:10   
                                                  draws/s                       
  ━━╺━━━━   609    0         0.026     127        145.21    0:00:04   0:00:10   
                                                  draws/s                       
  ━━╺━━━━   572    0         0.029     127        136.33    0:00:04   0:00:11   
                                                  draws/s                       
  ━━╺━━━━   600    0         0.028     127        143.04    0:00:04   0:00:10   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━╺━━━━   622    0         0.031     127        144.22    0:00:04   0:00:10   
                                                  draws/s                       
  ━━╺━━━━   626    0         0.028     127        145.28    0:00:04   0:00:10   
                                                  draws/s                       
  ━━╺━━━━   592    0         0.035     127        137.34    0:00:04   0:00:11   
                                                  draws/s                       
  ━━╺━━━━   614    0         0.030     127        142.62    0:00:04   0:00:10   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━╺━━━━   636    0         0.022     191        144.12    0:00:04   0:00:10   
                                                  draws/s                       
  ━━╺━━━━   642    0         0.033     127        145.23    0:00:04   0:00:10   
                                                  draws/s                       
  ━━╺━━━━   608    0         0.035     127        137.64    0:00:04   0:00:11   
                                                  draws/s                       
  ━━╺━━━━   628    0         0.036     63         142.45    0:00:04   0:00:10   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━╺━━━━   648    0         0.029     127        143.15    0:00:04   0:00:10   
                                                  draws/s                       
  ━━╺━━━━   658    0         0.028     127        145.30    0:00:04   0:00:10   
                                                  draws/s                       
  ━━╺━━━━   626    0         0.039     63         138.28    0:00:04   0:00:10   
                                                  draws/s                       
  ━━╺━━━━   646    0         0.029     127        142.81    0:00:04   0:00:10   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━╺━━━━   661    0         0.026     127        142.51    0:00:04   0:00:10   
                                                  draws/s                       
  ━━╺━━━━   676    0         0.039     31         145.69    0:00:04   0:00:10   
                                                  draws/s                       
  ━━╺━━━━   641    0         0.024     255        138.20    0:00:04   0:00:10   
                                                  draws/s                       
  ━━╺━━━━   661    0         0.029     127        142.79    0:00:04   0:00:10   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━╺━━━━   679    0         0.026     63         142.96    0:00:04   0:00:10   
                                                  draws/s                       
  ━━╺━━━━   694    0         0.038     127        146.09    0:00:04   0:00:09   
                                                  draws/s                       
  ━━╺━━━━   654    0         0.027     191        137.78    0:00:04   0:00:10   
                                                  draws/s                       
  ━━╺━━━━   677    0         0.030     127        142.76    0:00:04   0:00:10   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━╺━━━━   692    0         0.022     127        142.20    0:00:04   0:00:10   
                                                  draws/s                       
  ━━╺━━━━   711    0         0.029     127        146.21    0:00:04   0:00:09   
                                                  draws/s                       
  ━━╺━━━━   670    0         0.028     127        137.78    0:00:04   0:00:10   
                                                  draws/s                       
  ━━╺━━━━   697    0         0.037     127        143.55    0:00:04   0:00:10   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━╺━━━━   706    0         0.033     127        141.94    0:00:04   0:00:10   
                                                  draws/s                       
  ━━╸━━━━   727    0         0.031     127        146.21    0:00:04   0:00:09   
                                                  draws/s                       
  ━━╺━━━━   681    0         0.025     255        137.09    0:00:04   0:00:10   
                                                  draws/s                       
  ━━╺━━━━   714    0         0.034     63         143.74    0:00:04   0:00:09   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━╸━━━━   723    0         0.034     63         142.18    0:00:05   0:00:09   
                                                  draws/s                       
  ━━╸━━━━   743    0         0.031     127        146.25    0:00:05   0:00:09   
                                                  draws/s                       
  ━━╺━━━━   695    0         0.030     127        136.89    0:00:05   0:00:10   
                                                  draws/s                       
  ━━╸━━━━   728    0         0.019     127        143.41    0:00:05   0:00:09   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━╸━━━━   740    0         0.026     127        142.49    0:00:05   0:00:09   
                                                  draws/s                       
  ━━╸━━━━   762    0         0.032     63         146.64    0:00:05   0:00:09   
                                                  draws/s                       
  ━━╺━━━━   713    0         0.025     111        137.40    0:00:05   0:00:10   
                                                  draws/s                       
  ━━╸━━━━   744    0         0.030     127        143.33    0:00:05   0:00:09   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━╸━━━━   756    0         0.036     127        142.56    0:00:05   0:00:09   
                                                  draws/s                       
  ━━╸━━━━   777    0         0.028     127        146.50    0:00:05   0:00:09   
                                                  draws/s                       
  ━━╸━━━━   730    0         0.028     63         137.81    0:00:05   0:00:10   
                                                  draws/s                       
  ━━╸━━━━   761    0         0.032     63         143.61    0:00:05   0:00:09   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━╸━━━━   771    0         0.027     127        142.49    0:00:05   0:00:09   
                                                  draws/s                       
  ━━╸━━━━   791    0         0.030     127        146.09    0:00:05   0:00:09   
                                                  draws/s                       
  ━━╸━━━━   744    0         0.018     127        137.52    0:00:05   0:00:10   
                                                  draws/s                       
  ━━╸━━━━   779    0         0.026     127        144.07    0:00:05   0:00:09   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━╸━━━━   789    0         0.036     127        142.65    0:00:05   0:00:09   
                                                  draws/s                       
  ━━╸━━━━   806    0         0.029     127        145.93    0:00:05   0:00:09   
                                                  draws/s                       
  ━━╸━━━━   757    0         0.025     127        137.02    0:00:05   0:00:10   
                                                  draws/s                       
  ━━╸━━━━   798    0         0.027     127        144.45    0:00:05   0:00:09   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━╸━━━━   801    0         0.036     127        142.08    0:00:05   0:00:09   
                                                  draws/s                       
  ━━╸━━━━   817    0         0.033     63         144.99    0:00:05   0:00:09   
                                                  draws/s                       
  ━━╸━━━━   767    0         0.028     127        136.20    0:00:05   0:00:10   
                                                  draws/s                       
  ━━╸━━━━   808    0         0.034     127        143.58    0:00:05   0:00:09   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━╸━━━━   817    0         0.035     127        142.21    0:00:05   0:00:09   
                                                  draws/s                       
  ━━╸━━━━   834    0         0.025     63         145.14    0:00:05   0:00:09   
                                                  draws/s                       
  ━━╸━━━━   786    0         0.027     127        136.98    0:00:05   0:00:09   
                                                  draws/s                       
  ━━╸━━━━   825    0         0.040     79         143.83    0:00:05   0:00:09   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━╸━━━━   829    0         0.024     127        141.73    0:00:05   0:00:09   
                                                  draws/s                       
  ━━╸━━━━   841    0         0.028     255        144.08    0:00:05   0:00:09   
                                                  draws/s                       
  ━━╸━━━━   799    0         0.037     127        136.64    0:00:05   0:00:09   
                                                  draws/s                       
  ━━╸━━━━   839    0         0.038     127        143.48    0:00:05   0:00:09   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━╸━━━━   844    0         0.034     127        141.52    0:00:05   0:00:09   
                                                  draws/s                       
  ━━╸━━━━   854    0         0.029     127        143.37    0:00:05   0:00:08   
                                                  draws/s                       
  ━━╸━━━━   819    0         0.028     127        137.55    0:00:05   0:00:09   
                                                  draws/s                       
  ━━╸━━━━   852    0         0.029     127        143.10    0:00:05   0:00:09   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━╸━━━━   857    0         0.025     127        141.21    0:00:06   0:00:09   
                                                  draws/s                       
  ━━━╺━━━   870    0         0.026     191        143.51    0:00:06   0:00:08   
                                                  draws/s                       
  ━━╸━━━━   837    0         0.040     127        138.06    0:00:06   0:00:09   
                                                  draws/s                       
  ━━━╺━━━   864    0         0.032     127        142.53    0:00:06   0:00:08   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━╺━━━   872    0         0.032     127        141.14    0:00:06   0:00:08   
                                                  draws/s                       
  ━━━╺━━━   885    0         0.032     127        143.38    0:00:06   0:00:08   
                                                  draws/s                       
  ━━╸━━━━   856    0         0.039     63         138.69    0:00:06   0:00:09   
                                                  draws/s                       
  ━━━╺━━━   879    0         0.028     127        142.54    0:00:06   0:00:08   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━╺━━━   893    0         0.029     63         142.00    0:00:06   0:00:08   
                                                  draws/s                       
  ━━━╺━━━   899    0         0.030     255        143.03    0:00:06   0:00:08   
                                                  draws/s                       
  ━━━╺━━━   871    0         0.028     127        138.65    0:00:06   0:00:09   
                                                  draws/s                       
  ━━━╺━━━   895    0         0.031     255        142.59    0:00:06   0:00:08   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━╺━━━   907    0         0.033     191        141.84    0:00:06   0:00:08   
                                                  draws/s                       
  ━━━╺━━━   915    0         0.032     127        143.19    0:00:06   0:00:08   
                                                  draws/s                       
  ━━━╺━━━   888    0         0.042     63         139.00    0:00:06   0:00:09   
                                                  draws/s                       
  ━━━╺━━━   904    0         0.017     255        141.70    0:00:06   0:00:08   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━╺━━━   921    0         0.030     255        141.66    0:00:06   0:00:08   
                                                  draws/s                       
  ━━━╺━━━   931    0         0.038     63         143.20    0:00:06   0:00:08   
                                                  draws/s                       
  ━━━╺━━━   905    0         0.035     127        139.23    0:00:06   0:00:08   
                                                  draws/s                       
  ━━━╺━━━   913    0         0.023     255        140.69    0:00:06   0:00:08   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━╺━━━   936    0         0.033     63         141.49    0:00:06   0:00:08   
                                                  draws/s                       
  ━━━╺━━━   947    0         0.034     127        143.08    0:00:06   0:00:08   
                                                  draws/s                       
  ━━━╺━━━   924    0         0.046     127        139.67    0:00:06   0:00:08   
                                                  draws/s                       
  ━━━╺━━━   928    0         0.027     63         140.30    0:00:06   0:00:08   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━╺━━━   951    0         0.024     63         141.57    0:00:06   0:00:08   
                                                  draws/s                       
  ━━━╺━━━   961    0         0.033     47         142.88    0:00:06   0:00:08   
                                                  draws/s                       
  ━━━╺━━━   941    0         0.040     63         140.02    0:00:06   0:00:08   
                                                  draws/s                       
  ━━━╺━━━   943    0         0.032     127        140.26    0:00:06   0:00:08   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━╺━━━   961    0         0.027     127        140.71    0:00:06   0:00:08   
                                                  draws/s                       
  ━━━╺━━━   972    0         0.023     255        142.13    0:00:06   0:00:08   
                                                  draws/s                       
  ━━━╺━━━   954    0         0.022     255        139.55    0:00:06   0:00:08   
                                                  draws/s                       
  ━━━╺━━━   957    0         0.025     63         140.13    0:00:06   0:00:08   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━╺━━━   977    0         0.028     127        140.64    0:00:06   0:00:08   
                                                  draws/s                       
  ━━━╺━━━   984    0         0.027     127        141.90    0:00:06   0:00:08   
                                                  draws/s                       
  ━━━╺━━━   964    0         0.027     127        138.88    0:00:06   0:00:08   
                                                  draws/s                       
  ━━━╺━━━   972    0         0.030     127        140.07    0:00:06   0:00:08   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━╺━━━   988    0         0.026     127        140.14    0:00:07   0:00:08   
                                                  draws/s                       
  ━━━╺━━━   995    0         0.028     127        141.08    0:00:07   0:00:08   
                                                  draws/s                       
  ━━━╺━━━   974    0         0.036     127        138.19    0:00:07   0:00:08   
                                                  draws/s                       
  ━━━╺━━━   984    0         0.036     127        139.60    0:00:07   0:00:08   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━╺━━━   994    0         0.029     127        138.84    0:00:07   0:00:08   
                                                  draws/s                       
  ━━━╸━━━   1002   0         0.032     127        139.90    0:00:07   0:00:08   
                                                  draws/s                       
  ━━━╺━━━   981    0         0.038     127        137.00    0:00:07   0:00:08   
                                                  draws/s                       
  ━━━╺━━━   993    0         0.031     63         138.75    0:00:07   0:00:08   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━╸━━━   1010   0         0.029     63         138.84    0:00:07   0:00:08   
                                                  draws/s                       
  ━━━╸━━━   1016   0         0.032     127        139.87    0:00:07   0:00:07   
                                                  draws/s                       
  ━━━╸━━━   1000   0         0.039     31         137.50    0:00:07   0:00:08   
                                                  draws/s                       
  ━━━╸━━━   1008   0         0.021     127        138.67    0:00:07   0:00:08   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━╸━━━   1024   0         0.029     127        138.67    0:00:07   0:00:08   
                                                  draws/s                       
  ━━━╸━━━   1029   0         0.032     127        139.37    0:00:07   0:00:07   
                                                  draws/s                       
  ━━━╸━━━   1011   0         0.039     127        136.91    0:00:07   0:00:08   
                                                  draws/s                       
  ━━━╸━━━   1023   0         0.021     127        138.67    0:00:07   0:00:08   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━╸━━━   1038   1         0.029     127        138.48    0:00:07   0:00:08   
                                                  draws/s                       
  ━━━╸━━━   1047   0         0.032     127        139.68    0:00:07   0:00:07   
                                                  draws/s                       
  ━━━╸━━━   1026   0         0.039     127        136.92    0:00:07   0:00:08   
                                                  draws/s                       
  ━━━╸━━━   1035   0         0.021     127        138.15    0:00:07   0:00:07   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━╸━━━   1050   1         0.029     127        138.01    0:00:07   0:00:07   
                                                  draws/s                       
  ━━━╸━━━   1062   0         0.032     127        139.73    0:00:07   0:00:07   
                                                  draws/s                       
  ━━━╸━━━   1046   0         0.039     127        137.52    0:00:07   0:00:07   
                                                  draws/s                       
  ━━━╸━━━   1054   0         0.021     127        138.64    0:00:07   0:00:07   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━╸━━━   1066   1         0.029     63         138.08    0:00:07   0:00:07   
                                                  draws/s                       
  ━━━╸━━━   1076   0         0.032     127        139.38    0:00:07   0:00:07   
                                                  draws/s                       
  ━━━╸━━━   1061   0         0.039     127        137.41    0:00:07   0:00:07   
                                                  draws/s                       
  ━━━╸━━━   1073   0         0.021     63         139.13    0:00:07   0:00:07   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━╸━━━   1082   1         0.029     127        138.11    0:00:07   0:00:07   
                                                  draws/s                       
  ━━━╸━━━   1091   0         0.032     127        139.32    0:00:07   0:00:07   
                                                  draws/s                       
  ━━━╸━━━   1076   0         0.039     127        137.43    0:00:07   0:00:07   
                                                  draws/s                       
  ━━━╸━━━   1090   0         0.021     127        139.25    0:00:07   0:00:07   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━╸━━━   1098   1         0.029     127        138.23    0:00:07   0:00:07   
                                                  draws/s                       
  ━━━╸━━━   1106   0         0.032     127        139.24    0:00:07   0:00:07   
                                                  draws/s                       
  ━━━╸━━━   1093   0         0.039     127        137.64    0:00:07   0:00:07   
                                                  draws/s                       
  ━━━╸━━━   1107   0         0.021     63         139.51    0:00:07   0:00:07   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━╸━━━   1112   1         0.029     63         138.05    0:00:08   0:00:07   
                                                  draws/s                       
  ━━━╸━━━   1120   0         0.032     127        139.15    0:00:08   0:00:07   
                                                  draws/s                       
  ━━━╸━━━   1108   0         0.039     63         137.68    0:00:08   0:00:07   
                                                  draws/s                       
  ━━━╸━━━   1121   0         0.021     103        139.20    0:00:08   0:00:07   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━╸━━━   1126   1         0.029     127        137.91    0:00:08   0:00:07   
                                                  draws/s                       
  ━━━╸━━━   1135   0         0.032     63         139.01    0:00:08   0:00:07   
                                                  draws/s                       
  ━━━╸━━━   1125   0         0.039     127        137.80    0:00:08   0:00:07   
                                                  draws/s                       
  ━━━╸━━━   1137   0         0.021     127        139.26    0:00:08   0:00:07   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━╸━━━   1141   1         0.029     127        137.81    0:00:08   0:00:07   
                                                  draws/s                       
  ━━━━╺━━   1150   0         0.032     255        138.86    0:00:08   0:00:07   
                                                  draws/s                       
  ━━━╸━━━   1140   0         0.039     127        137.76    0:00:08   0:00:07   
                                                  draws/s                       
  ━━━━╺━━   1155   0         0.021     127        139.59    0:00:08   0:00:07   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━━╺━━   1152   1         0.029     127        137.30    0:00:08   0:00:07   
                                                  draws/s                       
  ━━━━╺━━   1161   0         0.032     127        138.47    0:00:08   0:00:07   
                                                  draws/s                       
  ━━━━╺━━   1153   0         0.039     63         137.57    0:00:08   0:00:07   
                                                  draws/s                       
  ━━━━╺━━   1169   0         0.021     127        139.40    0:00:08   0:00:07   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━━╺━━   1166   1         0.029     127        137.16    0:00:08   0:00:07   
                                                  draws/s                       
  ━━━━╺━━   1171   0         0.032     255        137.75    0:00:08   0:00:07   
                                                  draws/s                       
  ━━━━╺━━   1167   0         0.039     127        137.27    0:00:08   0:00:07   
                                                  draws/s                       
  ━━━━╺━━   1184   0         0.021     63         139.34    0:00:08   0:00:06   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━━╺━━   1175   1         0.029     127        136.31    0:00:08   0:00:07   
                                                  draws/s                       
  ━━━━╺━━   1181   0         0.032     127        137.00    0:00:08   0:00:07   
                                                  draws/s                       
  ━━━━╺━━   1178   0         0.039     127        136.68    0:00:08   0:00:06   
                                                  draws/s                       
  ━━━━╺━━   1194   0         0.021     63         138.69    0:00:08   0:00:06   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━━╺━━   1191   1         0.029     63         136.41    0:00:08   0:00:07   
                                                  draws/s                       
  ━━━━╺━━   1196   0         0.032     127        137.03    0:00:08   0:00:06   
                                                  draws/s                       
  ━━━━╺━━   1194   0         0.039     127        136.84    0:00:08   0:00:06   
                                                  draws/s                       
  ━━━━╺━━   1209   0         0.021     127        138.61    0:00:08   0:00:06   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━━╺━━   1204   1         0.029     95         136.33    0:00:08   0:00:07   
                                                  draws/s                       
  ━━━━╺━━   1211   0         0.032     63         137.17    0:00:08   0:00:06   
                                                  draws/s                       
  ━━━━╺━━   1211   0         0.039     63         137.15    0:00:08   0:00:06   
                                                  draws/s                       
  ━━━━╺━━   1224   0         0.021     127        138.66    0:00:08   0:00:06   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━━╺━━   1221   1         0.029     127        136.50    0:00:08   0:00:06   
                                                  draws/s                       
  ━━━━╺━━   1226   0         0.032     127        137.04    0:00:08   0:00:06   
                                                  draws/s                       
  ━━━━╺━━   1225   0         0.039     127        136.95    0:00:08   0:00:06   
                                                  draws/s                       
  ━━━━╺━━   1240   0         0.021     127        138.66    0:00:08   0:00:06   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━━╺━━   1233   1         0.029     127        136.09    0:00:09   0:00:06   
                                                  draws/s                       
  ━━━━╺━━   1240   0         0.032     127        136.91    0:00:09   0:00:06   
                                                  draws/s                       
  ━━━━╺━━   1239   0         0.039     127        136.89    0:00:09   0:00:06   
                                                  draws/s                       
  ━━━━╺━━   1258   0         0.021     63         138.99    0:00:09   0:00:06   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━━╺━━   1249   1         0.029     127        136.27    0:00:09   0:00:06   
                                                  draws/s                       
  ━━━━╺━━   1255   0         0.032     127        136.90    0:00:09   0:00:06   
                                                  draws/s                       
  ━━━━╺━━   1254   0         0.039     127        136.82    0:00:09   0:00:06   
                                                  draws/s                       
  ━━━━╺━━   1275   0         0.021     127        139.14    0:00:09   0:00:06   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━━╺━━   1262   1         0.029     127        136.07    0:00:09   0:00:06   
                                                  draws/s                       
  ━━━━╺━━   1270   0         0.032     127        136.95    0:00:09   0:00:06   
                                                  draws/s                       
  ━━━━╺━━   1269   0         0.039     127        136.88    0:00:09   0:00:06   
                                                  draws/s                       
  ━━━━╸━━   1294   0         0.021     127        139.52    0:00:09   0:00:06   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━━╺━━   1275   1         0.029     127        135.84    0:00:09   0:00:06   
                                                  draws/s                       
  ━━━━╸━━   1286   0         0.032     127        137.04    0:00:09   0:00:06   
                                                  draws/s                       
  ━━━━╺━━   1284   0         0.039     127        136.85    0:00:09   0:00:06   
                                                  draws/s                       
  ━━━━╸━━   1308   0         0.021     127        139.44    0:00:09   0:00:06   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━━╸━━   1288   1         0.029     63         135.64    0:00:09   0:00:06   
                                                  draws/s                       
  ━━━━╸━━   1301   0         0.032     63         137.03    0:00:09   0:00:06   
                                                  draws/s                       
  ━━━━╸━━   1299   0         0.039     63         136.90    0:00:09   0:00:06   
                                                  draws/s                       
  ━━━━╸━━   1323   0         0.021     63         139.41    0:00:09   0:00:05   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━━╸━━   1301   1         0.029     127        135.53    0:00:09   0:00:06   
                                                  draws/s                       
  ━━━━╸━━   1316   0         0.032     127        137.02    0:00:09   0:00:06   
                                                  draws/s                       
  ━━━━╸━━   1313   0         0.039     127        136.80    0:00:09   0:00:06   
                                                  draws/s                       
  ━━━━╸━━   1337   0         0.021     127        139.27    0:00:09   0:00:05   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━━╸━━   1317   1         0.029     127        135.58    0:00:09   0:00:06   
                                                  draws/s                       
  ━━━━╸━━   1331   0         0.032     127        137.08    0:00:09   0:00:05   
                                                  draws/s                       
  ━━━━╸━━   1328   0         0.039     127        136.77    0:00:09   0:00:05   
                                                  draws/s                       
  ━━━━╸━━   1353   0         0.021     127        139.40    0:00:09   0:00:05   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━━╸━━   1331   1         0.029     127        135.56    0:00:09   0:00:06   
                                                  draws/s                       
  ━━━━╸━━   1342   0         0.032     127        136.70    0:00:09   0:00:05   
                                                  draws/s                       
  ━━━━╸━━   1342   0         0.039     255        136.66    0:00:09   0:00:05   
                                                  draws/s                       
  ━━━━╸━━   1366   0         0.021     127        139.22    0:00:09   0:00:05   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━━╸━━   1339   1         0.029     127        134.82    0:00:09   0:00:06   
                                                  draws/s                       
  ━━━━╸━━   1349   0         0.032     127        135.85    0:00:09   0:00:05   
                                                  draws/s                       
  ━━━━╸━━   1352   0         0.039     63         136.13    0:00:09   0:00:05   
                                                  draws/s                       
  ━━━━╸━━   1374   0         0.021     127        138.42    0:00:09   0:00:05   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━━╸━━   1354   1         0.029     127        134.83    0:00:10   0:00:05   
                                                  draws/s                       
  ━━━━╸━━   1365   0         0.032     127        135.96    0:00:10   0:00:05   
                                                  draws/s                       
  ━━━━╸━━   1369   0         0.039     127        136.35    0:00:10   0:00:05   
                                                  draws/s                       
  ━━━━╸━━   1391   0         0.021     63         138.61    0:00:10   0:00:05   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━━╸━━   1370   1         0.029     127        134.91    0:00:10   0:00:05   
                                                  draws/s                       
  ━━━━╸━━   1382   0         0.032     63         136.13    0:00:10   0:00:05   
                                                  draws/s                       
  ━━━━╸━━   1385   0         0.039     63         136.48    0:00:10   0:00:05   
                                                  draws/s                       
  ━━━━╸━━   1408   0         0.021     127        138.76    0:00:10   0:00:05   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━━╸━━   1387   1         0.029     127        135.11    0:00:10   0:00:05   
                                                  draws/s                       
  ━━━━╸━━   1398   0         0.032     127        136.23    0:00:10   0:00:05   
                                                  draws/s                       
  ━━━━╸━━   1401   0         0.039     63         136.47    0:00:10   0:00:05   
                                                  draws/s                       
  ━━━━╸━━   1426   0         0.021     31         138.94    0:00:10   0:00:05   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━━╸━━   1403   1         0.029     63         135.17    0:00:10   0:00:05   
                                                  draws/s                       
  ━━━━╸━━   1414   0         0.032     127        136.25    0:00:10   0:00:05   
                                                  draws/s                       
  ━━━━╸━━   1419   0         0.039     127        136.74    0:00:10   0:00:05   
                                                  draws/s                       
  ━━━━━╺━   1444   0         0.021     63         139.25    0:00:10   0:00:05   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━━╸━━   1416   1         0.029     127        135.07    0:00:10   0:00:05   
                                                  draws/s                       
  ━━━━╸━━   1427   0         0.032     127        136.05    0:00:10   0:00:05   
                                                  draws/s                       
  ━━━━━╺━   1434   0         0.039     127        136.78    0:00:10   0:00:05   
                                                  draws/s                       
  ━━━━━╺━   1461   0         0.021     127        139.46    0:00:10   0:00:04   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━━━╺━   1430   1         0.029     63         134.95    0:00:10   0:00:05   
                                                  draws/s                       
  ━━━━━╺━   1441   0         0.032     127        136.02    0:00:10   0:00:05   
                                                  draws/s                       
  ━━━━━╺━   1450   0         0.039     127        136.87    0:00:10   0:00:05   
                                                  draws/s                       
  ━━━━━╺━   1477   0         0.021     63         139.47    0:00:10   0:00:04   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━━━╺━   1446   1         0.029     127        135.03    0:00:10   0:00:05   
                                                  draws/s                       
  ━━━━━╺━   1457   0         0.032     63         136.03    0:00:10   0:00:05   
                                                  draws/s                       
  ━━━━━╺━   1465   0         0.039     127        136.83    0:00:10   0:00:04   
                                                  draws/s                       
  ━━━━━╺━   1497   0         0.021     63         139.85    0:00:10   0:00:04   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━━━╺━   1458   1         0.029     127        134.75    0:00:10   0:00:05   
                                                  draws/s                       
  ━━━━━╺━   1472   0         0.032     63         136.11    0:00:10   0:00:05   
                                                  draws/s                       
  ━━━━━╺━   1482   0         0.039     127        137.07    0:00:10   0:00:04   
                                                  draws/s                       
  ━━━━━╺━   1514   0         0.021     63         140.05    0:00:10   0:00:04   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━━━╺━   1473   1         0.029     127        134.71    0:00:10   0:00:05   
                                                  draws/s                       
  ━━━━━╺━   1488   0         0.032     63         136.12    0:00:10   0:00:04   
                                                  draws/s                       
  ━━━━━╺━   1496   0         0.039     127        136.91    0:00:10   0:00:04   
                                                  draws/s                       
  ━━━━━╺━   1532   0         0.021     127        140.21    0:00:10   0:00:04   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━━━╺━   1490   1         0.029     63         134.92    0:00:11   0:00:04   
                                                  draws/s                       
  ━━━━━╺━   1500   0         0.032     127        135.79    0:00:11   0:00:04   
                                                  draws/s                       
  ━━━━━╺━   1516   0         0.039     63         137.28    0:00:11   0:00:04   
                                                  draws/s                       
  ━━━━━╺━   1547   0         0.021     127        140.11    0:00:11   0:00:04   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━━━╺━   1499   1         0.029     127        134.34    0:00:11   0:00:04   
                                                  draws/s                       
  ━━━━━╺━   1509   0         0.032     127        135.23    0:00:11   0:00:04   
                                                  draws/s                       
  ━━━━━╺━   1526   0         0.039     127        136.79    0:00:11   0:00:04   
                                                  draws/s                       
  ━━━━━╺━   1555   0         0.021     127        139.42    0:00:11   0:00:04   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━━━╺━   1512   1         0.029     127        134.27    0:00:11   0:00:04   
                                                  draws/s                       
  ━━━━━╺━   1523   0         0.032     127        135.23    0:00:11   0:00:04   
                                                  draws/s                       
  ━━━━━╺━   1542   0         0.039     127        136.93    0:00:11   0:00:04   
                                                  draws/s                       
  ━━━━━╺━   1571   0         0.021     127        139.49    0:00:11   0:00:04   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━━━╺━   1526   1         0.029     127        134.16    0:00:11   0:00:04   
                                                  draws/s                       
  ━━━━━╺━   1536   0         0.032     127        135.05    0:00:11   0:00:04   
                                                  draws/s                       
  ━━━━━╺━   1556   0         0.039     127        136.83    0:00:11   0:00:04   
                                                  draws/s                       
  ━━━━━╸   1587   0         0.021     127        139.53    0:00:11   0:00:04   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━━━╺━   1542   1         0.029     63         134.24    0:00:11   0:00:04   
                                                  draws/s                       
  ━━━━━╺━   1552   0         0.032     127        135.13    0:00:11   0:00:04   
                                                  draws/s                       
  ━━━━━╺━   1570   0         0.039     127        136.75    0:00:11   0:00:04   
                                                  draws/s                       
  ━━━━━╸   1603   0         0.021     63         139.62    0:00:11   0:00:03   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━━━╺━   1558   1         0.029     127        134.30    0:00:11   0:00:04   
                                                  draws/s                       
  ━━━━━╺━   1564   0         0.032     191        134.89    0:00:11   0:00:04   
                                                  draws/s                       
  ━━━━━╸   1589   0         0.039     127        137.01    0:00:11   0:00:04   
                                                  draws/s                       
  ━━━━━╸   1621   0         0.021     127        139.79    0:00:11   0:00:03   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━━━╸   1574   1         0.029     63         134.43    0:00:11   0:00:04   
                                                  draws/s                       
  ━━━━━╸   1578   0         0.032     255        134.78    0:00:11   0:00:04   
                                                  draws/s                       
  ━━━━━╸   1603   0         0.039     127        136.98    0:00:11   0:00:03   
                                                  draws/s                       
  ━━━━━╸   1638   0         0.021     127        140.01    0:00:11   0:00:03   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━━━╸   1591   1         0.029     127        134.57    0:00:11   0:00:04   
                                                  draws/s                       
  ━━━━━╸   1594   0         0.032     63         134.83    0:00:11   0:00:04   
                                                  draws/s                       
  ━━━━━╸   1618   0         0.039     191        136.89    0:00:11   0:00:03   
                                                  draws/s                       
  ━━━━━╸   1655   0         0.021     127        140.05    0:00:11   0:00:03   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━━━╸   1605   1         0.029     127        134.53    0:00:11   0:00:04   
                                                  draws/s                       
  ━━━━━╸   1608   0         0.032     127        134.79    0:00:11   0:00:04   
                                                  draws/s                       
  ━━━━━╸   1632   0         0.039     127        136.80    0:00:11   0:00:03   
                                                  draws/s                       
  ━━━━━╸   1672   0         0.021     127        140.22    0:00:11   0:00:03   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━━━╸   1622   1         0.029     127        134.63    0:00:12   0:00:03   
                                                  draws/s                       
  ━━━━━╸   1622   0         0.032     127        134.72    0:00:12   0:00:03   
                                                  draws/s                       
  ━━━━━╸   1648   0         0.039     127        136.88    0:00:12   0:00:03   
                                                  draws/s                       
  ━━━━━╸   1691   0         0.021     63         140.43    0:00:12   0:00:03   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━━━╸   1636   1         0.029     127        134.60    0:00:12   0:00:03   
                                                  draws/s                       
  ━━━━━╸   1636   0         0.032     127        134.60    0:00:12   0:00:03   
                                                  draws/s                       
  ━━━━━╸   1666   0         0.039     63         137.12    0:00:12   0:00:03   
                                                  draws/s                       
  ━━━━━╸   1710   0         0.021     127        140.75    0:00:12   0:00:03   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━━━╸   1642   1         0.029     223        133.94    0:00:12   0:00:03   
                                                  draws/s                       
  ━━━━━╸   1643   0         0.032     127        133.98    0:00:12   0:00:03   
                                                  draws/s                       
  ━━━━━╸   1675   0         0.039     63         136.68    0:00:12   0:00:03   
                                                  draws/s                       
  ━━━━━━   1720   0         0.021     127        140.38    0:00:12   0:00:03   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━━━╸   1657   1         0.029     127        133.95    0:00:12   0:00:03   
                                                  draws/s                       
  ━━━━━╸   1654   0         0.032     255        133.73    0:00:12   0:00:03   
                                                  draws/s                       
  ━━━━━╸   1690   0         0.039     63         136.63    0:00:12   0:00:03   
                                                  draws/s                       
  ━━━━━━   1741   0         0.021     31         140.84    0:00:12   0:00:02   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━━━╸   1671   1         0.029     127        133.87    0:00:12   0:00:03   
                                                  draws/s                       
  ━━━━━╸   1667   0         0.032     63         133.55    0:00:12   0:00:03   
                                                  draws/s                       
  ━━━━━╸   1709   0         0.039     127        136.94    0:00:12   0:00:03   
                                                  draws/s                       
  ━━━━━━   1757   0         0.021     63         140.83    0:00:12   0:00:02   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━━━╸   1687   1         0.029     127        133.96    0:00:12   0:00:03   
                                                  draws/s                       
  ━━━━━╸   1681   0         0.032     127        133.49    0:00:12   0:00:03   
                                                  draws/s                       
  ━━━━━━   1724   0         0.039     127        136.92    0:00:12   0:00:03   
                                                  draws/s                       
  ━━━━━━   1774   0         0.021     127        140.91    0:00:12   0:00:02   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━━━╸   1702   1         0.029     127        133.96    0:00:12   0:00:03   
                                                  draws/s                       
  ━━━━━╸   1696   0         0.032     127        133.49    0:00:12   0:00:03   
                                                  draws/s                       
  ━━━━━━   1738   0         0.039     127        136.80    0:00:12   0:00:02   
                                                  draws/s                       
  ━━━━━━   1790   0         0.021     127        140.99    0:00:12   0:00:02   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━━━━   1717   1         0.029     127        134.01    0:00:12   0:00:03   
                                                  draws/s                       
  ━━━━━╸   1711   0         0.032     127        133.53    0:00:12   0:00:03   
                                                  draws/s                       
  ━━━━━━   1756   0         0.039     63         137.04    0:00:12   0:00:02   
                                                  draws/s                       
  ━━━━━━   1806   0         0.021     63         141.01    0:00:12   0:00:02   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━━━━   1731   1         0.029     127        133.99    0:00:12   0:00:03   
                                                  draws/s                       
  ━━━━━━   1726   0         0.032     127        133.53    0:00:12   0:00:03   
                                                  draws/s                       
  ━━━━━━   1771   0         0.039     127        137.06    0:00:12   0:00:02   
                                                  draws/s                       
  ━━━━━━   1819   0         0.021     127        140.82    0:00:12   0:00:02   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━━━━   1742   1         0.029     127        133.73    0:00:13   0:00:03   
                                                  draws/s                       
  ━━━━━━   1739   0         0.032     127        133.39    0:00:13   0:00:03   
                                                  draws/s                       
  ━━━━━━   1787   0         0.039     127        137.09    0:00:13   0:00:02   
                                                  draws/s                       
  ━━━━━━   1835   0         0.021     127        140.83    0:00:13   0:00:02   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━━━━   1758   1         0.029     127        133.67    0:00:13   0:00:02   
                                                  draws/s                       
  ━━━━━━   1755   0         0.032     31         133.49    0:00:13   0:00:02   
                                                  draws/s                       
  ━━━━━━   1805   0         0.039     127        137.28    0:00:13   0:00:02   
                                                  draws/s                       
  ━━━━━━   1853   1         0.021     68         140.95    0:00:13   0:00:02   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━━━━   1766   1         0.029     63         133.19    0:00:13   0:00:02   
                                                  draws/s                       
  ━━━━━━   1763   0         0.032     31         132.94    0:00:13   0:00:02   
                                                  draws/s                       
  ━━━━━━   1814   0         0.039     63         136.92    0:00:13   0:00:02   
                                                  draws/s                       
  ━━━━━━╸   1863   1         0.021     127        140.51    0:00:13   0:00:01   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━━━━   1779   1         0.029     127        133.09    0:00:13   0:00:02   
                                                  draws/s                       
  ━━━━━━   1779   0         0.032     127        133.08    0:00:13   0:00:02   
                                                  draws/s                       
  ━━━━━━   1831   0         0.039     127        136.97    0:00:13   0:00:02   
                                                  draws/s                       
  ━━━━━━╸   1880   1         0.021     127        140.66    0:00:13   0:00:01   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━━━━   1793   1         0.029     127        133.04    0:00:13   0:00:02   
                                                  draws/s                       
  ━━━━━━   1795   0         0.032     63         133.17    0:00:13   0:00:02   
                                                  draws/s                       
  ━━━━━━   1847   0         0.039     127        137.08    0:00:13   0:00:02   
                                                  draws/s                       
  ━━━━━━╸   1894   1         0.021     127        140.57    0:00:13   0:00:01   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━━━━   1807   1         0.029     127        132.99    0:00:13   0:00:02   
                                                  draws/s                       
  ━━━━━━   1810   0         0.032     127        133.25    0:00:13   0:00:02   
                                                  draws/s                       
  ━━━━━━╸   1863   0         0.039     63         137.17    0:00:13   0:00:02   
                                                  draws/s                       
  ━━━━━━╸   1907   1         0.021     95         140.43    0:00:13   0:00:01   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━━━━   1823   1         0.029     127        133.11    0:00:13   0:00:02   
                                                  draws/s                       
  ━━━━━━   1823   0         0.032     127        133.11    0:00:13   0:00:02   
                                                  draws/s                       
  ━━━━━━╸   1876   0         0.039     127        137.05    0:00:13   0:00:01   
                                                  draws/s                       
  ━━━━━━╸   1924   1         0.021     63         140.57    0:00:13   0:00:01   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━━━━   1836   1         0.029     127        133.02    0:00:13   0:00:02   
                                                  draws/s                       
  ━━━━━━   1838   0         0.032     127        133.16    0:00:13   0:00:02   
                                                  draws/s                       
  ━━━━━━╸   1892   0         0.039     127        137.10    0:00:13   0:00:01   
                                                  draws/s                       
  ━━━━━━╸   1939   1         0.021     127        140.54    0:00:13   0:00:01   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━━━━   1849   1         0.029     127        132.94    0:00:13   0:00:02   
                                                  draws/s                       
  ━━━━━━   1855   0         0.032     63         133.39    0:00:13   0:00:02   
                                                  draws/s                       
  ━━━━━━╸   1905   0         0.039     63         136.99    0:00:13   0:00:01   
                                                  draws/s                       
  ━━━━━━╸   1955   1         0.021     127        140.58    0:00:13   0:00:01   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━━━━╸   1863   1         0.029     127        132.89    0:00:14   0:00:02   
                                                  draws/s                       
  ━━━━━━╸   1871   0         0.032     63         133.46    0:00:14   0:00:02   
                                                  draws/s                       
  ━━━━━━╸   1922   0         0.039     127        137.15    0:00:14   0:00:01   
                                                  draws/s                       
  ━━━━━━╸   1969   1         0.021     127        140.58    0:00:14   0:00:01   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━━━━╸   1878   1         0.029     127        132.88    0:00:14   0:00:01   
                                                  draws/s                       
  ━━━━━━╸   1885   0         0.032     127        133.43    0:00:14   0:00:01   
                                                  draws/s                       
  ━━━━━━╸   1940   0         0.039     127        137.27    0:00:14   0:00:01   
                                                  draws/s                       
  ━━━━━━╸   1982   1         0.021     127        140.27    0:00:14   0:00:01   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━━━━╸   1886   1         0.029     127        132.35    0:00:14   0:00:01   
                                                  draws/s                       
  ━━━━━━╸   1895   0         0.032     127        133.09    0:00:14   0:00:01   
                                                  draws/s                       
  ━━━━━━╸   1951   0         0.039     63         137.05    0:00:14   0:00:01   
                                                  draws/s                       
  ━━━━━━╸   1991   1         0.021     127        139.79    0:00:14   0:00:01   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━━━━╸   1894   1         0.029     127        132.32    0:00:14   0:00:01   
                                                  draws/s                       
  ━━━━━━╸   1904   0         0.032     127        133.01    0:00:14   0:00:01   
                                                  draws/s                       
  ━━━━━━╸   1962   0         0.039     63         137.12    0:00:14   0:00:01   
                                                  draws/s                       
  ━━━━━━━   2000   1         0.021     63         139.71    0:00:14   0:00:00   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━━━━╸   1901   1         0.029     127        132.34    0:00:14   0:00:01   
                                                  draws/s                       
  ━━━━━━╸   1911   0         0.032     127        133.06    0:00:14   0:00:01   
                                                  draws/s                       
  ━━━━━━╸   1972   0         0.039     127        137.32    0:00:14   0:00:01   
                                                  draws/s                       
  ━━━━━━━   2000   1         0.021     63         139.71    0:00:14   0:00:00   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━━━━╸   1917   1         0.029     127        132.43    0:00:14   0:00:01   
                                                  draws/s                       
  ━━━━━━╸   1927   0         0.032     95         133.10    0:00:14   0:00:01   
                                                  draws/s                       
  ━━━━━━╸   1989   0         0.039     63         137.45    0:00:14   0:00:01   
                                                  draws/s                       
  ━━━━━━━   2000   1         0.021     63         139.71    0:00:14   0:00:00   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━━━━╸   1926   1         0.029     31         132.56    0:00:14   0:00:01   
                                                  draws/s                       
  ━━━━━━╸   1935   0         0.032     127        133.20    0:00:14   0:00:01   
                                                  draws/s                       
  ━━━━━━━   2000   0         0.039     127        137.64    0:00:14   0:00:00   
                                                  draws/s                       
  ━━━━━━━   2000   1         0.021     63         139.71    0:00:14   0:00:00   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━━━━╸   1934   1         0.029     127        132.60    0:00:14   0:00:01   
                                                  draws/s                       
  ━━━━━━╸   1943   0         0.032     127        133.21    0:00:14   0:00:01   
                                                  draws/s                       
  ━━━━━━━   2000   0         0.039     127        137.64    0:00:14   0:00:00   
                                                  draws/s                       
  ━━━━━━━   2000   1         0.021     63         139.71    0:00:14   0:00:00   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━━━━╸   1951   1         0.029     127        132.81    0:00:14   0:00:01   
                                                  draws/s                       
  ━━━━━━╸   1960   0         0.032     127        133.41    0:00:14   0:00:01   
                                                  draws/s                       
  ━━━━━━━   2000   0         0.039     127        137.64    0:00:14   0:00:00   
                                                  draws/s                       
  ━━━━━━━   2000   1         0.021     63         139.71    0:00:14   0:00:00   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━━━━╸   1968   1         0.029     127        132.97    0:00:14   0:00:01   
                                                  draws/s                       
  ━━━━━━╸   1977   0         0.032     63         133.59    0:00:14   0:00:01   
                                                  draws/s                       
  ━━━━━━━   2000   0         0.039     127        137.64    0:00:14   0:00:00   
                                                  draws/s                       
  ━━━━━━━   2000   1         0.021     63         139.71    0:00:14   0:00:00   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━━━━╸   1980   1         0.029     127        132.78    0:00:14   0:00:01   
                                                  draws/s                       
  ━━━━━━╸   1991   0         0.032     63         133.60    0:00:14   0:00:01   
                                                  draws/s                       
  ━━━━━━━   2000   0         0.039     127        137.64    0:00:14   0:00:00   
                                                  draws/s                       
  ━━━━━━━   2000   1         0.021     63         139.71    0:00:14   0:00:00   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━━━━╸   1991   1         0.029     127        132.98    0:00:14   0:00:01   
                                                  draws/s                       
  ━━━━━━━   2000   0         0.032     95         133.56    0:00:14   0:00:00   
                                                  draws/s                       
  ━━━━━━━   2000   0         0.039     127        137.64    0:00:14   0:00:00   
                                                  draws/s                       
  ━━━━━━━   2000   1         0.021     63         139.71    0:00:14   0:00:00   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━━━━╸   1999   1         0.029     63         133.12    0:00:15   0:00:01   
                                                  draws/s                       
  ━━━━━━━   2000   0         0.032     95         133.56    0:00:14   0:00:00   
                                                  draws/s                       
  ━━━━━━━   2000   0         0.039     127        137.64    0:00:14   0:00:00   
                                                  draws/s                       
  ━━━━━━━   2000   1         0.021     63         139.71    0:00:14   0:00:00   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━━━━━   2000   1         0.029     127        133.12    0:00:15   0:00:00   
                                                  draws/s                       
  ━━━━━━━   2000   0         0.032     95         133.56    0:00:14   0:00:00   
                                                  draws/s                       
  ━━━━━━━   2000   0         0.039     127        137.64    0:00:14   0:00:00   
                                                  draws/s                       
  ━━━━━━━   2000   1         0.021     63         139.71    0:00:14   0:00:00   
                                                  draws/s                       
                                                                                
                             Step      Grad                                     
  Progre…   Draw   Diverg…   size      evals      Speed     Elapsed   Remaini…  
 ────────────────────────────────────────────────────────────────────────────── 
  ━━━━━━━   2000   1         0.029     127        133.12    0:00:15   0:00:00   
                                                  draws/s                       
  ━━━━━━━   2000   0         0.032     95         133.56    0:00:14   0:00:00   
                                                  draws/s                       
  ━━━━━━━   2000   0         0.039     127        137.64    0:00:14   0:00:00   
                                                  draws/s                       
  ━━━━━━━   2000   1         0.021     63         139.71    0:00:14   0:00:00   
                                                  draws/s                       
                                                                                
?25hSampling 4 chains for 1_000 tune and 1_000 draw iterations (4_000 + 4_000 draws total) took 15 seconds.
There were 2 divergences after tuning. Increase `target_accept` or reparameterize.
Sampling: [beta, y_hat, y_hat_sigma]
Sampling: [y_hat]
Sampling: [y_hat]
Sampling: [y_hat]
Sampling: [y_hat]

If we wanted to take a closer look to diagnose the sampling process we could do the following:

az.summary(result.idata, round_to=2)
az.plot_trace(result.idata, var_names=["~mu"], compact=False);

Results#

Let’s use Arviz examine the posterior parameter estimates for each of the beta weightings for each country, along with the estimate of the measurement standard deviation, sigma.

az.plot_forest(result.idata, var_names=["~mu"], figsize=(8, 3), combined=True);
../_images/b07478eec66229d3d40151a3d07c625cb24269a73abb4a6b68d0e1e87bf95ec1.png

If you are new to Bayesian methods, then I would recommend checking out these resources:

We can now use the plot method of the result object that we get back from CausalPy. This will give us a pretty detailed visual output.

fig, ax = result.plot(plot_predictors=False)

# formatting
for i in [0, 1, 2]:
    ax[i].set(ylabel="Sales (thousands)")
../_images/70e438c570cd04b1bb8e1c77f249fe062c49ec5c5a910247537f6f9cda92802a.png

Nice! By creating a simple model formula and one call to CausalPy, we have been able to evaluate the lift generated in the treated unit.

In this example, there is quite a lot of measurement noise, but because we are using Bayesian inference methods here, we have a precise and principled quantification in our uncertainty.

We can see that the Bayesian \(R^2\) value for the pre-treatment data is about 0.5. This is not excellent, but pretty good for real-world data. It shows that the linear weighted combination model (the core of synthetic control) is doing a reasonable job at constructing a faux (i.e. synthetic) Denmark up until the treatment period.

This synthetic control Denmark is our estimated counterfactual - what the sales would have been if the store renovation project had not been carried out. By looking at the difference we can estimate the causal impact, or we could also call it ‘geographical lift’.

Over the year since implementation, we can see that the cumulative causal impact of sales in Denmark was close to 10,000 units. Let’s examine that in more detail. Below we look at the posterior distribution of the cumulative causal impact at the end of our time series, after the scheme had been in place for 1 year.

# get index of the final time point
index = result.post_impact_cumulative.obs_ind.max()
# grab the posterior distribution of the cumulative impact at this final time point
last_cumulative_estimate = result.post_impact_cumulative.sel({"obs_ind": index})
# get summary stats
ax = az.plot_posterior(last_cumulative_estimate, figsize=(8, 4))
ax.set(
    title="Estimated cumulative causal impact (at end of 2022)",
    xlabel="Sales (thousands)",
);
../_images/fef5860bcbb8477406acc262f867d1eea991507d8df29b507b4caaba0f4bd1ab.png

If we want, we can also extract these statistics out numerically:

az.summary(last_cumulative_estimate, kind="stats")
mean sd hdi_3% hdi_97%
x[Denmark] 11.1 0.4 10.34 11.83

Effect Summary Reporting#

For a decision-ready summary, you can use the effect_summary() method which provides a concise report with average and cumulative effects, HDI intervals, tail probabilities, and relative effects. This provides a comprehensive summary without manual post-processing.

# Generate effect summary for the full post-period
stats = result.effect_summary(treated_unit="Denmark")
stats.table
mean median hdi_lower hdi_upper p_gt_0 relative_mean relative_hdi_lower relative_hdi_upper
average 0.21 0.21 0.20 0.23 1.0 10.2 9.42 10.99
cumulative 11.10 11.10 10.32 11.88 1.0 10.2 9.42 10.99
print(stats.text)
During the Post-period (2022-01-02 00:00:00 to 2022-12-25 00:00:00), the response variable had an average value of approx. 2.31. By contrast, in the absence of an intervention, we would have expected an average response of 2.09. The 95% interval of this counterfactual prediction is [2.08, 2.11]. Subtracting this prediction from the observed response yields an estimate of the causal effect the intervention had on the response variable. This effect is 0.21 with a 95% interval of [0.20, 0.23].

Summing up the individual data points during the Post-period, the response variable had an overall value of 119.90. By contrast, had the intervention not taken place, we would have expected a sum of 108.81. The 95% interval of this prediction is [108.03, 109.58].

The 95% HDI of the effect [0.20, 0.23] does not include zero. The posterior probability of an increase is 1.000. Relative to the counterfactual, the effect represents a 10.20% change (95% HDI [9.42%, 10.99%]).

This analysis assumes that the control units used to construct the synthetic counterfactual were not themselves affected by the intervention, and that the pre-treatment relationship between control and treated units remains stable throughout the post-treatment period. We recommend inspecting model fit, examining pre-intervention trends, and conducting sensitivity analyses (e.g., placebo tests) to support any causal conclusions drawn from this analysis.

So at the end of our causal modelling endeavours we can report to our boss something along the lines of: “We believe that the store refurbishment scheme was causally responsible for driving a total of about 11,100 additional sales. But we have uncertainty in the exact number of additional sales - our 94% credible regions span 10,310 to 11,920”.

There are of course caveats worth bearing in mind. The analysis we’ve conducted has assumed that the only major event that might selectively influence sales in Denmark was the store renovation project. If this is a reasonable assumption then we may be on relatively stable ground in making causal claims. But if there were other events which selectively effected some units (countries) and not others, then we may need to be much more cautious in our claims and resort to more in-depth modelling approaches.

But our estimated cumulative causal impact of \(11100^{11920}_{10310}\) is exactly the information needed by others in the company. They can use this figure (and even the uncertainty associated with it) and estimate how long it would take for the cost of renovating other stores to result in a net profit.

Your boss is very happy. You get a big end-of-year bonus.

Beyond GeoLift#

This example used geographical regions as the treatment units, but there is no requirement for units to be geographical regions. For example, your units could be products. Maybe your company sells many different products and one (or a few of these) are chosen to be discounted. Did this intervention of discounting the price causally increase sales volumes? Synthetic control methods can answer this (and similar questions) just as easily.

We can get nicely formatted tables from our integration with the maketables package.

from maketables import ETable

ETable(result, coef_fmt="b:.3f")
y
(1)
coef
Austria 0.034
Belgium 0.433
Bulgaria 0.131
Croatia 0.055
Cyprus 0.106
Czech_Republic 0.059
Estonia 0.079
Finland 0.075
Hungary 0.029
stats
N 208
Bayesian R2 0.943
Format of coefficient cell: Coefficient