Bayesian geolift with CausalPy#
This notebook covers how to use CausalPy’s Bayesian synthetic control functionality to assess ‘geolift’. Our hypothetical scenario is:
We are a data scientist within a company that operates over Europe. We have been given a historical dataset of sales volumes, in units of 1000’s. The data is broken down by country and was collected at weekly frequency. We have data for the past 4 years.
At the start of 2022, the marketing department decided to refurbish all the stores in Denmark. Now, at the end of 2022, the company wants you to assess whether this refurbishment programme increased sales. If you tell them that the store refurbishment scheme increased sales volumes then they will roll out the scheme to other countries. Nobody said this, but in the back of your mind you worry that if you tell them that refurbishments increase sales but that doesn’t actually happen in the future, then the companies profits will drop, the value of your shares will decrease, and your job security may be at risk.
Your boss is pretty tuned in. She also has these concerns. She knows that while it might be easy to establish an association between the store refurbishments and changes in sales volumes, we really want to know if the store refurbishments caused an increase in sales.
We know that the best way to make causal claims is to run a randomized control trial (sometimes known as an A/B test). If we have randomly assigned stores across Europe (or picked a country) at random, then perhaps an A/B test would do the job. But we did not pick Denmark at random - so we are worried about confounding variables.
But we heard about synthetic control methods and a thing called GeoLift. After some research, we decide this is exactly what we want to do. But we are particularly interested in how certain we are in the level of any uplift we detect, so we want to use Bayesian methods and get easy to interpret Bayesian credible intervals. You find a library called
CausalPythat supports exactly that use case and are delighted.
Let’s go!
import arviz as az
import pandas as pd
import causalpy as cp
%load_ext autoreload
%autoreload 2
%config InlineBackend.figure_format = 'retina'
pd.set_option("display.precision", 2)
seed = 42
Load the dataset#
CausalPy includes an example (simulated) dataset suitable to explore ideas around geographical lift testing. All we need to do is to load that, get our observation dates set up appropriately in a pandas dataframe, and define the treatment time.
df = (
cp.load_data("geolift1")
.assign(time=lambda x: pd.to_datetime(x["time"]))
.set_index("time")
)
treatment_time = pd.to_datetime("2022-01-01")
df.head()
| Austria | Belgium | Bulgaria | Croatia | Cyprus | Czech_Republic | Estonia | Finland | Greece | Hungary | Denmark | |
|---|---|---|---|---|---|---|---|---|---|---|---|
| time | |||||||||||
| 2019-01-06 | 1.63 | 1.75 | 1.51 | 1.38 | 1.97 | 1.21 | 2.06 | 1.69 | 2.98 | 4.02 | 1.71 |
| 2019-01-13 | 1.48 | 1.65 | 1.37 | 1.46 | 1.86 | 1.53 | 1.97 | 1.82 | 2.98 | 4.19 | 1.77 |
| 2019-01-20 | 1.73 | 1.66 | 1.47 | 1.31 | 2.30 | 1.35 | 1.80 | 1.70 | 2.81 | 4.03 | 1.93 |
| 2019-01-27 | 1.77 | 1.57 | 1.26 | 1.23 | 2.01 | 1.31 | 1.96 | 1.59 | 2.87 | 4.11 | 1.80 |
| 2019-02-03 | 1.70 | 1.73 | 1.29 | 1.30 | 2.07 | 1.28 | 1.90 | 1.86 | 2.85 | 4.23 | 1.64 |
In our dataset, columns represent the different European countries that we operate in. We also have an index which labels each row with the date - the observations are weekly in frequency. The values in the table represent the sales volumes and are in units of 1000’s. So a value of 2.4 represents 2,400 units sold per week.
So let’s plot that.
untreated = list(set(df.columns).difference({"Denmark"}))
ax = df[untreated].plot(color=[0.7, 0.7, 0.7])
df["Denmark"].plot(color="r", ax=ax)
ax.axvline(treatment_time, color="k", linestyle="--")
ax.set(title="Observed data", ylabel="Sales volume (thousands)");
This looks pretty nice, but also disappointing. It is not immediately obvious from a visual inspection that there is a clear change in sales data in Denmark after the stores were refurbished (shown by the dashed line). This is made worse by the presence of annual seasonality in the data (which is different in each country), and the inherent stochasticisty in the weekly sales data.
Pre-experiment diagnostics: market correlations#
Before fitting a model, it is worth checking how similar the candidate control markets are to each other and to the treated market. Markets that move together in the pre-treatment period are more likely to produce a reliable synthetic control – the weighted combination of controls will more easily replicate the treated unit’s trajectory.
CausalPy provides plot_correlations() to visualise the pairwise Pearson correlations between all locations in the panel data. High positive correlations between the treated market and several controls suggest that the control pool can plausibly reconstruct the treated unit’s counterfactual.
pre_treatment = df.loc[df.index < treatment_time]
corr, ax = cp.plot_correlations(pre_treatment)
ax.set(title="Pairwise correlation of pre-treatment sales");
The heatmap reveals that most countries are strongly positively correlated with Denmark, but two countries – Greece and Hungary – are negatively correlated with the treated unit. This is a sign that those markets are driven by different underlying factors and would be poor donors for the synthetic control.
Donor pool selection#
The synthetic control method works by reconstructing the treated unit as a weighted combination of control units. Including controls that move in the opposite direction to the treated unit can introduce interpolation bias and push the synthetic control outside the convex hull condition of the donor pool [Abadie et al., 2010, Abadie, 2021]. Abadie and L'Hour [2021] further show that dissimilar donors amplify estimation error in disaggregated settings.
Practical guideline
Before fitting a synthetic control model, inspect the pre-treatment correlation heatmap and exclude any control units that are negatively correlated (or only weakly correlated) with the treated unit. This is especially important for Bayesian implementations that use a Dirichlet prior, since all donors receive non-zero weight by construction.
denmark_corr = corr["Denmark"].drop("Denmark")
good_donors = list(denmark_corr[denmark_corr > 0].index)
excluded = list(denmark_corr[denmark_corr <= 0].index)
print(f"Keeping {len(good_donors)} donors: {good_donors}")
print(f"Excluding {len(excluded)} units: {excluded}")
Keeping 9 donors: ['Austria', 'Belgium', 'Bulgaria', 'Croatia', 'Cyprus', 'Czech_Republic', 'Estonia', 'Finland', 'Hungary']
Excluding 1 units: ['Greece']
With the contrarian markets removed, the remaining donor pool consists of countries that co-move with Denmark during the pre-treatment period. We can now proceed to fit the synthetic control model using these curated donors.
Was there a geolift in sales in Denmark?#
In order to calculate what (if any) causal effect there is from the store refurbishment we need to compare the actual sales in Denmark after the intervention and the counterfactual sales in Denmark if the intervention had not taken place. We can see why this is called the counterfactual - we did refurbish the stores in Denmark, so this is a completely hypothetical scenario that runs counter to the facts. But if we could measure (or more realistically estimate) this, that would be our control group.
In this case, we generate a synthetic control, which is the name of the technique we will be using to estimate our counterfactual sales data in Denmark if the refurbishment had not taken place. You can read more about the synthetic control method on the synthetic control wikipedia page, but the basic idea is as follows. For those familiar with traditional (non-Bayesian) modelling methods, the basic synthetic control algorithm can be described like this:
import my_custom_scikit_learn_model as weighted_combination
# fit the model to pre-intervention (training) data
weighted_combination.fit(X_train, y_train)
# estimate the counterfactual given post-intervention (test) data
counterfactual = weighted_combination.predict(X_test)
# estimate the causal impact
causal_impact = y_test - counterfactual
# cumulative causal impact
np.cumsum(causal_impact)
So there is no magic involved - we simply estimate a synthetic Denmark as a weighted sum of the untreated units. We do this based on the ‘training’ data observed before the intervention. We then use that weighted sum model to predict our synthetic Denmark based on ‘test’ data of untreated countries observed after the intervention. We can then simply compare this counterfactual estimate with the observed sales data in Denmark and obtain our estimate of the causal impact. We can then (optionally) calculate the cumulative causal impact to answer the question “How many more sales were caused by the store refurbishment over 2022?”
We can use CausalPy’s API to run this procedure, but using Bayesian inference methods as follows:
Note
The random_seed keyword argument for the PyMC sampler is not necessary. We use it here so that the results are reproducible.
result = cp.SyntheticControl(
df,
treatment_time,
treated_units=["Denmark"],
control_units=good_donors,
model=cp.pymc_models.WeightedSumFitter(
sample_kwargs={"target_accept": 0.95, "random_seed": seed}
),
)
Initializing NUTS using jitter+adapt_diag...
Multiprocess sampling (4 chains in 4 jobs)
NUTS: [beta, y_hat_sigma]
?25l
Step Grad
Progress Draw Diverge… size evals Speed Elapsed Remain…
──────────────────────────────────────────────────────────────────────────────
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━━━━━ 0 0 0.000 0 0.00 0:00:00 -:--:--
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━━━━━ 0 0 0.000 0 0.00 0:00:00 -:--:--
draws/s
━━━━━━━ 0 0 0.000 0 0.00 0:00:00 -:--:--
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━━━━━ 0 0 0.000 0 0.00 0:00:00 -:--:--
draws/s
━━━━━━━ 0 0 0.000 0 0.00 0:00:00 -:--:--
draws/s
━━━━━━━ 0 0 0.000 0 0.00 0:00:00 -:--:--
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━━━━━ 0 0 0.000 0 0.00 0:00:00 -:--:--
draws/s
━━━━━━━ 0 0 0.000 0 0.00 0:00:00 -:--:--
draws/s
━━━━━━━ 0 0 0.000 0 0.00 0:00:00 -:--:--
draws/s
━━━━━━━ 0 0 0.000 0 0.00 0:00:00 -:--:--
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━━━━━ 16 0 0.029 127 158.83 0:00:00 0:00:13
draws/s
━━━━━━━ 11 0 0.003 255 115.95 0:00:00 0:00:18
draws/s
━━━━━━━ 19 0 0.003 255 194.99 0:00:00 0:00:11
draws/s
━━━━━━━ 10 0 0.009 255 108.83 0:00:00 0:00:20
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━━━━━ 39 0 0.026 63 181.59 0:00:00 0:00:11
draws/s
━━━━━━━ 24 0 0.027 127 111.92 0:00:00 0:00:18
draws/s
━━━━━━━ 28 0 0.015 255 137.58 0:00:00 0:00:15
draws/s
━━━━━━━ 24 0 0.033 63 114.03 0:00:00 0:00:18
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━━━━━ 64 0 0.025 127 194.98 0:00:00 0:00:10
draws/s
━━━━━━━ 47 0 0.011 255 143.59 0:00:00 0:00:14
draws/s
━━━━━━━ 45 0 0.018 127 138.75 0:00:00 0:00:15
draws/s
━━━━━━━ 41 0 0.020 63 129.24 0:00:00 0:00:16
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━━━━━ 82 0 0.027 127 189.45 0:00:00 0:00:11
draws/s
━━━━━━━ 61 0 0.027 127 140.28 0:00:00 0:00:14
draws/s
━━━━━━━ 64 0 0.023 127 148.30 0:00:00 0:00:14
draws/s
━━━━━━━ 61 0 0.023 127 141.79 0:00:00 0:00:14
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━━━━━ 97 0 0.016 127 177.36 0:00:00 0:00:11
draws/s
━━━━━━━ 79 0 0.023 127 144.43 0:00:00 0:00:14
draws/s
━━━━━━━ 82 0 0.022 127 150.69 0:00:00 0:00:13
draws/s
━━━━━━━ 74 0 0.014 255 138.82 0:00:00 0:00:14
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━━━━━ 109 0 0.030 127 166.43 0:00:00 0:00:12
draws/s
━━━━━━━ 90 0 0.027 127 137.20 0:00:00 0:00:14
draws/s
━━━━━━━ 92 0 0.008 255 140.93 0:00:00 0:00:14
draws/s
━━━━━━━ 86 0 0.031 63 131.95 0:00:00 0:00:15
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━━━━━ 122 0 0.026 127 158.96 0:00:00 0:00:12
draws/s
━━━━━━━ 107 0 0.026 255 139.26 0:00:00 0:00:14
draws/s
━━━━━━━ 104 0 0.024 127 136.21 0:00:00 0:00:14
draws/s
━━━━━━━ 110 0 0.049 127 143.94 0:00:00 0:00:14
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━━━━━ 137 0 0.029 127 155.72 0:00:00 0:00:12
draws/s
━━━━━━━ 118 0 0.033 127 133.83 0:00:00 0:00:15
draws/s
━━━━━━━ 120 0 0.031 127 136.45 0:00:00 0:00:14
draws/s
━━━━━━━ 124 0 0.044 127 141.62 0:00:00 0:00:14
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
╸━━━━━━ 157 0 0.036 63 158.04 0:00:00 0:00:12
draws/s
━━━━━━━ 134 0 0.014 255 135.74 0:00:00 0:00:14
draws/s
━━━━━━━ 132 0 0.016 255 133.76 0:00:00 0:00:14
draws/s
━━━━━━━ 140 0 0.033 255 141.71 0:00:00 0:00:14
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
╸━━━━━━ 179 0 0.015 255 161.99 0:00:01 0:00:12
draws/s
╸━━━━━━ 145 0 0.026 63 131.95 0:00:01 0:00:15
draws/s
━━━━━━━ 141 0 0.017 191 128.21 0:00:01 0:00:15
draws/s
╸━━━━━━ 156 0 0.041 127 142.26 0:00:01 0:00:13
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
╸━━━━━━ 191 0 0.031 127 157.78 0:00:01 0:00:12
draws/s
╸━━━━━━ 163 0 0.030 127 134.97 0:00:01 0:00:14
draws/s
╸━━━━━━ 154 0 0.030 63 128.10 0:00:01 0:00:15
draws/s
╸━━━━━━ 177 0 0.053 63 146.97 0:00:01 0:00:13
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
╸━━━━━━ 206 0 0.019 255 156.80 0:00:01 0:00:12
draws/s
╸━━━━━━ 187 0 0.052 63 141.87 0:00:01 0:00:13
draws/s
╸━━━━━━ 171 0 0.028 127 130.00 0:00:01 0:00:15
draws/s
╸━━━━━━ 194 0 0.034 95 147.63 0:00:01 0:00:13
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
╸━━━━━━ 219 0 0.018 255 153.53 0:00:01 0:00:12
draws/s
╸━━━━━━ 204 0 0.018 255 143.84 0:00:01 0:00:13
draws/s
╸━━━━━━ 183 0 0.021 127 128.82 0:00:01 0:00:15
draws/s
╸━━━━━━ 210 0 0.037 127 148.00 0:00:01 0:00:13
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
╸━━━━━━ 235 0 0.038 63 152.81 0:00:01 0:00:12
draws/s
╸━━━━━━ 215 0 0.024 127 140.27 0:00:01 0:00:13
draws/s
╸━━━━━━ 199 0 0.016 127 130.05 0:00:01 0:00:14
draws/s
╸━━━━━━ 227 0 0.037 63 148.31 0:00:01 0:00:12
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
╸━━━━━━ 250 0 0.028 127 152.04 0:00:01 0:00:12
draws/s
╸━━━━━━ 229 0 0.012 127 139.45 0:00:01 0:00:13
draws/s
╸━━━━━━ 210 0 0.029 127 127.81 0:00:01 0:00:15
draws/s
╸━━━━━━ 241 0 0.024 255 146.85 0:00:01 0:00:12
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
╸━━━━━━ 266 0 0.043 127 151.49 0:00:01 0:00:12
draws/s
╸━━━━━━ 236 0 0.019 127 134.99 0:00:01 0:00:14
draws/s
╸━━━━━━ 229 0 0.038 127 130.52 0:00:01 0:00:14
draws/s
╸━━━━━━ 254 0 0.027 127 145.46 0:00:01 0:00:13
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
╸━━━━━━ 283 0 0.030 255 151.59 0:00:01 0:00:12
draws/s
╸━━━━━━ 254 0 0.031 63 135.94 0:00:01 0:00:13
draws/s
╸━━━━━━ 245 0 0.020 127 131.48 0:00:01 0:00:14
draws/s
╸━━━━━━ 278 0 0.031 63 149.15 0:00:01 0:00:12
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━╺━━━━━ 293 0 0.014 255 149.34 0:00:01 0:00:12
draws/s
╸━━━━━━ 271 0 0.030 127 136.92 0:00:01 0:00:13
draws/s
╸━━━━━━ 261 0 0.036 63 132.19 0:00:01 0:00:14
draws/s
━╺━━━━━ 292 0 0.020 127 147.99 0:00:01 0:00:12
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━╺━━━━━ 302 0 0.023 127 144.99 0:00:02 0:00:12
draws/s
━╺━━━━━ 287 0 0.036 127 137.78 0:00:02 0:00:13
draws/s
╸━━━━━━ 277 0 0.036 127 133.27 0:00:02 0:00:13
draws/s
━╺━━━━━ 302 0 0.026 255 145.49 0:00:02 0:00:12
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━╺━━━━━ 314 0 0.025 255 143.46 0:00:02 0:00:12
draws/s
━╺━━━━━ 311 0 0.042 15 141.95 0:00:02 0:00:12
draws/s
━╺━━━━━ 297 0 0.040 63 135.60 0:00:02 0:00:13
draws/s
━╺━━━━━ 315 0 0.025 255 144.35 0:00:02 0:00:12
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━╺━━━━━ 332 0 0.030 63 144.26 0:00:02 0:00:12
draws/s
━╺━━━━━ 335 0 0.036 127 145.71 0:00:02 0:00:12
draws/s
━╺━━━━━ 314 0 0.024 255 137.00 0:00:02 0:00:13
draws/s
━╺━━━━━ 334 0 0.024 127 145.59 0:00:02 0:00:12
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━╺━━━━━ 347 0 0.032 127 143.75 0:00:02 0:00:12
draws/s
━╺━━━━━ 352 0 0.031 127 145.96 0:00:02 0:00:12
draws/s
━╺━━━━━ 327 0 0.020 255 136.24 0:00:02 0:00:13
draws/s
━╺━━━━━ 346 0 0.021 127 143.95 0:00:02 0:00:12
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━╺━━━━━ 366 0 0.031 127 145.02 0:00:02 0:00:12
draws/s
━╺━━━━━ 370 0 0.031 63 146.65 0:00:02 0:00:12
draws/s
━╺━━━━━ 341 0 0.025 127 135.07 0:00:02 0:00:13
draws/s
━╺━━━━━ 359 0 0.025 127 142.67 0:00:02 0:00:12
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━╺━━━━━ 382 0 0.024 159 144.87 0:00:02 0:00:12
draws/s
━╺━━━━━ 389 0 0.021 127 147.74 0:00:02 0:00:11
draws/s
━╺━━━━━ 353 0 0.035 127 134.25 0:00:02 0:00:13
draws/s
━╺━━━━━ 376 0 0.030 127 143.05 0:00:02 0:00:12
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━╺━━━━━ 397 0 0.033 127 144.20 0:00:02 0:00:12
draws/s
━╺━━━━━ 407 0 0.037 127 147.79 0:00:02 0:00:11
draws/s
━╺━━━━━ 373 0 0.026 127 135.61 0:00:02 0:00:13
draws/s
━╺━━━━━ 397 0 0.034 31 144.32 0:00:02 0:00:12
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━╺━━━━━ 420 0 0.035 63 146.58 0:00:02 0:00:11
draws/s
━╺━━━━━ 425 0 0.019 127 148.33 0:00:02 0:00:11
draws/s
━╺━━━━━ 392 0 0.037 127 136.99 0:00:02 0:00:12
draws/s
━╺━━━━━ 412 0 0.029 127 144.02 0:00:02 0:00:12
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━╸━━━━━ 438 0 0.023 255 147.35 0:00:02 0:00:11
draws/s
━╸━━━━━ 439 0 0.031 127 147.47 0:00:02 0:00:11
draws/s
━╺━━━━━ 405 0 0.028 127 136.43 0:00:02 0:00:12
draws/s
━╸━━━━━ 432 0 0.029 127 145.30 0:00:02 0:00:11
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━╸━━━━━ 452 0 0.035 127 146.21 0:00:03 0:00:11
draws/s
━╸━━━━━ 457 0 0.030 127 148.17 0:00:03 0:00:11
draws/s
━╺━━━━━ 423 0 0.027 127 136.99 0:00:03 0:00:12
draws/s
━╸━━━━━ 447 0 0.024 127 145.01 0:00:03 0:00:11
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━╸━━━━━ 468 0 0.015 511 146.34 0:00:03 0:00:11
draws/s
━╸━━━━━ 478 0 0.030 127 149.29 0:00:03 0:00:11
draws/s
━╸━━━━━ 441 0 0.034 63 137.89 0:00:03 0:00:12
draws/s
━╸━━━━━ 459 0 0.018 255 143.70 0:00:03 0:00:11
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━╸━━━━━ 473 0 0.019 255 142.92 0:00:03 0:00:11
draws/s
━╸━━━━━ 488 0 0.020 127 147.27 0:00:03 0:00:11
draws/s
━╸━━━━━ 453 0 0.035 127 136.79 0:00:03 0:00:12
draws/s
━╸━━━━━ 467 0 0.025 127 141.07 0:00:03 0:00:11
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━╸━━━━━ 485 0 0.030 127 141.52 0:00:03 0:00:11
draws/s
━╸━━━━━ 501 0 0.030 63 146.29 0:00:03 0:00:11
draws/s
━╸━━━━━ 465 0 0.023 255 135.71 0:00:03 0:00:12
draws/s
━╸━━━━━ 486 0 0.040 127 141.90 0:00:03 0:00:11
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━╸━━━━━ 501 0 0.032 127 141.64 0:00:03 0:00:11
draws/s
━╸━━━━━ 516 0 0.029 127 146.15 0:00:03 0:00:11
draws/s
━╸━━━━━ 474 0 0.023 127 134.37 0:00:03 0:00:12
draws/s
━╸━━━━━ 503 0 0.029 63 142.46 0:00:03 0:00:11
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━╸━━━━━ 519 0 0.036 127 142.36 0:00:03 0:00:11
draws/s
━╸━━━━━ 535 0 0.031 127 146.79 0:00:03 0:00:10
draws/s
━╸━━━━━ 491 0 0.028 127 134.73 0:00:03 0:00:12
draws/s
━╸━━━━━ 523 0 0.035 31 143.63 0:00:03 0:00:11
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━╸━━━━━ 537 0 0.036 127 143.10 0:00:03 0:00:11
draws/s
━╸━━━━━ 551 0 0.027 127 146.88 0:00:03 0:00:10
draws/s
━╸━━━━━ 501 0 0.031 159 133.63 0:00:03 0:00:12
draws/s
━╸━━━━━ 540 0 0.038 119 144.29 0:00:03 0:00:11
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━╸━━━━━ 557 0 0.033 127 144.33 0:00:03 0:00:10
draws/s
━╸━━━━━ 563 0 0.024 127 145.87 0:00:03 0:00:10
draws/s
━╸━━━━━ 514 0 0.022 127 133.28 0:00:03 0:00:12
draws/s
━╸━━━━━ 558 0 0.038 127 144.69 0:00:03 0:00:10
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━╺━━━━ 572 0 0.022 127 143.78 0:00:03 0:00:10
draws/s
━━╺━━━━ 580 0 0.022 127 145.97 0:00:03 0:00:10
draws/s
━╸━━━━━ 534 0 0.036 63 134.30 0:00:03 0:00:11
draws/s
━━╺━━━━ 572 0 0.025 255 144.12 0:00:03 0:00:10
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━╺━━━━ 586 0 0.032 127 143.39 0:00:04 0:00:10
draws/s
━━╺━━━━ 593 0 0.028 127 145.15 0:00:04 0:00:10
draws/s
━╸━━━━━ 552 0 0.036 127 135.23 0:00:04 0:00:11
draws/s
━━╺━━━━ 583 0 0.024 255 143.17 0:00:04 0:00:10
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━╺━━━━ 605 0 0.021 63 144.31 0:00:04 0:00:10
draws/s
━━╺━━━━ 609 0 0.026 127 145.21 0:00:04 0:00:10
draws/s
━━╺━━━━ 572 0 0.029 127 136.33 0:00:04 0:00:11
draws/s
━━╺━━━━ 600 0 0.028 127 143.04 0:00:04 0:00:10
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━╺━━━━ 622 0 0.031 127 144.22 0:00:04 0:00:10
draws/s
━━╺━━━━ 626 0 0.028 127 145.28 0:00:04 0:00:10
draws/s
━━╺━━━━ 592 0 0.035 127 137.34 0:00:04 0:00:11
draws/s
━━╺━━━━ 614 0 0.030 127 142.62 0:00:04 0:00:10
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━╺━━━━ 636 0 0.022 191 144.12 0:00:04 0:00:10
draws/s
━━╺━━━━ 642 0 0.033 127 145.23 0:00:04 0:00:10
draws/s
━━╺━━━━ 608 0 0.035 127 137.64 0:00:04 0:00:11
draws/s
━━╺━━━━ 628 0 0.036 63 142.45 0:00:04 0:00:10
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━╺━━━━ 648 0 0.029 127 143.15 0:00:04 0:00:10
draws/s
━━╺━━━━ 658 0 0.028 127 145.30 0:00:04 0:00:10
draws/s
━━╺━━━━ 626 0 0.039 63 138.28 0:00:04 0:00:10
draws/s
━━╺━━━━ 646 0 0.029 127 142.81 0:00:04 0:00:10
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━╺━━━━ 661 0 0.026 127 142.51 0:00:04 0:00:10
draws/s
━━╺━━━━ 676 0 0.039 31 145.69 0:00:04 0:00:10
draws/s
━━╺━━━━ 641 0 0.024 255 138.20 0:00:04 0:00:10
draws/s
━━╺━━━━ 661 0 0.029 127 142.79 0:00:04 0:00:10
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━╺━━━━ 679 0 0.026 63 142.96 0:00:04 0:00:10
draws/s
━━╺━━━━ 694 0 0.038 127 146.09 0:00:04 0:00:09
draws/s
━━╺━━━━ 654 0 0.027 191 137.78 0:00:04 0:00:10
draws/s
━━╺━━━━ 677 0 0.030 127 142.76 0:00:04 0:00:10
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━╺━━━━ 692 0 0.022 127 142.20 0:00:04 0:00:10
draws/s
━━╺━━━━ 711 0 0.029 127 146.21 0:00:04 0:00:09
draws/s
━━╺━━━━ 670 0 0.028 127 137.78 0:00:04 0:00:10
draws/s
━━╺━━━━ 697 0 0.037 127 143.55 0:00:04 0:00:10
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━╺━━━━ 706 0 0.033 127 141.94 0:00:04 0:00:10
draws/s
━━╸━━━━ 727 0 0.031 127 146.21 0:00:04 0:00:09
draws/s
━━╺━━━━ 681 0 0.025 255 137.09 0:00:04 0:00:10
draws/s
━━╺━━━━ 714 0 0.034 63 143.74 0:00:04 0:00:09
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━╸━━━━ 723 0 0.034 63 142.18 0:00:05 0:00:09
draws/s
━━╸━━━━ 743 0 0.031 127 146.25 0:00:05 0:00:09
draws/s
━━╺━━━━ 695 0 0.030 127 136.89 0:00:05 0:00:10
draws/s
━━╸━━━━ 728 0 0.019 127 143.41 0:00:05 0:00:09
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━╸━━━━ 740 0 0.026 127 142.49 0:00:05 0:00:09
draws/s
━━╸━━━━ 762 0 0.032 63 146.64 0:00:05 0:00:09
draws/s
━━╺━━━━ 713 0 0.025 111 137.40 0:00:05 0:00:10
draws/s
━━╸━━━━ 744 0 0.030 127 143.33 0:00:05 0:00:09
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━╸━━━━ 756 0 0.036 127 142.56 0:00:05 0:00:09
draws/s
━━╸━━━━ 777 0 0.028 127 146.50 0:00:05 0:00:09
draws/s
━━╸━━━━ 730 0 0.028 63 137.81 0:00:05 0:00:10
draws/s
━━╸━━━━ 761 0 0.032 63 143.61 0:00:05 0:00:09
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━╸━━━━ 771 0 0.027 127 142.49 0:00:05 0:00:09
draws/s
━━╸━━━━ 791 0 0.030 127 146.09 0:00:05 0:00:09
draws/s
━━╸━━━━ 744 0 0.018 127 137.52 0:00:05 0:00:10
draws/s
━━╸━━━━ 779 0 0.026 127 144.07 0:00:05 0:00:09
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━╸━━━━ 789 0 0.036 127 142.65 0:00:05 0:00:09
draws/s
━━╸━━━━ 806 0 0.029 127 145.93 0:00:05 0:00:09
draws/s
━━╸━━━━ 757 0 0.025 127 137.02 0:00:05 0:00:10
draws/s
━━╸━━━━ 798 0 0.027 127 144.45 0:00:05 0:00:09
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━╸━━━━ 801 0 0.036 127 142.08 0:00:05 0:00:09
draws/s
━━╸━━━━ 817 0 0.033 63 144.99 0:00:05 0:00:09
draws/s
━━╸━━━━ 767 0 0.028 127 136.20 0:00:05 0:00:10
draws/s
━━╸━━━━ 808 0 0.034 127 143.58 0:00:05 0:00:09
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━╸━━━━ 817 0 0.035 127 142.21 0:00:05 0:00:09
draws/s
━━╸━━━━ 834 0 0.025 63 145.14 0:00:05 0:00:09
draws/s
━━╸━━━━ 786 0 0.027 127 136.98 0:00:05 0:00:09
draws/s
━━╸━━━━ 825 0 0.040 79 143.83 0:00:05 0:00:09
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━╸━━━━ 829 0 0.024 127 141.73 0:00:05 0:00:09
draws/s
━━╸━━━━ 841 0 0.028 255 144.08 0:00:05 0:00:09
draws/s
━━╸━━━━ 799 0 0.037 127 136.64 0:00:05 0:00:09
draws/s
━━╸━━━━ 839 0 0.038 127 143.48 0:00:05 0:00:09
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━╸━━━━ 844 0 0.034 127 141.52 0:00:05 0:00:09
draws/s
━━╸━━━━ 854 0 0.029 127 143.37 0:00:05 0:00:08
draws/s
━━╸━━━━ 819 0 0.028 127 137.55 0:00:05 0:00:09
draws/s
━━╸━━━━ 852 0 0.029 127 143.10 0:00:05 0:00:09
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━╸━━━━ 857 0 0.025 127 141.21 0:00:06 0:00:09
draws/s
━━━╺━━━ 870 0 0.026 191 143.51 0:00:06 0:00:08
draws/s
━━╸━━━━ 837 0 0.040 127 138.06 0:00:06 0:00:09
draws/s
━━━╺━━━ 864 0 0.032 127 142.53 0:00:06 0:00:08
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━╺━━━ 872 0 0.032 127 141.14 0:00:06 0:00:08
draws/s
━━━╺━━━ 885 0 0.032 127 143.38 0:00:06 0:00:08
draws/s
━━╸━━━━ 856 0 0.039 63 138.69 0:00:06 0:00:09
draws/s
━━━╺━━━ 879 0 0.028 127 142.54 0:00:06 0:00:08
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━╺━━━ 893 0 0.029 63 142.00 0:00:06 0:00:08
draws/s
━━━╺━━━ 899 0 0.030 255 143.03 0:00:06 0:00:08
draws/s
━━━╺━━━ 871 0 0.028 127 138.65 0:00:06 0:00:09
draws/s
━━━╺━━━ 895 0 0.031 255 142.59 0:00:06 0:00:08
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━╺━━━ 907 0 0.033 191 141.84 0:00:06 0:00:08
draws/s
━━━╺━━━ 915 0 0.032 127 143.19 0:00:06 0:00:08
draws/s
━━━╺━━━ 888 0 0.042 63 139.00 0:00:06 0:00:09
draws/s
━━━╺━━━ 904 0 0.017 255 141.70 0:00:06 0:00:08
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━╺━━━ 921 0 0.030 255 141.66 0:00:06 0:00:08
draws/s
━━━╺━━━ 931 0 0.038 63 143.20 0:00:06 0:00:08
draws/s
━━━╺━━━ 905 0 0.035 127 139.23 0:00:06 0:00:08
draws/s
━━━╺━━━ 913 0 0.023 255 140.69 0:00:06 0:00:08
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━╺━━━ 936 0 0.033 63 141.49 0:00:06 0:00:08
draws/s
━━━╺━━━ 947 0 0.034 127 143.08 0:00:06 0:00:08
draws/s
━━━╺━━━ 924 0 0.046 127 139.67 0:00:06 0:00:08
draws/s
━━━╺━━━ 928 0 0.027 63 140.30 0:00:06 0:00:08
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━╺━━━ 951 0 0.024 63 141.57 0:00:06 0:00:08
draws/s
━━━╺━━━ 961 0 0.033 47 142.88 0:00:06 0:00:08
draws/s
━━━╺━━━ 941 0 0.040 63 140.02 0:00:06 0:00:08
draws/s
━━━╺━━━ 943 0 0.032 127 140.26 0:00:06 0:00:08
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━╺━━━ 961 0 0.027 127 140.71 0:00:06 0:00:08
draws/s
━━━╺━━━ 972 0 0.023 255 142.13 0:00:06 0:00:08
draws/s
━━━╺━━━ 954 0 0.022 255 139.55 0:00:06 0:00:08
draws/s
━━━╺━━━ 957 0 0.025 63 140.13 0:00:06 0:00:08
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━╺━━━ 977 0 0.028 127 140.64 0:00:06 0:00:08
draws/s
━━━╺━━━ 984 0 0.027 127 141.90 0:00:06 0:00:08
draws/s
━━━╺━━━ 964 0 0.027 127 138.88 0:00:06 0:00:08
draws/s
━━━╺━━━ 972 0 0.030 127 140.07 0:00:06 0:00:08
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━╺━━━ 988 0 0.026 127 140.14 0:00:07 0:00:08
draws/s
━━━╺━━━ 995 0 0.028 127 141.08 0:00:07 0:00:08
draws/s
━━━╺━━━ 974 0 0.036 127 138.19 0:00:07 0:00:08
draws/s
━━━╺━━━ 984 0 0.036 127 139.60 0:00:07 0:00:08
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━╺━━━ 994 0 0.029 127 138.84 0:00:07 0:00:08
draws/s
━━━╸━━━ 1002 0 0.032 127 139.90 0:00:07 0:00:08
draws/s
━━━╺━━━ 981 0 0.038 127 137.00 0:00:07 0:00:08
draws/s
━━━╺━━━ 993 0 0.031 63 138.75 0:00:07 0:00:08
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━╸━━━ 1010 0 0.029 63 138.84 0:00:07 0:00:08
draws/s
━━━╸━━━ 1016 0 0.032 127 139.87 0:00:07 0:00:07
draws/s
━━━╸━━━ 1000 0 0.039 31 137.50 0:00:07 0:00:08
draws/s
━━━╸━━━ 1008 0 0.021 127 138.67 0:00:07 0:00:08
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━╸━━━ 1024 0 0.029 127 138.67 0:00:07 0:00:08
draws/s
━━━╸━━━ 1029 0 0.032 127 139.37 0:00:07 0:00:07
draws/s
━━━╸━━━ 1011 0 0.039 127 136.91 0:00:07 0:00:08
draws/s
━━━╸━━━ 1023 0 0.021 127 138.67 0:00:07 0:00:08
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━╸━━━ 1038 1 0.029 127 138.48 0:00:07 0:00:08
draws/s
━━━╸━━━ 1047 0 0.032 127 139.68 0:00:07 0:00:07
draws/s
━━━╸━━━ 1026 0 0.039 127 136.92 0:00:07 0:00:08
draws/s
━━━╸━━━ 1035 0 0.021 127 138.15 0:00:07 0:00:07
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━╸━━━ 1050 1 0.029 127 138.01 0:00:07 0:00:07
draws/s
━━━╸━━━ 1062 0 0.032 127 139.73 0:00:07 0:00:07
draws/s
━━━╸━━━ 1046 0 0.039 127 137.52 0:00:07 0:00:07
draws/s
━━━╸━━━ 1054 0 0.021 127 138.64 0:00:07 0:00:07
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━╸━━━ 1066 1 0.029 63 138.08 0:00:07 0:00:07
draws/s
━━━╸━━━ 1076 0 0.032 127 139.38 0:00:07 0:00:07
draws/s
━━━╸━━━ 1061 0 0.039 127 137.41 0:00:07 0:00:07
draws/s
━━━╸━━━ 1073 0 0.021 63 139.13 0:00:07 0:00:07
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━╸━━━ 1082 1 0.029 127 138.11 0:00:07 0:00:07
draws/s
━━━╸━━━ 1091 0 0.032 127 139.32 0:00:07 0:00:07
draws/s
━━━╸━━━ 1076 0 0.039 127 137.43 0:00:07 0:00:07
draws/s
━━━╸━━━ 1090 0 0.021 127 139.25 0:00:07 0:00:07
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━╸━━━ 1098 1 0.029 127 138.23 0:00:07 0:00:07
draws/s
━━━╸━━━ 1106 0 0.032 127 139.24 0:00:07 0:00:07
draws/s
━━━╸━━━ 1093 0 0.039 127 137.64 0:00:07 0:00:07
draws/s
━━━╸━━━ 1107 0 0.021 63 139.51 0:00:07 0:00:07
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━╸━━━ 1112 1 0.029 63 138.05 0:00:08 0:00:07
draws/s
━━━╸━━━ 1120 0 0.032 127 139.15 0:00:08 0:00:07
draws/s
━━━╸━━━ 1108 0 0.039 63 137.68 0:00:08 0:00:07
draws/s
━━━╸━━━ 1121 0 0.021 103 139.20 0:00:08 0:00:07
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━╸━━━ 1126 1 0.029 127 137.91 0:00:08 0:00:07
draws/s
━━━╸━━━ 1135 0 0.032 63 139.01 0:00:08 0:00:07
draws/s
━━━╸━━━ 1125 0 0.039 127 137.80 0:00:08 0:00:07
draws/s
━━━╸━━━ 1137 0 0.021 127 139.26 0:00:08 0:00:07
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━╸━━━ 1141 1 0.029 127 137.81 0:00:08 0:00:07
draws/s
━━━━╺━━ 1150 0 0.032 255 138.86 0:00:08 0:00:07
draws/s
━━━╸━━━ 1140 0 0.039 127 137.76 0:00:08 0:00:07
draws/s
━━━━╺━━ 1155 0 0.021 127 139.59 0:00:08 0:00:07
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━━╺━━ 1152 1 0.029 127 137.30 0:00:08 0:00:07
draws/s
━━━━╺━━ 1161 0 0.032 127 138.47 0:00:08 0:00:07
draws/s
━━━━╺━━ 1153 0 0.039 63 137.57 0:00:08 0:00:07
draws/s
━━━━╺━━ 1169 0 0.021 127 139.40 0:00:08 0:00:07
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━━╺━━ 1166 1 0.029 127 137.16 0:00:08 0:00:07
draws/s
━━━━╺━━ 1171 0 0.032 255 137.75 0:00:08 0:00:07
draws/s
━━━━╺━━ 1167 0 0.039 127 137.27 0:00:08 0:00:07
draws/s
━━━━╺━━ 1184 0 0.021 63 139.34 0:00:08 0:00:06
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━━╺━━ 1175 1 0.029 127 136.31 0:00:08 0:00:07
draws/s
━━━━╺━━ 1181 0 0.032 127 137.00 0:00:08 0:00:07
draws/s
━━━━╺━━ 1178 0 0.039 127 136.68 0:00:08 0:00:06
draws/s
━━━━╺━━ 1194 0 0.021 63 138.69 0:00:08 0:00:06
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━━╺━━ 1191 1 0.029 63 136.41 0:00:08 0:00:07
draws/s
━━━━╺━━ 1196 0 0.032 127 137.03 0:00:08 0:00:06
draws/s
━━━━╺━━ 1194 0 0.039 127 136.84 0:00:08 0:00:06
draws/s
━━━━╺━━ 1209 0 0.021 127 138.61 0:00:08 0:00:06
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━━╺━━ 1204 1 0.029 95 136.33 0:00:08 0:00:07
draws/s
━━━━╺━━ 1211 0 0.032 63 137.17 0:00:08 0:00:06
draws/s
━━━━╺━━ 1211 0 0.039 63 137.15 0:00:08 0:00:06
draws/s
━━━━╺━━ 1224 0 0.021 127 138.66 0:00:08 0:00:06
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━━╺━━ 1221 1 0.029 127 136.50 0:00:08 0:00:06
draws/s
━━━━╺━━ 1226 0 0.032 127 137.04 0:00:08 0:00:06
draws/s
━━━━╺━━ 1225 0 0.039 127 136.95 0:00:08 0:00:06
draws/s
━━━━╺━━ 1240 0 0.021 127 138.66 0:00:08 0:00:06
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━━╺━━ 1233 1 0.029 127 136.09 0:00:09 0:00:06
draws/s
━━━━╺━━ 1240 0 0.032 127 136.91 0:00:09 0:00:06
draws/s
━━━━╺━━ 1239 0 0.039 127 136.89 0:00:09 0:00:06
draws/s
━━━━╺━━ 1258 0 0.021 63 138.99 0:00:09 0:00:06
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━━╺━━ 1249 1 0.029 127 136.27 0:00:09 0:00:06
draws/s
━━━━╺━━ 1255 0 0.032 127 136.90 0:00:09 0:00:06
draws/s
━━━━╺━━ 1254 0 0.039 127 136.82 0:00:09 0:00:06
draws/s
━━━━╺━━ 1275 0 0.021 127 139.14 0:00:09 0:00:06
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━━╺━━ 1262 1 0.029 127 136.07 0:00:09 0:00:06
draws/s
━━━━╺━━ 1270 0 0.032 127 136.95 0:00:09 0:00:06
draws/s
━━━━╺━━ 1269 0 0.039 127 136.88 0:00:09 0:00:06
draws/s
━━━━╸━━ 1294 0 0.021 127 139.52 0:00:09 0:00:06
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━━╺━━ 1275 1 0.029 127 135.84 0:00:09 0:00:06
draws/s
━━━━╸━━ 1286 0 0.032 127 137.04 0:00:09 0:00:06
draws/s
━━━━╺━━ 1284 0 0.039 127 136.85 0:00:09 0:00:06
draws/s
━━━━╸━━ 1308 0 0.021 127 139.44 0:00:09 0:00:06
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━━╸━━ 1288 1 0.029 63 135.64 0:00:09 0:00:06
draws/s
━━━━╸━━ 1301 0 0.032 63 137.03 0:00:09 0:00:06
draws/s
━━━━╸━━ 1299 0 0.039 63 136.90 0:00:09 0:00:06
draws/s
━━━━╸━━ 1323 0 0.021 63 139.41 0:00:09 0:00:05
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━━╸━━ 1301 1 0.029 127 135.53 0:00:09 0:00:06
draws/s
━━━━╸━━ 1316 0 0.032 127 137.02 0:00:09 0:00:06
draws/s
━━━━╸━━ 1313 0 0.039 127 136.80 0:00:09 0:00:06
draws/s
━━━━╸━━ 1337 0 0.021 127 139.27 0:00:09 0:00:05
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━━╸━━ 1317 1 0.029 127 135.58 0:00:09 0:00:06
draws/s
━━━━╸━━ 1331 0 0.032 127 137.08 0:00:09 0:00:05
draws/s
━━━━╸━━ 1328 0 0.039 127 136.77 0:00:09 0:00:05
draws/s
━━━━╸━━ 1353 0 0.021 127 139.40 0:00:09 0:00:05
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━━╸━━ 1331 1 0.029 127 135.56 0:00:09 0:00:06
draws/s
━━━━╸━━ 1342 0 0.032 127 136.70 0:00:09 0:00:05
draws/s
━━━━╸━━ 1342 0 0.039 255 136.66 0:00:09 0:00:05
draws/s
━━━━╸━━ 1366 0 0.021 127 139.22 0:00:09 0:00:05
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━━╸━━ 1339 1 0.029 127 134.82 0:00:09 0:00:06
draws/s
━━━━╸━━ 1349 0 0.032 127 135.85 0:00:09 0:00:05
draws/s
━━━━╸━━ 1352 0 0.039 63 136.13 0:00:09 0:00:05
draws/s
━━━━╸━━ 1374 0 0.021 127 138.42 0:00:09 0:00:05
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━━╸━━ 1354 1 0.029 127 134.83 0:00:10 0:00:05
draws/s
━━━━╸━━ 1365 0 0.032 127 135.96 0:00:10 0:00:05
draws/s
━━━━╸━━ 1369 0 0.039 127 136.35 0:00:10 0:00:05
draws/s
━━━━╸━━ 1391 0 0.021 63 138.61 0:00:10 0:00:05
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━━╸━━ 1370 1 0.029 127 134.91 0:00:10 0:00:05
draws/s
━━━━╸━━ 1382 0 0.032 63 136.13 0:00:10 0:00:05
draws/s
━━━━╸━━ 1385 0 0.039 63 136.48 0:00:10 0:00:05
draws/s
━━━━╸━━ 1408 0 0.021 127 138.76 0:00:10 0:00:05
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━━╸━━ 1387 1 0.029 127 135.11 0:00:10 0:00:05
draws/s
━━━━╸━━ 1398 0 0.032 127 136.23 0:00:10 0:00:05
draws/s
━━━━╸━━ 1401 0 0.039 63 136.47 0:00:10 0:00:05
draws/s
━━━━╸━━ 1426 0 0.021 31 138.94 0:00:10 0:00:05
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━━╸━━ 1403 1 0.029 63 135.17 0:00:10 0:00:05
draws/s
━━━━╸━━ 1414 0 0.032 127 136.25 0:00:10 0:00:05
draws/s
━━━━╸━━ 1419 0 0.039 127 136.74 0:00:10 0:00:05
draws/s
━━━━━╺━ 1444 0 0.021 63 139.25 0:00:10 0:00:05
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━━╸━━ 1416 1 0.029 127 135.07 0:00:10 0:00:05
draws/s
━━━━╸━━ 1427 0 0.032 127 136.05 0:00:10 0:00:05
draws/s
━━━━━╺━ 1434 0 0.039 127 136.78 0:00:10 0:00:05
draws/s
━━━━━╺━ 1461 0 0.021 127 139.46 0:00:10 0:00:04
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━━━╺━ 1430 1 0.029 63 134.95 0:00:10 0:00:05
draws/s
━━━━━╺━ 1441 0 0.032 127 136.02 0:00:10 0:00:05
draws/s
━━━━━╺━ 1450 0 0.039 127 136.87 0:00:10 0:00:05
draws/s
━━━━━╺━ 1477 0 0.021 63 139.47 0:00:10 0:00:04
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━━━╺━ 1446 1 0.029 127 135.03 0:00:10 0:00:05
draws/s
━━━━━╺━ 1457 0 0.032 63 136.03 0:00:10 0:00:05
draws/s
━━━━━╺━ 1465 0 0.039 127 136.83 0:00:10 0:00:04
draws/s
━━━━━╺━ 1497 0 0.021 63 139.85 0:00:10 0:00:04
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━━━╺━ 1458 1 0.029 127 134.75 0:00:10 0:00:05
draws/s
━━━━━╺━ 1472 0 0.032 63 136.11 0:00:10 0:00:05
draws/s
━━━━━╺━ 1482 0 0.039 127 137.07 0:00:10 0:00:04
draws/s
━━━━━╺━ 1514 0 0.021 63 140.05 0:00:10 0:00:04
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━━━╺━ 1473 1 0.029 127 134.71 0:00:10 0:00:05
draws/s
━━━━━╺━ 1488 0 0.032 63 136.12 0:00:10 0:00:04
draws/s
━━━━━╺━ 1496 0 0.039 127 136.91 0:00:10 0:00:04
draws/s
━━━━━╺━ 1532 0 0.021 127 140.21 0:00:10 0:00:04
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━━━╺━ 1490 1 0.029 63 134.92 0:00:11 0:00:04
draws/s
━━━━━╺━ 1500 0 0.032 127 135.79 0:00:11 0:00:04
draws/s
━━━━━╺━ 1516 0 0.039 63 137.28 0:00:11 0:00:04
draws/s
━━━━━╺━ 1547 0 0.021 127 140.11 0:00:11 0:00:04
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━━━╺━ 1499 1 0.029 127 134.34 0:00:11 0:00:04
draws/s
━━━━━╺━ 1509 0 0.032 127 135.23 0:00:11 0:00:04
draws/s
━━━━━╺━ 1526 0 0.039 127 136.79 0:00:11 0:00:04
draws/s
━━━━━╺━ 1555 0 0.021 127 139.42 0:00:11 0:00:04
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━━━╺━ 1512 1 0.029 127 134.27 0:00:11 0:00:04
draws/s
━━━━━╺━ 1523 0 0.032 127 135.23 0:00:11 0:00:04
draws/s
━━━━━╺━ 1542 0 0.039 127 136.93 0:00:11 0:00:04
draws/s
━━━━━╺━ 1571 0 0.021 127 139.49 0:00:11 0:00:04
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━━━╺━ 1526 1 0.029 127 134.16 0:00:11 0:00:04
draws/s
━━━━━╺━ 1536 0 0.032 127 135.05 0:00:11 0:00:04
draws/s
━━━━━╺━ 1556 0 0.039 127 136.83 0:00:11 0:00:04
draws/s
━━━━━╸━ 1587 0 0.021 127 139.53 0:00:11 0:00:04
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━━━╺━ 1542 1 0.029 63 134.24 0:00:11 0:00:04
draws/s
━━━━━╺━ 1552 0 0.032 127 135.13 0:00:11 0:00:04
draws/s
━━━━━╺━ 1570 0 0.039 127 136.75 0:00:11 0:00:04
draws/s
━━━━━╸━ 1603 0 0.021 63 139.62 0:00:11 0:00:03
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━━━╺━ 1558 1 0.029 127 134.30 0:00:11 0:00:04
draws/s
━━━━━╺━ 1564 0 0.032 191 134.89 0:00:11 0:00:04
draws/s
━━━━━╸━ 1589 0 0.039 127 137.01 0:00:11 0:00:04
draws/s
━━━━━╸━ 1621 0 0.021 127 139.79 0:00:11 0:00:03
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━━━╸━ 1574 1 0.029 63 134.43 0:00:11 0:00:04
draws/s
━━━━━╸━ 1578 0 0.032 255 134.78 0:00:11 0:00:04
draws/s
━━━━━╸━ 1603 0 0.039 127 136.98 0:00:11 0:00:03
draws/s
━━━━━╸━ 1638 0 0.021 127 140.01 0:00:11 0:00:03
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━━━╸━ 1591 1 0.029 127 134.57 0:00:11 0:00:04
draws/s
━━━━━╸━ 1594 0 0.032 63 134.83 0:00:11 0:00:04
draws/s
━━━━━╸━ 1618 0 0.039 191 136.89 0:00:11 0:00:03
draws/s
━━━━━╸━ 1655 0 0.021 127 140.05 0:00:11 0:00:03
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━━━╸━ 1605 1 0.029 127 134.53 0:00:11 0:00:04
draws/s
━━━━━╸━ 1608 0 0.032 127 134.79 0:00:11 0:00:04
draws/s
━━━━━╸━ 1632 0 0.039 127 136.80 0:00:11 0:00:03
draws/s
━━━━━╸━ 1672 0 0.021 127 140.22 0:00:11 0:00:03
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━━━╸━ 1622 1 0.029 127 134.63 0:00:12 0:00:03
draws/s
━━━━━╸━ 1622 0 0.032 127 134.72 0:00:12 0:00:03
draws/s
━━━━━╸━ 1648 0 0.039 127 136.88 0:00:12 0:00:03
draws/s
━━━━━╸━ 1691 0 0.021 63 140.43 0:00:12 0:00:03
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━━━╸━ 1636 1 0.029 127 134.60 0:00:12 0:00:03
draws/s
━━━━━╸━ 1636 0 0.032 127 134.60 0:00:12 0:00:03
draws/s
━━━━━╸━ 1666 0 0.039 63 137.12 0:00:12 0:00:03
draws/s
━━━━━╸━ 1710 0 0.021 127 140.75 0:00:12 0:00:03
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━━━╸━ 1642 1 0.029 223 133.94 0:00:12 0:00:03
draws/s
━━━━━╸━ 1643 0 0.032 127 133.98 0:00:12 0:00:03
draws/s
━━━━━╸━ 1675 0 0.039 63 136.68 0:00:12 0:00:03
draws/s
━━━━━━╺ 1720 0 0.021 127 140.38 0:00:12 0:00:03
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━━━╸━ 1657 1 0.029 127 133.95 0:00:12 0:00:03
draws/s
━━━━━╸━ 1654 0 0.032 255 133.73 0:00:12 0:00:03
draws/s
━━━━━╸━ 1690 0 0.039 63 136.63 0:00:12 0:00:03
draws/s
━━━━━━╺ 1741 0 0.021 31 140.84 0:00:12 0:00:02
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━━━╸━ 1671 1 0.029 127 133.87 0:00:12 0:00:03
draws/s
━━━━━╸━ 1667 0 0.032 63 133.55 0:00:12 0:00:03
draws/s
━━━━━╸━ 1709 0 0.039 127 136.94 0:00:12 0:00:03
draws/s
━━━━━━╺ 1757 0 0.021 63 140.83 0:00:12 0:00:02
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━━━╸━ 1687 1 0.029 127 133.96 0:00:12 0:00:03
draws/s
━━━━━╸━ 1681 0 0.032 127 133.49 0:00:12 0:00:03
draws/s
━━━━━━╺ 1724 0 0.039 127 136.92 0:00:12 0:00:03
draws/s
━━━━━━╺ 1774 0 0.021 127 140.91 0:00:12 0:00:02
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━━━╸━ 1702 1 0.029 127 133.96 0:00:12 0:00:03
draws/s
━━━━━╸━ 1696 0 0.032 127 133.49 0:00:12 0:00:03
draws/s
━━━━━━╺ 1738 0 0.039 127 136.80 0:00:12 0:00:02
draws/s
━━━━━━╺ 1790 0 0.021 127 140.99 0:00:12 0:00:02
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━━━━╺ 1717 1 0.029 127 134.01 0:00:12 0:00:03
draws/s
━━━━━╸━ 1711 0 0.032 127 133.53 0:00:12 0:00:03
draws/s
━━━━━━╺ 1756 0 0.039 63 137.04 0:00:12 0:00:02
draws/s
━━━━━━╺ 1806 0 0.021 63 141.01 0:00:12 0:00:02
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━━━━╺ 1731 1 0.029 127 133.99 0:00:12 0:00:03
draws/s
━━━━━━╺ 1726 0 0.032 127 133.53 0:00:12 0:00:03
draws/s
━━━━━━╺ 1771 0 0.039 127 137.06 0:00:12 0:00:02
draws/s
━━━━━━╺ 1819 0 0.021 127 140.82 0:00:12 0:00:02
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━━━━╺ 1742 1 0.029 127 133.73 0:00:13 0:00:03
draws/s
━━━━━━╺ 1739 0 0.032 127 133.39 0:00:13 0:00:03
draws/s
━━━━━━╺ 1787 0 0.039 127 137.09 0:00:13 0:00:02
draws/s
━━━━━━╺ 1835 0 0.021 127 140.83 0:00:13 0:00:02
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━━━━╺ 1758 1 0.029 127 133.67 0:00:13 0:00:02
draws/s
━━━━━━╺ 1755 0 0.032 31 133.49 0:00:13 0:00:02
draws/s
━━━━━━╺ 1805 0 0.039 127 137.28 0:00:13 0:00:02
draws/s
━━━━━━╺ 1853 1 0.021 68 140.95 0:00:13 0:00:02
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━━━━╺ 1766 1 0.029 63 133.19 0:00:13 0:00:02
draws/s
━━━━━━╺ 1763 0 0.032 31 132.94 0:00:13 0:00:02
draws/s
━━━━━━╺ 1814 0 0.039 63 136.92 0:00:13 0:00:02
draws/s
━━━━━━╸ 1863 1 0.021 127 140.51 0:00:13 0:00:01
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━━━━╺ 1779 1 0.029 127 133.09 0:00:13 0:00:02
draws/s
━━━━━━╺ 1779 0 0.032 127 133.08 0:00:13 0:00:02
draws/s
━━━━━━╺ 1831 0 0.039 127 136.97 0:00:13 0:00:02
draws/s
━━━━━━╸ 1880 1 0.021 127 140.66 0:00:13 0:00:01
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━━━━╺ 1793 1 0.029 127 133.04 0:00:13 0:00:02
draws/s
━━━━━━╺ 1795 0 0.032 63 133.17 0:00:13 0:00:02
draws/s
━━━━━━╺ 1847 0 0.039 127 137.08 0:00:13 0:00:02
draws/s
━━━━━━╸ 1894 1 0.021 127 140.57 0:00:13 0:00:01
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━━━━╺ 1807 1 0.029 127 132.99 0:00:13 0:00:02
draws/s
━━━━━━╺ 1810 0 0.032 127 133.25 0:00:13 0:00:02
draws/s
━━━━━━╸ 1863 0 0.039 63 137.17 0:00:13 0:00:02
draws/s
━━━━━━╸ 1907 1 0.021 95 140.43 0:00:13 0:00:01
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━━━━╺ 1823 1 0.029 127 133.11 0:00:13 0:00:02
draws/s
━━━━━━╺ 1823 0 0.032 127 133.11 0:00:13 0:00:02
draws/s
━━━━━━╸ 1876 0 0.039 127 137.05 0:00:13 0:00:01
draws/s
━━━━━━╸ 1924 1 0.021 63 140.57 0:00:13 0:00:01
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━━━━╺ 1836 1 0.029 127 133.02 0:00:13 0:00:02
draws/s
━━━━━━╺ 1838 0 0.032 127 133.16 0:00:13 0:00:02
draws/s
━━━━━━╸ 1892 0 0.039 127 137.10 0:00:13 0:00:01
draws/s
━━━━━━╸ 1939 1 0.021 127 140.54 0:00:13 0:00:01
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━━━━╺ 1849 1 0.029 127 132.94 0:00:13 0:00:02
draws/s
━━━━━━╺ 1855 0 0.032 63 133.39 0:00:13 0:00:02
draws/s
━━━━━━╸ 1905 0 0.039 63 136.99 0:00:13 0:00:01
draws/s
━━━━━━╸ 1955 1 0.021 127 140.58 0:00:13 0:00:01
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━━━━╸ 1863 1 0.029 127 132.89 0:00:14 0:00:02
draws/s
━━━━━━╸ 1871 0 0.032 63 133.46 0:00:14 0:00:02
draws/s
━━━━━━╸ 1922 0 0.039 127 137.15 0:00:14 0:00:01
draws/s
━━━━━━╸ 1969 1 0.021 127 140.58 0:00:14 0:00:01
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━━━━╸ 1878 1 0.029 127 132.88 0:00:14 0:00:01
draws/s
━━━━━━╸ 1885 0 0.032 127 133.43 0:00:14 0:00:01
draws/s
━━━━━━╸ 1940 0 0.039 127 137.27 0:00:14 0:00:01
draws/s
━━━━━━╸ 1982 1 0.021 127 140.27 0:00:14 0:00:01
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━━━━╸ 1886 1 0.029 127 132.35 0:00:14 0:00:01
draws/s
━━━━━━╸ 1895 0 0.032 127 133.09 0:00:14 0:00:01
draws/s
━━━━━━╸ 1951 0 0.039 63 137.05 0:00:14 0:00:01
draws/s
━━━━━━╸ 1991 1 0.021 127 139.79 0:00:14 0:00:01
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━━━━╸ 1894 1 0.029 127 132.32 0:00:14 0:00:01
draws/s
━━━━━━╸ 1904 0 0.032 127 133.01 0:00:14 0:00:01
draws/s
━━━━━━╸ 1962 0 0.039 63 137.12 0:00:14 0:00:01
draws/s
━━━━━━━ 2000 1 0.021 63 139.71 0:00:14 0:00:00
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━━━━╸ 1901 1 0.029 127 132.34 0:00:14 0:00:01
draws/s
━━━━━━╸ 1911 0 0.032 127 133.06 0:00:14 0:00:01
draws/s
━━━━━━╸ 1972 0 0.039 127 137.32 0:00:14 0:00:01
draws/s
━━━━━━━ 2000 1 0.021 63 139.71 0:00:14 0:00:00
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━━━━╸ 1917 1 0.029 127 132.43 0:00:14 0:00:01
draws/s
━━━━━━╸ 1927 0 0.032 95 133.10 0:00:14 0:00:01
draws/s
━━━━━━╸ 1989 0 0.039 63 137.45 0:00:14 0:00:01
draws/s
━━━━━━━ 2000 1 0.021 63 139.71 0:00:14 0:00:00
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━━━━╸ 1926 1 0.029 31 132.56 0:00:14 0:00:01
draws/s
━━━━━━╸ 1935 0 0.032 127 133.20 0:00:14 0:00:01
draws/s
━━━━━━━ 2000 0 0.039 127 137.64 0:00:14 0:00:00
draws/s
━━━━━━━ 2000 1 0.021 63 139.71 0:00:14 0:00:00
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━━━━╸ 1934 1 0.029 127 132.60 0:00:14 0:00:01
draws/s
━━━━━━╸ 1943 0 0.032 127 133.21 0:00:14 0:00:01
draws/s
━━━━━━━ 2000 0 0.039 127 137.64 0:00:14 0:00:00
draws/s
━━━━━━━ 2000 1 0.021 63 139.71 0:00:14 0:00:00
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━━━━╸ 1951 1 0.029 127 132.81 0:00:14 0:00:01
draws/s
━━━━━━╸ 1960 0 0.032 127 133.41 0:00:14 0:00:01
draws/s
━━━━━━━ 2000 0 0.039 127 137.64 0:00:14 0:00:00
draws/s
━━━━━━━ 2000 1 0.021 63 139.71 0:00:14 0:00:00
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━━━━╸ 1968 1 0.029 127 132.97 0:00:14 0:00:01
draws/s
━━━━━━╸ 1977 0 0.032 63 133.59 0:00:14 0:00:01
draws/s
━━━━━━━ 2000 0 0.039 127 137.64 0:00:14 0:00:00
draws/s
━━━━━━━ 2000 1 0.021 63 139.71 0:00:14 0:00:00
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━━━━╸ 1980 1 0.029 127 132.78 0:00:14 0:00:01
draws/s
━━━━━━╸ 1991 0 0.032 63 133.60 0:00:14 0:00:01
draws/s
━━━━━━━ 2000 0 0.039 127 137.64 0:00:14 0:00:00
draws/s
━━━━━━━ 2000 1 0.021 63 139.71 0:00:14 0:00:00
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━━━━╸ 1991 1 0.029 127 132.98 0:00:14 0:00:01
draws/s
━━━━━━━ 2000 0 0.032 95 133.56 0:00:14 0:00:00
draws/s
━━━━━━━ 2000 0 0.039 127 137.64 0:00:14 0:00:00
draws/s
━━━━━━━ 2000 1 0.021 63 139.71 0:00:14 0:00:00
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━━━━╸ 1999 1 0.029 63 133.12 0:00:15 0:00:01
draws/s
━━━━━━━ 2000 0 0.032 95 133.56 0:00:14 0:00:00
draws/s
━━━━━━━ 2000 0 0.039 127 137.64 0:00:14 0:00:00
draws/s
━━━━━━━ 2000 1 0.021 63 139.71 0:00:14 0:00:00
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━━━━━ 2000 1 0.029 127 133.12 0:00:15 0:00:00
draws/s
━━━━━━━ 2000 0 0.032 95 133.56 0:00:14 0:00:00
draws/s
━━━━━━━ 2000 0 0.039 127 137.64 0:00:14 0:00:00
draws/s
━━━━━━━ 2000 1 0.021 63 139.71 0:00:14 0:00:00
draws/s
Step Grad
Progre… Draw Diverg… size evals Speed Elapsed Remaini…
──────────────────────────────────────────────────────────────────────────────
━━━━━━━ 2000 1 0.029 127 133.12 0:00:15 0:00:00
draws/s
━━━━━━━ 2000 0 0.032 95 133.56 0:00:14 0:00:00
draws/s
━━━━━━━ 2000 0 0.039 127 137.64 0:00:14 0:00:00
draws/s
━━━━━━━ 2000 1 0.021 63 139.71 0:00:14 0:00:00
draws/s
?25hSampling 4 chains for 1_000 tune and 1_000 draw iterations (4_000 + 4_000 draws total) took 15 seconds.
There were 2 divergences after tuning. Increase `target_accept` or reparameterize.
Sampling: [beta, y_hat, y_hat_sigma]
Sampling: [y_hat]
Sampling: [y_hat]
Sampling: [y_hat]
Sampling: [y_hat]
If we wanted to take a closer look to diagnose the sampling process we could do the following:
az.summary(result.idata, round_to=2)
az.plot_trace(result.idata, var_names=["~mu"], compact=False);
Results#
Let’s use Arviz examine the posterior parameter estimates for each of the beta weightings for each country, along with the estimate of the measurement standard deviation, sigma.
az.plot_forest(result.idata, var_names=["~mu"], figsize=(8, 3), combined=True);
If you are new to Bayesian methods, then I would recommend checking out these resources:
The PyMC website pymc.io, especially the examples page.
There are also a whole bunch of video resources including:
We can now use the plot method of the result object that we get back from CausalPy. This will give us a pretty detailed visual output.
fig, ax = result.plot(plot_predictors=False)
# formatting
for i in [0, 1, 2]:
ax[i].set(ylabel="Sales (thousands)")
Nice! By creating a simple model formula and one call to CausalPy, we have been able to evaluate the lift generated in the treated unit.
In this example, there is quite a lot of measurement noise, but because we are using Bayesian inference methods here, we have a precise and principled quantification in our uncertainty.
We can see that the Bayesian \(R^2\) value for the pre-treatment data is about 0.5. This is not excellent, but pretty good for real-world data. It shows that the linear weighted combination model (the core of synthetic control) is doing a reasonable job at constructing a faux (i.e. synthetic) Denmark up until the treatment period.
This synthetic control Denmark is our estimated counterfactual - what the sales would have been if the store renovation project had not been carried out. By looking at the difference we can estimate the causal impact, or we could also call it ‘geographical lift’.
Over the year since implementation, we can see that the cumulative causal impact of sales in Denmark was close to 10,000 units. Let’s examine that in more detail. Below we look at the posterior distribution of the cumulative causal impact at the end of our time series, after the scheme had been in place for 1 year.
# get index of the final time point
index = result.post_impact_cumulative.obs_ind.max()
# grab the posterior distribution of the cumulative impact at this final time point
last_cumulative_estimate = result.post_impact_cumulative.sel({"obs_ind": index})
# get summary stats
ax = az.plot_posterior(last_cumulative_estimate, figsize=(8, 4))
ax.set(
title="Estimated cumulative causal impact (at end of 2022)",
xlabel="Sales (thousands)",
);
If we want, we can also extract these statistics out numerically:
az.summary(last_cumulative_estimate, kind="stats")
| mean | sd | hdi_3% | hdi_97% | |
|---|---|---|---|---|
| x[Denmark] | 11.1 | 0.4 | 10.34 | 11.83 |
Effect Summary Reporting#
For a decision-ready summary, you can use the effect_summary() method which provides a concise report with average and cumulative effects, HDI intervals, tail probabilities, and relative effects. This provides a comprehensive summary without manual post-processing.
# Generate effect summary for the full post-period
stats = result.effect_summary(treated_unit="Denmark")
stats.table
| mean | median | hdi_lower | hdi_upper | p_gt_0 | relative_mean | relative_hdi_lower | relative_hdi_upper | |
|---|---|---|---|---|---|---|---|---|
| average | 0.21 | 0.21 | 0.20 | 0.23 | 1.0 | 10.2 | 9.42 | 10.99 |
| cumulative | 11.10 | 11.10 | 10.32 | 11.88 | 1.0 | 10.2 | 9.42 | 10.99 |
print(stats.text)
During the Post-period (2022-01-02 00:00:00 to 2022-12-25 00:00:00), the response variable had an average value of approx. 2.31. By contrast, in the absence of an intervention, we would have expected an average response of 2.09. The 95% interval of this counterfactual prediction is [2.08, 2.11]. Subtracting this prediction from the observed response yields an estimate of the causal effect the intervention had on the response variable. This effect is 0.21 with a 95% interval of [0.20, 0.23].
Summing up the individual data points during the Post-period, the response variable had an overall value of 119.90. By contrast, had the intervention not taken place, we would have expected a sum of 108.81. The 95% interval of this prediction is [108.03, 109.58].
The 95% HDI of the effect [0.20, 0.23] does not include zero. The posterior probability of an increase is 1.000. Relative to the counterfactual, the effect represents a 10.20% change (95% HDI [9.42%, 10.99%]).
This analysis assumes that the control units used to construct the synthetic counterfactual were not themselves affected by the intervention, and that the pre-treatment relationship between control and treated units remains stable throughout the post-treatment period. We recommend inspecting model fit, examining pre-intervention trends, and conducting sensitivity analyses (e.g., placebo tests) to support any causal conclusions drawn from this analysis.
So at the end of our causal modelling endeavours we can report to our boss something along the lines of: “We believe that the store refurbishment scheme was causally responsible for driving a total of about 11,100 additional sales. But we have uncertainty in the exact number of additional sales - our 94% credible regions span 10,310 to 11,920”.
There are of course caveats worth bearing in mind. The analysis we’ve conducted has assumed that the only major event that might selectively influence sales in Denmark was the store renovation project. If this is a reasonable assumption then we may be on relatively stable ground in making causal claims. But if there were other events which selectively effected some units (countries) and not others, then we may need to be much more cautious in our claims and resort to more in-depth modelling approaches.
But our estimated cumulative causal impact of \(11100^{11920}_{10310}\) is exactly the information needed by others in the company. They can use this figure (and even the uncertainty associated with it) and estimate how long it would take for the cost of renovating other stores to result in a net profit.
Your boss is very happy. You get a big end-of-year bonus.
Beyond GeoLift#
This example used geographical regions as the treatment units, but there is no requirement for units to be geographical regions. For example, your units could be products. Maybe your company sells many different products and one (or a few of these) are chosen to be discounted. Did this intervention of discounting the price causally increase sales volumes? Synthetic control methods can answer this (and similar questions) just as easily.
We can get nicely formatted tables from our integration with the maketables package.
from maketables import ETable
ETable(result, coef_fmt="b:.3f")
| y | |
|---|---|
| (1) | |
| coef | |
| Austria | 0.034 |
| Belgium | 0.433 |
| Bulgaria | 0.131 |
| Croatia | 0.055 |
| Cyprus | 0.106 |
| Czech_Republic | 0.059 |
| Estonia | 0.079 |
| Finland | 0.075 |
| Hungary | 0.029 |
| stats | |
| N | 208 |
| Bayesian R2 | 0.943 |
| Format of coefficient cell: Coefficient | |