Loud And Proud: Verbose In Machine Learning

In machine learning, there are two types: those who like to keep things short and sweet and those who want to explain everything in detail.

I fall into the latter category – I love verbosity.

Some might call it overkill, but I see it as a way to ensure no stone is left unturned.

In this guide, we will explore the ins and outs of verbose in machine learning, including when it should be used and how to implement it correctly in your models.

By the end, you’ll know the following:

What Verbosity is
Understanding The Output From Verbose
Setting Up Verbose with Two Famous Machine Learning Algos
When You Should and Shouldn’t use Verbose Settings

Table of Contents show

What is Verbose in Machine Learning?

In machine learning, “verbose” refers to a particular setting used when training and validating models.

When verbose is turned on, the algorithm will provide more detailed information about its progress as your model iterates through the training process.

It’ll push this output right to your console!

This can be useful for:

Debugging
Error Finding
Understanding your Models progression with offline metrics
Early Stopping In Deep Learning

As a warning, the verbose output can sometimes slow down the training process since printing output to your console is much slower than just running the model.

I like to run the algorithm without any verbose setting output and only use it if there are problems with the results.

Realize in most standard modeling packages; there are different “levels” of verbosity.

For example, here are the levels for the famous Sklearn package.

We will use the GridsearchCV for this example:

Setting Verbose = 0

Silent Modeling!

Setting Verbose = 1

This will display the computation time for each fold and the parameter candidate.

Setting Verbose = 2:

This will display everything from 1, and the score will also be displayed;

Setting Verbose = 3:

This will display everything from 1 and 2, along with candidate parameter indexes and the computation time.

This will be slightly different for each model we choose, but we get the general gist: as verbosity increases, we get more output information to our console.

Reference: https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.GridSearchCV.html

Understanding Verbose Output Within Data Science

One way to approach verbose output from your models is to break it down into smaller chunks.

If you’re looking at a massive block of text, try focusing on one section at a time.

Before you go through each word of your log, you should understand what value you’re trying to optimize (score, computation time, etc.) and focus mainly on that.

What I like to do is to look for patterns in the output. Watching as the numbers increase or decrease is usually much more helpful than the EXACT number at that EXACT time.

Verbosity settings are to be used as a guide, and if you come at your modeling process with a goal in mind, verbosity can help you get to the finish line.

Below, we’ll explain how to set up the verbose setting in python with some famous models so you can use it in any situation.

How To Set Up Verbose in XGBoost Models

For our XGBoost model, we only changed the verbosity setting from 1 to 3.

In the below models, we used the “verbose” setting, but in XGBoost, this setting is called “verbosity.”

Here is the code we used,

from sklearn.datasets import load_wine
import xgboost as xgb
import numpy as np
from sklearn.metrics import mean_squared_error

wine_df = load_wine()

X = wine_df.data
y = wine_df.target

xgb_model = xgb.XGBRegressor(objective="reg:squarederror", random_state=42, verbosity=1)

xgb_model.fit(X, y)

y_pred = xgb_model.predict(X)

mse=mean_squared_error(y, y_pred)

print(f'\n\nModel Mean squared error {np.sqrt(mse)}')

We can see that our model is silent when verbosity is set to 1.

When increasing this to 2, we see much more output.

Finally, once we’ve passed 3 into our model, we see everything from 2, plus some final metrics and KPIs on model performance.

How To Set Up Verbose in Scikit Learn Models

Here is our code; we only changed the verbose setting in our gradient-boosted model to get the images below.

from sklearn.datasets import load_wine
import xgboost as xgb
import numpy as np
from sklearn.metrics import mean_squared_error

wine_df = load_wine()

X = wine_df.data
y = wine_df.target

xgb_model = xgb.XGBRegressor(objective="reg:squarederror", random_state=42, verbosity=1)

xgb_model.fit(X, y)

y_pred = xgb_model.predict(X)

mse=mean_squared_error(y, y_pred)

print(f'\n\nModel Mean squared error {np.sqrt(mse)}')

With Verbose set to 0, we see that our model is “silent.”

When we upgrade this to 1, we see gaps in our iterations.

Finally, when we push this to 2, we see a complete breakdown from each iteration of our model.

How To Set Up Verbose in Deep Learning (Keras) Models

Here is the code we used for our Deep learning example, only changing the verbose setting in the .fit method.

from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense
from sklearn.datasets import load_wine

wine_df = load_wine()

X = wine_df.data
y = wine_df.target

model = Sequential()
model.add(Dense(12, input_shape=(X.shape[1],), activation='relu'))
model.add(Dense(8, activation='relu'))
model.add(Dense(1, activation='sigmoid'))

model.compile(loss='mean_squared_error', optimizer='adam', metrics=['accuracy'])

model.fit(X, y, epochs=150, batch_size=10, verbose=1)

_, accuracy = model.evaluate(X, y)
print('Accuracy: %.2f' % (accuracy*100))

When verbose is set to 0, our model is silent.

When this is pushed up to 1, we only get an update after each epoch.

Finally, when this is set to 2, we get an output showing each iteration in each epoch.

When Should You Use Verbose Settings in Machine Learning?

Verbosity will give you a ton of information while building out your models.

This can be super helpful when you’re in the fine-tuning stage or trying to dive deep into a problem you need help with.

Since most verbosity settings are in levels, you can choose a level that gives you the amount of output you need.

When Do We Turn Off Verbose?

When working in data science and on models, it’s critical to strike the right balance between training speed and model performance.

Your boss needs to see results, but they also need to see the right results.

If you build inaccurate models, nobody will want them – but if you never get models out the door, you’ll find yourself needing a new job.

One way to strike this balance is to be selective with the verbose parameter.

You can start your modeling without verbose turned on, and if you do not see the results you want, turn it on, as it can help you dive deeper into your model and find areas that need improvement.

Loud and Proud: Verbose in Machine Learning