PyTorch - Linear Regression

In this chapter, we will be focusing on basic example of linear regression implementation using PyTorch. Logistic regression or linear regression is a supervised machine learning approach for the classification of order discrete categories. Our goal in this chapter is to build a model by which a user can predict the relationship between predictor variables and one or more independent variables.

The relationship between these two variables is considered linear i.e., if y is the dependent variable and x is considered as the independent variable, then the linear regression relationship of two variables will look like the equation which is mentioned as below −

$Y = Ax+b$

Next, we shall design an algorithm for linear regression which allows us to understand two important concepts given below −

Two Concepts

• Cost Function
• Gradient Descent Algorithms

The schematic representation of linear regression is mentioned below

Interpreting the result

$$Y=ax+b$$

Info

• The value of $a$ is the slope.
• The value of $b$ is the y − intercept.
• $r$ is the correlation coefficient.
• $r^2$ is the correlation coefficient.

The graphical view of the equation of linear regression is mentioned below −

Following steps are used for implementing linear regression using PyTorch −

Step 1: Import Necessary Packages

Import the necessary packages for creating a linear regression in PyTorch using the below code −

import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
import pandas as pd
import torch
import torch.nn as nn
from torch.autograd import Variable

Step 2: Prepare Dataset

Create a single training set with the available data set as shown below −

# setup plot style
sns.set_style(style='whitegrid')
plt.rcParams["patch.force_edgecolor"] = True

# generating data
m = 2  # slope
c = 3  # intercept
x = np.random.rand(256)
noise = np.random.randn(256) / 4
y = x * m + c + noise

# adjust dataset
df = pd.DataFrame()
df['x'] = x
df['y'] = y
sns.lmplot(x='x', y='y', data=df)

Step 3: Initialize model

Implement linear regression with PyTorch libraries as mentioned below −

# convert data to pytorch type
x_train = x.reshape(-1, 1).astype('float32')
y_train = y.reshape(-1, 1).astype('float32')

# define linear model
class LinearRegressionModel(nn.Module):
    def __init__(self, input_dim, output_dim):
        super(LinearRegressionModel, self).__init__()
        self.linear = nn.Linear(input_dim, output_dim)

    def forward(self, x):
        return self.linear(x)

# plot function
def plot_current_fit(title=""):
    plt.figure(figsize=(12, 4))
    plt.title(title)
    plt.scatter(x, y, s=8)
    w1 = w.data.item()  # get weights
    b1 = b.data.item()  # get bias
    x1 = np.array([0., 1.])
    y1 = x1 * w1 + b1
    plt.plot(x1, y1, 'r', label=f'Current Fit ({w1:.3f}, {b1:.3f})')
    plt.xlabel('x (input)')
    plt.ylabel('y (target)')
    plt.legend()
    plt.show()


# initilize model
input_dim = 1
output_dim = 1
model = LinearRegressionModel(input_dim, output_dim)

# define loss and optimizer
criterion = nn.MSELoss()
optimizer = torch.optim.SGD(model.parameters(), lr=0.01)

# get model parameters
[w, b] = model.parameters()

# draw the pretraining data
plot_current_fit('Before training')

Step 4: Training Model

# Training 
num_epochs = 1000
for epoch in range(num_epochs):
    inputs = Variable(torch.from_numpy(x_train))
    targets = Variable(torch.from_numpy(y_train))

    # forward spread
    outputs = model(inputs)
    loss = criterion(outputs, targets)

    # backward spread and optimization
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()

    # print loss
    if (epoch+1) % 100 == 0:
        print(f'Epoch [{epoch+1}/{num_epochs}], Loss: {loss.item():.4f}')

# plot the training data
plot_current_fit('After training')

References

PyTorch - Loading Data
My Practice