PyTorch: Multi-Layer Fully Connected Neural Network
Introduction
Deep learning has become a crucial technology in science and technology, and PyTorch is a popular framework for building and training deep learning models. In this chapter, we will introduce the basics of PyTorch, including tensor operations, variable calculations, and data processing. We will also cover linear regression, logistic regression, and multi-layer fully connected neural networks.
3.1 Warm-up: PyTorch Basics
Before diving into complex deep learning concepts, we need to understand the basics of PyTorch. In this section, we will cover the following topics:
3.1.1 Tensor (Tensor)
The most basic operation in PyTorch is the tensor. A tensor is a multi-dimensional matrix, similar to a NumPy array. PyTorch tensors can be created with different data types, such as 32-bit floating-point numbers (torch.FloatTensor) or 64-bit floating-point numbers (torch.DoubleTensor). We can also create tensors with specific shapes and sizes.
a = torch.Tensor([[2, 3], [4, 8], [7, 9]])
print('a is: {}'.format(a))
print('a size is {}'.format(a.size()))
We can also create tensors with specific data types, such as 32-bit integers (torch.IntTensor) or 64-bit integers (torch.LongTensor).
b = torch.LongTensor([[2, 3], [4, 8], [7, 9]])
print('b is: {}'.format(b))
We can also create tensors with random initial values.
c = torch.zeros((3, 2))
print('zero tensor: {}'.format(c))
d = torch.randn((3, 2))
print('normal random tensor: {}'.format(d))
3.1.2 Variable (Variable)
In PyTorch, a variable is an object that represents a tensor with automatic differentiation. This means that when we compute the gradient of a variable, PyTorch will automatically compute the gradients of all variables in the computation graph.
x = Variable(torch.Tensor([1]), requires_grad=True)
w = Variable(torch.Tensor([2]), requires_grad=True)
b = Variable(torch.Tensor([3]), requires_grad=True)
y = w * x + b
y.backward()
print(x.grad)
print(w.grad)
print(b.grad)
In this example, we create three variables (x, w, and b) and compute the gradient of y with respect to x, w, and b.
3.1.3 Dataset (Data Set)
Before processing any machine learning problem, we need to read and preprocess the data. PyTorch provides a simple way to read and preprocess data using the torch.utils.data.Dataset class.
class MyDataset(Dataset):
def __init__(self, csv_file, txt_file, root_dir, other_file):
self.csv_data = pd.read_csv(csv_file)
with open(txt_file, 'r') as f:
data_list = f.readlines()
self.txt_data = data_list
self.root_dir = root_dir
def __len__(self):
return len(self.csv_data)
def __getitem__(self, idx):
data = (self.csv_data[idx], self.txt_data[idx])
return data
In this example, we define a custom dataset class that reads a CSV file and a text file.
3.1.4 Data Loader (DataLoader)
PyTorch provides a simple way to create a data loader using the torch.utils.data.DataLoader class.
dataiter = DataLoader(MyDataset, batch_size=32, shuffle=True, collate_fn=default_collate)
In this example, we create a data loader that reads the custom dataset class and shuffles the data.
3.1.5 Image Folder (ImageFolder)
PyTorch provides a simple way to read and preprocess image data using the torchvision.datasets.ImageFolder class.
dset = ImageFolder(root='root_path', transform=None, loader=default_loader)
In this example, we create an image folder dataset that reads images from a directory.
These are the basics of PyTorch, including tensor operations, variable calculations, and data processing. In the next section, we will cover linear regression, logistic regression, and multi-layer fully connected neural networks.