Introduction to neural networks

Terry Benzschawel

A neural network is a method in artificial intelligence that teaches computers to process data in a way that is inspired by the human brain. For example, Figure 5.1 shows a diagram of the retina of the human eye and a corresponding computer system. The mathematical model for colour vision is designed to mimic the neurons in the brain and to process light energy analogously. Before introducing networks, this chapter explores the development of the artificial neuron and its functions. In addition, neurons’ activation functions are described along with their inclusion in multilayer perceptrons. Finally, the method of training neural networks using error back-propagation is introduced.

5.1 INTRODUCTION AND EARLY HISTORY

Recall from Chapter 3 the artificial neuron introduced by McCulloch and Pitts. That neuron had binary (0 or 1) inputs that were summed and measured against a threshold for a binary output signal. The McCulloch–Pitts neuron, shown in Figure 5.2, can be represented formally as:

$$g\left( x_1,x_2,x_3,\dots ,x_n\right)=g(x)=\underset{i=1 }{\overset{n}{{\sum }^{\text{}}}}{x}_i,y=f(g(x))=\{ \begin{array}{c}1,g(x)\ge \theta \\ 0,g(x)<\theta \end{array}(5.1)$$

In Equation 5.1 the function g(x) is a summation of the inputs, x_i, and θ is called the