From inputs to prediction: how a neural network computes

Neural networks power image recognition, translation and many recommendation systems. They can sound mysterious, but the core calculation is simple arithmetic repeated many times. Once you can trace one prediction by hand, the rest of this course will make sense.

The artificial neuron

An artificial neuron takes several numbers as input. It multiplies each input by a weight, adds the results together, and then adds one more number called the bias. In symbols, for inputs x1, x2 and weights w1, w2, the neuron computes z = w1*x1 + w2*x2 + b.

A weight says how strongly an input pushes the result up or down. The bias shifts the whole result, so the neuron can produce a useful output even when all inputs are zero.

Next, the neuron passes z through an activation function, f, to get its output a = f(z). The activation adds a bend, or nonlinearity, to the calculation.

Common activation functions:

• ReLU: f(z) = max(0, z). Negative values become 0; positive values pass through.
• Sigmoid: f(z) = 1/(1 + e^(-z)). Squeezes any number into the range 0 to 1.
• Tanh: squeezes any number into the range -1 to 1.

Stacking neurons into layers

Neurons are arranged in layers. The input layer holds the raw data, such as pixel values or measurements. One or more hidden layers transform it. The output layer gives the final prediction. In a fully connected (dense) layer, every neuron receives every output from the layer before.

Computing from inputs to output, layer by layer, is called the forward pass. Each layer's outputs become the next layer's inputs.

Here is the same calculation in Python. Matrix multiplication (the @ symbol) does all the weighted sums for a layer at once.

import numpy as np

def relu(z):
    return np.maximum(0, z)

def sigmoid(z):
    return 1 / (1 + np.exp(-z))

x = np.array([2.0, 1.0])
W1 = np.array([[0.5, -1.0], [1.0, 1.0]])
b1 = np.array([0.5, -4.0])
h = relu(W1 @ x + b1)      # [0.5, 0.0]

W2 = np.array([2.0, 3.0])
b2 = -0.5
y = sigmoid(W2 @ h + b2)   # about 0.62
print(y)

Why the activation matters

Without activation functions, every layer would only multiply and add. Stacking such linear layers still gives one linear function overall, so extra layers would add nothing. The nonlinear activations let a network represent curved, complicated relationships between inputs and outputs.

Next time you see a network diagram, trace one path: weighted sum, add bias, apply activation, pass it on. That loop is the whole forward pass.

Key takeaways

• Each neuron computes a weighted sum of its inputs, adds a bias, and applies an activation function.
• The forward pass sends data layer by layer from inputs to the output prediction.
• Nonlinear activations are what let stacked layers model complex patterns.

Check your understanding

Question 1 of 3

A neuron has inputs 3 and 2, weights 1 and -2, bias 4, and uses ReLU. What is its output?

Choose an answer

03-111

Check answer

Question 2 of 3

Why do neural networks use nonlinear activation functions?

Choose an answer

To make the forward pass fasterTo keep all outputs between 0 and 1So that stacked layers can represent more than a single linear functionBecause weights cannot be negative otherwise

Check answer

Question 3 of 3

What is the forward pass?

Choose an answer

Computing outputs layer by layer from the inputs to the predictionAdjusting the weights to reduce errorChoosing how many layers a network should haveRemoving neurons that output zero

Check answer