fully connected layer
A fully connected (dense) layer is a layer in which every input value influences every output value. Imagine a complete bipartite wiring between inputs and outputs: with n inputs and m outputs there are n times m connections, each with its own weight. It is the most general, least-constrained layer; it makes no assumption about how the inputs relate to one another, which is both its flexibility and its cost.
The layer is just a matrix-vector product plus a bias, optionally followed by an activation: y = phi(Wx + b), where x in R^n is the input, W in R^(m x n) the weight matrix, b in R^m the bias vector, and y in R^m the output. Entry W_(ij) is the weight from input j to output i, and row i of W together with b_i is exactly one artificial neuron. The layer has n·m + m learnable parameters. Operating on a batch of B inputs stacked as a matrix X in R^(B x n), the computation is a single matrix multiply X·W^T + b, which is why GPUs accelerate these layers so well.
Because every input touches every output, the parameter count and compute grow as the product of the dimensions, which is ruinous for images: a 224 by 224 by 3 image flattened is 150,528 inputs, and one modest 1,000-unit dense layer over it would need about 150 million weights. A convolutional layer instead reuses one small filter across all spatial positions (weight sharing) and connects each output only to a local neighborhood (sparse connectivity), slashing parameters and building in translation equivariance. This contrast is the core reason CNNs, not dense nets, dominate pixel-level vision.
Fully connected layers survive at the ends of vision models (the classifier head mapping pooled features to class logits is dense) and inside the feed-forward sub-block of every Transformer (the two dense layers around the nonlinearity). A 1x1 convolution is mathematically a fully connected layer applied independently at each spatial location, a useful identity for channel mixing. Knowing where dense layers belong (after spatial structure has been summarized) versus where convolutions belong (while spatial structure still matters) is a basic architectural skill.
Pitfall: the transition from a convolutional feature map to a dense head requires flattening or global pooling. Global average pooling (used since GoogLeNet and ResNet) is usually preferred over flattening because it is parameter-free, removes the input-size dependence, and reduces overfitting.