How a Camera Makes an Image: Lenses, Sensors and Pixels

From Light to a Grid of Numbers

In the first article of this path we said that a computer does not see a photograph the way you do. It sees a grid of numbers, one or more numbers per tiny square called a pixel, and every computer vision technique we will cover later in this path works on that grid rather than on the scene itself. That raised an obvious question we left unanswered: how does a photograph of a cat, a road or your own face turn into a grid of numbers in the first place? That is the whole subject of this article.

Understanding this pipeline is not just trivia. When a photo looks grainy in low light, when a webcam feed is blurry, when colours look slightly off, or when a small image loses detail after resizing, the cause usually traces back to one specific stage between the lens and the pixel grid. Knowing those stages makes those problems easy to diagnose instead of mysterious.

The Journey Light Takes Through a Camera

Every digital camera, from a phone to a dedicated webcam to a DSLR, turns a scene into a stored image through roughly the same sequence of physical and electronic steps. Light bounces off objects in the world, travels toward the camera, and is reshaped and measured several times before it becomes the file or video frame you eventually load with OpenCV.

• Light leaves the scene, reflected or emitted from every object in view, travelling in all directions.
• The lens gathers some of that light and bends it so it converges into a sharp, focused image.
• The sensor, a grid of light-sensitive photosites, turns the arriving light into tiny electrical charges.
• A colour filter sitting in front of the sensor lets each photosite see mostly red, green or blue light.
• An analogue-to-digital converter (ADC) measures each charge and rounds it to a whole number.
• Onboard processing fills in missing colour information and produces the final pixel grid you can save or display.

The rest of this article goes through these stages one at a time, starting with the lens.

The Lens: Focusing Light Into an Image

Imagine a closed box with no lens at all, just a tiny pinhole in one side and a sensor on the opposite inner wall. Light from a point in the scene, say the tip of a candle flame, travels in straight lines in every direction, but only the thin bundle of rays that happens to line up with the pinhole makes it through. That narrow bundle lands on one spot on the sensor, and light from a different point in the scene lands on a different spot. The result is a real, if dim and slightly blurry, image. This is a pinhole camera, and it is the simplest possible way to form an image: no glass, no curves, just geometry.

A pinhole has two problems. Make the hole very small and the image is sharp but almost no light gets through, so you need an extremely long exposure to gather enough light. Make the hole bigger to let in more light and rays from each scene point spread across a larger area of the sensor, so the image gets blurrier. A lens solves both problems at once. Instead of a tiny hole, a lens uses a much larger curved piece of glass or plastic that bends every ray from a given scene point back together at a single point on the sensor, no matter which part of the lens the ray passed through. You get both more light and a sharp image, which is why every practical camera uses a lens rather than a pinhole.

The focal length of a lens describes how strongly it bends light, and it determines the field of view, meaning how wide a slice of the scene fits into the frame. A short focal length (a wide-angle lens) bends light sharply and captures a broad scene but makes distant objects look small. A long focal length (a telephoto lens) bends light gently and captures a narrow scene, magnifying distant objects. A typical laptop webcam uses a short, fixed focal length chosen to fit a reasonably wide view of a desk or face into the frame; it is not designed to zoom.

The aperture is the opening inside the lens that controls how much light passes through, similar in spirit to the pinhole but adjustable and much larger. A wider aperture lets in more light, useful in dim rooms, but it also narrows the depth of field, which is the range of distances that appear acceptably sharp at the same time. This is why a portrait photo taken with a wide aperture often has a crisp face and a softly blurred background, while a photo taken on a narrow aperture, such as most phone and webcam cameras in their default mode, tends to keep both near and far objects reasonably sharp.

Focus is the process of moving the lens, or elements within it, back and forth until light from the chosen distance converges exactly on the sensor plane. Autofocus systems do this automatically by analysing contrast or using a dedicated sensor, and most webcams and phone cameras focus continuously as the scene changes. It is worth knowing that a lens can only form a genuinely sharp image for one distance at a time; everything closer or farther is slightly out of focus, more so with a wide aperture, as described above.

One more thing worth knowing about lenses, because it will matter when we start measuring shapes and distances later in this path, is lens distortion. Cheap or very wide lenses bend straight lines in the world into slightly curved lines in the image, usually bulging outward near the edges of the frame, an effect called barrel distortion. This is a property of the lens, not the sensor or the software, and it is one reason photographs of, say, a chessboard taken with a wide-angle webcam can look subtly warped near the corners.

The Sensor: Counting Photons

Once the lens has focused light, it lands on the sensor, a flat chip covered edge to edge with a grid of millions of tiny light-sensitive cells called photosites. Each photosite is a small well that converts arriving light into an electrical charge: more light during the exposure means more charge accumulates, less light means less charge. A photosite does not know or care about colour on its own, it only measures how much light hit it, which is really a measurement of brightness.

You will sometimes see the terms CCD and CMOS attached to sensors. These are two different manufacturing technologies for building that grid of photosites and reading their charges back out. CMOS sensors are cheaper to produce, use less power and are now used in the vast majority of consumer devices, including essentially every laptop webcam, phone camera and modern DSLR. CCD sensors are still used in some specialised and scientific equipment. For everything in this path, you can assume you are working with a CMOS sensor, and the distinction will not change anything about how you process the resulting images.

Camera marketing often emphasises megapixels, the total count of photosites on the sensor, a 12 megapixel sensor having around twelve million of them arranged in a grid. More photosites sound like an obvious improvement, and up to a point they are, since they let you capture finer detail. But photosites live on a sensor of a fixed physical size, and squeezing more of them into the same area means each one is smaller and can collect less light. A phone camera with a tiny sensor and twenty million tiny photosites can perform worse in dim conditions than a larger sensor with half as many, larger photosites, because each larger photosite gathers more light and produces a cleaner signal. Resolution and image quality are related but not the same thing, and a sensor's physical size matters as much as its pixel count.

Colour From a Grid That Only Sees Brightness

This raises a puzzle. If every photosite only measures brightness, how does a camera produce a colour photograph at all? The answer is a thin layer called a colour filter array sitting directly on top of the sensor, with one tiny coloured filter glued over each individual photosite. The most common layout, called a Bayer filter, covers photosites with red, green or blue filters in a repeating pattern, with twice as many green filters as red or blue, because the human eye is more sensitive to green light and this layout produces a visually sharper result.

The effect is that each photosite only ever measures one colour component. A photosite under a red filter records how much red light arrived and knows nothing about the green or blue light that hit the same spot. So after the sensor has been read, the camera has a single grid of brightness values where each cell is tagged red, green or blue, but it does not yet have a full red, green and blue value for every pixel location. Producing that is the job of a process called demosaicing, where the camera's onboard processor estimates the missing two colour values at every pixel by looking at the neighbouring photosites of those colours and interpolating between them.

Demosaicing is a clever piece of engineering, but it is an estimate, not a direct measurement, which is one reason very fine, closely spaced coloured patterns in a scene can sometimes produce faint rainbow artefacts in a photo, since the interpolation has to guess at detail finer than the spacing of same-coloured filters. We will not need to implement demosaicing ourselves; it happens inside the camera or its driver before an image ever reaches Python. We mention it here because it explains why a colour image is really three interleaved brightness grids rather than one, a point the next article in this path builds on when it introduces colour spaces properly.

From Charge to Numbers: Pixels and Bit Depth

After demosaicing, every pixel location has an estimated amount of red, green and blue light, but those amounts still exist as continuous electrical charges, not as the tidy integers we work with in code. A component called an analogue-to-digital converter, or ADC, measures each charge and rounds it to the nearest value on a fixed scale. This rounding step, called quantisation, is what finally produces the grid of whole numbers that a program like OpenCV can load and manipulate.

The simplest way to see what that grid actually looks like is to build a tiny one by hand with NumPy, which is the library OpenCV uses underneath to store every image.

import numpy as np

# A tiny greyscale "image": 4 rows, 5 columns, one brightness value per pixel
tiny_image = np.array([
    [  0,  40,  80, 120, 160],
    [ 40,  80, 120, 160, 200],
    [ 80, 120, 160, 200, 240],
    [120, 160, 200, 240, 255]
], dtype=np.uint8)

print("Shape (rows, columns):", tiny_image.shape)
print("Data type:", tiny_image.dtype)
print("Pixel at row 2, column 3:", tiny_image[2, 3])

Running this (tested against numpy 1.x, though the behaviour has been stable for years) prints a shape of (4, 5), meaning 4 rows and 5 columns, a data type of uint8, and the value 200 for the pixel at row 2, column 3. The dtype uint8 means each pixel is stored as an unsigned 8-bit integer, which can only hold whole numbers from 0 to 255. This is exactly the output of the ADC we just described: the sensor's continuous charge at that location was rounded to the nearest of 256 possible brightness levels, where 0 means no light detected and 255 means the brightest level the sensor can record before it saturates.

A real colour photograph is the same idea with one extra dimension, three numbers per pixel instead of one, holding red, green and blue respectively. OpenCV, somewhat famously and confusingly for newcomers, stores those three numbers in the order blue, green, red rather than the red, green, blue order most other libraries use. We will rely on this fact properly in the next article when we cover colour spaces, but it is worth seeing it once here so it does not come as a surprise.

import numpy as np

# A 3x3 colour image: one pixel per cell, 3 channels per pixel
# OpenCV stores channels in the order Blue, Green, Red
colour_image = np.zeros((3, 3, 3), dtype=np.uint8)
colour_image[0, 0] = [0, 0, 255]   # pure red
colour_image[1, 1] = [0, 255, 0]   # pure green
colour_image[2, 2] = [255, 0, 0]   # pure blue

print("Shape (rows, columns, channels):", colour_image.shape)
print("Pixel at (0, 0):", colour_image[0, 0])

The shape printed here is (3, 3, 3): three rows, three columns, three channels. The pixel at (0, 0) prints as [0, 0, 255], which is the blue, green, red triplet for pure red, since the blue and green channels are zero and the red channel is at its maximum. If you loaded this same array with a library that expects red, green, blue order, it would display as pure blue instead, which is exactly the kind of bug that shows up later as oddly swapped colours and is worth remembering now.

How many distinct brightness levels are available per channel depends on the bit depth the ADC uses. An 8-bit camera, the overwhelming default for consumer photos and video, produces 256 levels per channel, numbered 0 to 255, so a colour pixel can represent one of roughly sixteen million combinations across its three channels. Some professional and scientific cameras use 10, 12 or even 16 bits per channel for finer gradations, which is useful when an image will be heavily edited afterward, but almost everything you will do in this path, including every example in later articles, works with standard 8-bit images. The following example shows what quantisation actually does to a smooth signal.

import numpy as np

# A smooth brightness ramp from 0.0 to 1.0, as a sensor might measure incoming light
light_levels = np.linspace(0, 1, 12)
print("Raw light levels:   ", np.round(light_levels, 3))

# An 8-bit camera rounds this into 256 discrete steps (0 to 255)
eight_bit = np.round(light_levels * 255).astype(np.uint8)
print("8-bit pixel values: ", eight_bit)

# A hypothetical 2-bit camera would only have 4 discrete steps (0 to 3)
two_bit = np.round(light_levels * 3).astype(np.uint8)
print("2-bit pixel values: ", two_bit)

The raw light levels are twelve evenly spaced values between 0 and 1, representing a perfectly smooth gradient of brightness in the real world. The 8-bit version rounds each of those to one of 256 possible integers, and because 256 levels is a fine-grained scale, the result still looks like a smooth gradient to the eye even though it is technically a staircase of discrete steps. The 2-bit version, with only 4 possible values, rounds the same smooth gradient into clearly visible bands, known as banding or posterisation. This is exactly why low bit depth produces visible stripes in smooth skies or shadows, and it is a useful mental model for why 8 bits per channel has remained the practical standard: it is fine enough that the steps are invisible in most ordinary photos.

Resolution: More Pixels, More Data, Not Always More Detail

Resolution simply means how many rows and columns of pixels an image has, often written as width by height, such as 1920 by 1080. A higher resolution image packs more pixels into the same frame, which can capture finer detail, but only if the lens and sensor are actually good enough to resolve that detail in the first place. A blurry or poorly focused photo taken at a high resolution is still blurry; extra pixels cannot recreate detail that the lens never delivered to the sensor. This is the same point made earlier about sensor size and megapixels, now extended to the final image file.

Resolution also matters practically in computer vision work, often in the opposite direction from what marketing suggests: higher resolution is not automatically better for a program, because every extra pixel is extra data that has to be processed, which costs time and memory. A very common first step in many computer vision pipelines is deliberately shrinking an image to a smaller resolution before doing anything else, trading a little detail for a large gain in processing speed. The following example builds a simple checkerboard pattern, shrinks it the way a lower resolution sensor or a deliberate resize would, then enlarges it again to show what is lost.

import cv2
import numpy as np

# Build a simple checkerboard pattern to stand in for a photographed scene
square = 20
board = np.zeros((square * 8, square * 8), dtype=np.uint8)
for row in range(8):
    for col in range(8):
        if (row + col) % 2 == 0:
            board[row*square:(row+1)*square, col*square:(col+1)*square] = 255

print("Original resolution:", board.shape)

# Simulate a much lower resolution sensor by shrinking the image
small = cv2.resize(board, (20, 20), interpolation=cv2.INTER_AREA)
print("Downsampled resolution:", small.shape)

# Try to recover detail by enlarging the small version back up
restored = cv2.resize(small, (160, 160), interpolation=cv2.INTER_NEAREST)
print("Enlarged back to:", restored.shape)

This example assumes opencv-python 4.x and numpy are installed. It prints three shapes: the original 160 by 160 checkerboard, the downsampled 20 by 20 version, and the version enlarged back to 160 by 160. If you display board and restored side by side with cv2.imshow, which we will cover properly in the next article, you would see that restored looks blocky and has lost the crisp edges of the original squares, even though it technically has the same number of pixels again. Enlarging an image never adds back real detail; it can only guess at the missing pixels, much like the demosaicing step described earlier. This is the same reason a tiny, blurry security camera frame cannot be zoomed in on to reveal a clear face, no matter how often that happens in films.

Exposure, Gain and Noise

Two more camera settings shape the pixel values you eventually work with: exposure time and gain. Exposure time is how long the sensor's photosites are left open to collect light before being read out, similar to how long a pinhole camera's shutter stays open. A longer exposure collects more light, which helps in dim scenes, but if anything in the scene moves during that time, the result blurs, which is why photos taken in low light with a moving subject often come out streaky.

Gain, often labelled ISO on cameras, is an electronic amplification applied to the charge collected by each photosite before or during the ADC stage. Raising the gain makes a dim signal produce a brighter final image without lengthening the exposure, which is useful for photographing a moving subject indoors. The cost is that gain amplifies not only the genuine light signal but also the small random fluctuations inherent in any electronic sensor, so a high-gain image looks grainy. This graininess is called noise, and it is one of the most common and most visible imperfections you will see in webcam footage, especially in a dim room.

We can simulate this effect directly, because noise behaves statistically like adding small random numbers to otherwise identical pixel values.

import numpy as np

rng = np.random.default_rng(seed=0)

# A flat grey patch, as if photographing an evenly lit, plain wall
clean_patch = np.full((5, 5), 128, dtype=np.uint8)

# Simulate the kind of random noise a sensor adds under high gain / low light
noise = rng.normal(loc=0, scale=15, size=clean_patch.shape)
noisy_patch = np.clip(clean_patch.astype(int) + noise, 0, 255).astype(np.uint8)

print("Clean patch:\n", clean_patch)
print("Noisy patch:\n", noisy_patch)

The clean patch is a perfectly uniform grid of the value 128, representing an evenly lit wall with no detail at all. The noisy patch, after adding small random values and clipping the result back into the valid 0 to 255 range, shows a grid of values scattered around 128 rather than a perfectly flat block, even though the underlying scene did not change at all. This is exactly what noise looks like in a real photo: pixel-to-pixel variation that carries no real information about the scene, caused purely by the electronics. Later in this path, in the article on filters and blurs, we will look at techniques that reduce noise like this by averaging nearby pixels, which is one of the most common first steps in any practical computer vision pipeline.

A closely related idea is dynamic range, meaning the span between the darkest and brightest light a sensor can usefully distinguish in a single shot. Point a camera at a scene with both deep shadow and bright sky, and you will often find the shadows turn into solid black blobs or the sky turns into a featureless white patch, because the sensor's range of 256 levels per channel had to be stretched across the whole scene and ran out of room at one or both ends. This is simply a limit of the hardware, not something that can be fixed afterward by software, since the detail that overflowed the sensor's range was never recorded in the first place.

Common Mistakes and Misconceptions

• Assuming more megapixels always means a better image: sensor size, lens quality and noise levels matter at least as much as pixel count, especially in dim conditions.
• Treating pixel values as a precise measurement of the true colour of a surface: they are the end result of lens characteristics, a colour filter, demosaicing, gain and quantisation, each of which introduces some approximation.
• Expecting enlarging a low resolution image to recover lost detail: resizing up can only interpolate between existing pixels, it cannot invent information the sensor never captured.
• Forgetting that OpenCV stores colour images in blue, green, red order rather than red, green, blue, which silently swaps colours if you mix OpenCV arrays with code that expects the other order.
• Blaming software for noise or blur that actually comes from the capture stage, such as a dim room, a short exposure or a moving subject, none of which can be fully undone after the fact.
• Assuming a sharp-looking preview on a camera's small screen means the full-resolution image is equally sharp; focus errors and lens softness are often only visible once you view the image at full size.

Summary and What's Next

A digital photograph is the end point of a short but intricate physical pipeline. Light leaves a scene and is bent by a lens into a focused image; a sensor made of millions of tiny photosites turns that light into electrical charge; a colour filter array lets each photosite see mostly one of red, green or blue; an analogue-to-digital converter rounds each charge into a whole number within a fixed range such as 0 to 255; and onboard processing fills in missing colour detail to produce the final grid of pixels. Along the way, choices about aperture, exposure time and gain trade off sharpness, brightness, motion blur and noise against each other, and the sensor's resolution and dynamic range put hard limits on how much real detail ever makes it into that grid.

Everything we do from here on in this path works on the far end of that pipeline, the finished grid of numbers, so this article was really about building a mental model for why that grid looks the way it does and where its limitations come from. The next article picks up exactly where we left off with colour, looking closely at how those red, green and blue numbers combine into the RGB colour space, why OpenCV's blue, green, red ordering matters in practice, and why a second colour space called HSV often makes certain computer vision tasks, like picking out a specific coloured object, considerably easier.