Why this article comes first

This is the first article in a path called Computer Vision From Zero. Before touching a single line of OpenCV code, it helps to slow down and ask a simple question: what does it actually mean for a computer to see? The word 'see' is doing a lot of work here, and it is easy to assume a computer sees the way you do, with a sense of shapes, objects and meaning. It does not. A computer only ever receives a grid of numbers. Everything else, lines, faces, text, traffic signs, is something we or a program have to work out from those numbers. Once that idea is solid, the rest of this path, cameras, colour spaces, filters, edges and masks, will make a lot more sense, because each of those topics is really just a different way of asking questions about a grid of numbers.

An image is a grid of numbers

Picture a black and white photograph zoomed in so far that you can see individual tiny squares. Each square is called a pixel, short for 'picture element'. In a greyscale image, every pixel holds a single number that describes how bright it is, usually somewhere between 0 and 255. A value of 0 means pure black, 255 means pure white, and everything in between is a shade of grey. A colour image usually stores three such numbers per pixel, one each for red, green and blue, but we will leave colour for the next article in this path and stay with greyscale for now, because it is simpler to reason about.

Here is a genuinely tiny image, five pixels wide and five pixels tall, small enough to write out by hand. Imagine it shows a dark square sitting on a light background.

🐍Python
import numpy as np

# A 5x5 greyscale image as a NumPy array.
# 0 = black, 255 = white, values in between = grey.
image = np.array([
    [230, 230, 230, 230, 230],
    [230, 20,  20,  20,  230],
    [230, 20,  20,  20,  230],
    [230, 20,  20,  20,  230],
    [230, 230, 230, 230, 230],
], dtype=np.uint8)

print("shape:", image.shape)
print("dtype:", image.dtype)
print("average brightness:", image.mean())

Running this with NumPy (any recent 1.x or 2.x version works) prints a shape of (5, 5), meaning five rows and five columns, a dtype of uint8, meaning each number is stored as an unsigned 8-bit integer from 0 to 255, and an average brightness of 128.4. That single average number already tells you something: a bright, mostly white image would average closer to 230, a dark one closer to 20, and this image sits in between because it is a dark square surrounded by a light border. This is, in miniature, exactly what every computer vision program works with: a grid of numbers, and a bit of arithmetic to decide what they mean.

Now suppose you wanted to find where the dark square is, without being told its position. One simple idea is to look at how much the brightness changes as you move from one pixel to its neighbour. Where the numbers are almost the same, you are probably inside a flat region, either the background or the square. Where the numbers jump sharply, for example from 230 to 20, you have likely crossed an edge, a boundary between two regions. That jump is the whole idea behind edge detection, which you will meet properly in part six of this path. For now, the point is simpler: a concept that sounds visual, 'the edge of a square', turns out to be something you can find with subtraction.

🐍Python
import numpy as np

image = np.array([
    [230, 230, 230, 230, 230],
    [230, 20,  20,  20,  230],
    [230, 20,  20,  20,  230],
    [230, 20,  20,  20,  230],
    [230, 230, 230, 230, 230],
], dtype=np.int16)  # int16 so differences can go negative

# Difference between each pixel and the one to its right.
horizontal_change = np.diff(image, axis=1)
print(horizontal_change)

The output is a 5 by 4 grid of differences. Most entries are 0, because neighbouring pixels inside the background or inside the square have the same value. But in the middle rows you will see a -210 where the image goes from light to dark, and a +210 further along where it goes from dark back to light. Those large numbers mark exactly where the edges of the square are. Nobody told the program there was a square, it found the boundary purely by noticing where the numbers changed a lot. This is the pattern you will see again and again in this path: a visual idea gets turned into a small, explainable calculation on an array of numbers.

A working definition of computer vision

With that picture in mind, here is a working definition. Computer vision is the field of methods and software that take images or video as input and produce useful information as output, such as the location of an object, a count, a measurement, a classification, or a decision. The input is always numbers arranged in a grid, or a sequence of such grids for video. The output is something a person or another program can act on.

It helps to separate three related terms that often get blurred together.

  • Image processing: transforming an image into another image, for example making it sharper, removing noise, or converting it to black and white. The input and output are both images.
  • Computer vision: extracting information from an image or video that is not itself an image, for example 'there are 7 coins in this photo' or 'the face in this frame belongs to the registered user'. The output is a number, a label, a location, or a decision.
  • Image processing is often a step inside a computer vision system, used to clean up or prepare an image before the information is extracted from it. Several later parts of this path, on filters, thresholding and edges, are really image processing techniques that computer vision systems rely on.

It is also worth being honest about what computer vision is not. It is not a single algorithm, and it is not inherently about artificial intelligence or deep learning, even though modern systems increasingly use learned models. Long before deep learning became practical, people built working vision systems using careful measurement, geometry and statistics, the kind of classical methods this path starts with. Part nine of this path will show how the field moved from these hand-made rules to features learned automatically from data, but that move happened on top of the same basic idea: numbers in, useful information out.

Where you already use computer vision

It is tempting to think of computer vision as something exotic, confined to research labs or self-driving cars. In practice it is quietly embedded in dozens of ordinary things. Here is a selection, grouped loosely by how familiar they probably are.

On your phone

  • Unlocking your phone by looking at it, which compares the geometry of your face against a stored pattern.
  • Your photo app grouping pictures by the people in them, or letting you search photos for 'dog' or 'beach' without you ever typing a tag.
  • The camera app finding faces to focus on, or drawing a box around them before you take the shot.
  • Scanning a document with your phone and having the app automatically find its edges, straighten it and crop out the background, which is exactly the hands-on project in part ten of this path.
  • Portrait mode blurring the background behind a subject, which requires the software to work out which pixels belong to a person and which belong to everything behind them.
  • Augmented reality filters that stick animated ears or glasses onto a face in a video call, which requires tracking facial features frame by frame.

Out and about

  • Scanning a QR code or barcode at a shop till, which is a camera locating a specific pattern and decoding it.
  • A car park barrier reading your number plate to open automatically.
  • Self-checkout machines in supermarkets recognising items by their packaging or shape.
  • Automatic number plate recognition used for toll roads and speed cameras.
  • Driver assistance features such as lane departure warnings and automatic braking, which rely on a camera finding lane markings or obstacles in the road ahead.
  • Airport or office security gates that match a live camera feed against a passport photo.

Behind the scenes

  • Postal and courier services sorting mail by reading handwritten or printed addresses automatically.
  • Banks letting you deposit a cheque by photographing it, reading the amount and account details from the image.
  • Manufacturing lines using a camera above the conveyor belt to spot a cracked bottle or a missing label before it reaches a customer.
  • Warehouses tracking stock by scanning barcodes and labels as items move through.
  • Doctors using image analysis to help spot patterns in X-rays, scans or microscope slides, as one input among several in a diagnosis.
  • Farmers using drone or satellite images to see which parts of a field are under stress and need water or fertiliser.
  • Video platforms automatically detecting and blurring faces or licence plates for privacy, or flagging content for review.
  • Sports broadcasts tracking a ball or a player's position to draw the graphics you see overlaid during a match.

None of these systems are single magic boxes. Each is built from smaller, understandable steps, many of which are exactly the steps this path will walk through: getting a clean image, choosing the right colour representation, filtering out noise, finding edges and shapes, and deciding what counts as foreground and what counts as background. The webcam motion detector you will build in part twelve, for instance, uses ideas that are not so different from the ones behind a supermarket security camera that raises an alert when something changes in an aisle after closing time.

The pipeline behind most vision systems

Although applications differ wildly, from reading a postcode to counting coins on a table, most computer vision systems share a similar shape. It helps to think of it as a pipeline with five stages, where the output of one stage feeds the next.

  • Acquire: get an image or a stream of frames from a camera, a scanner, a satellite, or a stored file. The quality of everything downstream depends heavily on this step, which is why part two of this path is devoted entirely to how a camera turns light into pixels.
  • Preprocess: clean the image up. This might mean converting it to greyscale, adjusting brightness, removing noise, or resizing it. Parts three to five of this path cover colour spaces, the OpenCV basics needed to do this in code, and filters and blurs for cleaning noisy images.
  • Extract features: pull out the parts of the image that matter for the task, such as edges, corners, regions of a particular colour, or distinctive patterns. Parts six and seven, on edges, contours, shapes, thresholding and masks, cover the classical way of doing this.
  • Analyse and decide: turn those features into the answer you actually want, for example 'this is a five pence coin', 'this document has four corners here', or 'something moved in this frame compared to the last one'.
  • Act: do something with the result, such as display it on screen, sound an alarm, sort an item on a conveyor belt, or simply save a count to a file.

Take the cheque deposit example. The camera on your phone acquires an image of the cheque. Preprocessing might straighten the image and convert it to greyscale. Feature extraction finds the printed and handwritten regions, often by locating boxes and lines on the cheque's layout. Analysis reads the digits and letters inside those regions, often using a specialised kind of recognition for text, sometimes called optical character recognition. Acting on it means filling in the deposit amount automatically and asking you to confirm it. Every one of those stages can fail in a small way, blur, poor lighting, smudged ink, which is exactly why real systems check their own confidence and ask a human when they are unsure, rather than pretending to be certain.

Two broad ways to extract information

There are, broadly, two families of techniques for the 'extract features' and 'analyse and decide' stages, and it is useful to know both exist before you start, even though this path spends most of its early parts on the first family.

The first family is classical, hand-crafted computer vision. Here, a person designs an explicit rule or calculation based on an understanding of the problem. The edge-finding subtraction you saw earlier is a hand-crafted idea: someone decided that a large difference between neighbouring pixels usually means an edge, and wrote that rule down directly. Thresholding, which separates an image into foreground and background based on brightness, is another hand-crafted rule: pixels brighter than some chosen number are treated as one thing, darker pixels as another. These techniques are transparent, you can always explain exactly why the program made a particular decision, and they often run fast on modest hardware, including the webcam-based projects you will build later in this path.

The second family is learned computer vision, where instead of writing the rule by hand, you show a model many examples of images paired with the correct answer, for instance thousands of photos labelled 'cat' or 'not cat', and let a training process work out the rule itself. Modern systems for face recognition, object detection in photos, or reading handwriting at scale typically rely on this approach, often using a kind of model called a neural network. These models can handle far more variety and complexity than hand-crafted rules, but they need much more data, more computing power, and they are harder to explain, since the rule they learned is encoded in thousands or millions of numbers rather than a short formula a person wrote.

This path deliberately starts with the classical family. Thresholds, filters, edges and contours are not old-fashioned curiosities, they are still used inside modern systems, often as a preprocessing step before a learned model, or as a complete, lightweight solution when the task is well defined, like counting coins against a plain background. Understanding how a computer separates foreground from background using plain arithmetic will make it much clearer, later on in part nine, what a learned model is actually doing differently, and why it needs so much data to do it.

Common misconceptions worth clearing up now

A few beliefs tend to trip up people new to this field, and it is worth naming them early so you can recognise them later in this path.

  • Computer vision is not one algorithm that 'understands pictures'. It is a toolbox of techniques, chosen and combined for a specific task. A system built to count coins will not, on its own, recognise a face.
  • The camera and the lighting matter as much as the code. A perfectly good edge-detection method will fail on a blurry, badly lit photo, because the numbers it relies on are already corrupted before any processing starts. Part two of this path looks at exactly how cameras and sensors shape the pixels you get to work with.
  • Colour is not as simple as red, green and blue values being a reliable fingerprint for an object. The same red apple looks like a different set of numbers in bright sunlight versus indoor lighting. Part three on colour spaces explains why many vision tasks are done in a different colour representation for this reason.
  • Detection, recognition and classification are different tasks, even though people use the words loosely. Detecting a face means finding where a face is in an image. Recognising a face means working out whose face it is. Classifying an image means assigning it one label, such as 'cat' or 'dog', without necessarily saying where in the image anything is.
  • More pixels and more expensive cameras do not automatically mean better results. A sharp, well-lit, low-resolution image of a coin on a plain table is often easier to work with correctly than a high-resolution but blurry, cluttered photo.
  • A method that works perfectly on your test photo may fail on a slightly different one, taken at a different time of day, with a different phone, or at a different angle. Testing on a handful of varied, realistic examples, not just the one photo you first tried, is the single most useful habit to build early.

What you will build in this path

The summary for this path is straightforward: how computers see, from the first pixel. That is deliberate. Rather than jumping into a pretrained model that already 'knows' how to find objects, this path builds understanding from the ground up, starting with how a camera turns light into numbers, how those numbers are arranged into colour, how to read and manipulate them with OpenCV, and how to combine simple operations, blurring, edge finding, thresholding, into something that solves a real small problem. By the end you will have built a document scanner, a coin counter and a webcam motion detector, each using only the classical techniques covered along the way, with no need for a GPU or a large dataset.

Summary and what comes next

An image, to a computer, is a grid of numbers, and computer vision is the set of techniques for turning those numbers into useful information, whether that is a location, a count, a label or a decision. It is already embedded in things you use every day, from unlocking a phone to scanning a QR code to a car park camera reading a number plate, and nearly all of these systems share the same rough pipeline: acquire an image, clean it up, extract the relevant features, decide what they mean, and act on that decision. Some of those steps are done with transparent, hand-written rules, and some with models learned from data, and this path starts with the former because it builds the intuition you need to understand the latter later on.

The next article in this path, How a Camera Makes an Image, goes one level deeper into the acquisition stage you just read about. It looks at how a lens focuses light, how a sensor turns that light into the numbers you saw in the arrays above, and why those physical choices, exposure, focus, resolution, already shape everything a vision program can and cannot do before a single line of processing code runs.