08 — Camera Calibration: Video Tutorials

Calibration
OpenCV
Raspberry Pi
Step-by-step tutorials for the nine recommended camera-calibration videos (Cyrill Stachniss, Kevin Wood, Nicolai Nielsen, The Drone Dojo): plain-English notes, updated OpenCV 5 code, quizzes, and a hands-on lab for every video, ending with calibrating a Raspberry Pi camera in 15 minutes.

How to use this page

Lecture 6 recommends nine short videos about camera calibration. This page turns each one into a tutorial: watch the video, read the notes in plain English, run the updated code, answer the quiz, then do the lab notebook made for that video. The videos are by other people; the notes follow what each video teaches and point out where the code needs changing for OpenCV 5 and Raspberry Pi OS Bookworm.

The idea in one sentence

Show the camera a pattern whose real shape you know (a printed chessboard), see where its corners land in the picture, and solve for the numbers (focal length, image centre, lens distortion) that explain the difference. Then use those numbers to straighten every future picture and to measure in millimetres.

Analogy. An eye doctor shows you a chart whose letters she knows, sees how you read it, and writes down your prescription. The chessboard is the chart, the camera is your eye, and the calibration result (camera matrix K + distortion coefficients) is the camera's prescription.

Learning path

StepVideoYou learnLab
1 · IdeasIntrinsics & extrinsics, intrinsic calibration (Cyrill Stachniss, 5 min each)what the numbers mean6M
2 · MathsZhang's method (Cyrill Stachniss, lecture)how a chessboard gives K
3 · OverviewCalibration explained, OpenCV Python (Kevin Wood)the full workflow & how to improve it6K
4 · CodeIn < 5 minutes, the script line by line (Nicolai Nielsen)capture images, calibrate, save, undistort6J
5 · Better targetsChArUco boards (Nicolai Nielsen)partial views, outliers, JSON6L
6 · Your PiCalibrate a camera in 15 minutes (The Drone Dojo)the Raspberry Pi workflow + a ruler test6I

Before you start: you need Python with opencv-python numpy matplotlib (Colab has them), a printed 9 × 6 chessboard glued to something flat, and a ruler.

1 · Intrinsics & extrinsics

Cyrill Stachniss (University of Bonn) · Camera Intrinsics and Extrinsics — 5 Minutes with Cyrill

If the video does not play here, open it on YouTube.

In plain words

Extrinsics say where the camera is and where it looks: 3 numbers for position and 3 for direction (6 degrees of freedom). Intrinsics say what the camera is like inside: how a ray of light becomes a pixel when the camera sits at the origin.

Key ideas

  • The camera model is a mathematical description of how the camera works; intrinsics and extrinsics are its parameters (also called interior and exterior orientation).
  • The location of the camera is the projection centre, the point where all rays meet in the pinhole model.
  • Basic intrinsics: the camera constant (distance image plane ↔ projection centre, i.e. the focal length), a scale difference between x and y (equivalently fx and fy), the principal point (where the optical axis hits the image, near but never exactly at the centre) and a shear (≈ 0 for digital cameras).
  • 6 extrinsic + 5 intrinsic = 11 degrees of freedom, all inside the 3 × 4 projection matrix P: x = P X (homogeneous coordinates). This ideal model has no lens distortion; real lenses need extra non-linear parameters.
  • Estimating the intrinsics = calibration. Estimating only the extrinsics = camera localization.

Try it

import numpy as np, cv2
K = np.array([[800, 0, 640], [0, 800, 360], [0, 0, 1.0]])   # intrinsics: f = 800 px, principal point (640, 360)
R, _ = cv2.Rodrigues(np.array([0.0, 0.2, 0.0]))             # extrinsics: turned 0.2 rad around y ...
t = np.array([[0.1], [0.0], [2.0]])                          # ... and 2 m in front of the camera
P = K @ np.hstack([R, t])                                    # 3 x 4 projection matrix
X = np.array([0.0, 0.0, 0.0, 1.0])                           # a world point (homogeneous)
x = P @ X
print("pixel:", x[:2] / x[2])                                # divide by the 3rd coordinate

✏️ Check yourself

  1. How many degrees of freedom does P have, and why not 12?
    Show answer

    11: 6 extrinsic + 5 intrinsic. P has 12 numbers, but it is only defined up to scale (multiplying P by 2 gives the same pixels).

  2. You know K and want to find where a drone camera is. Calibration or localization?
    Show answer

    Localization: only the extrinsics (R, t) are unknown, e.g. with cv2.solvePnP on known markers.

2 · Intrinsic calibration

Cyrill Stachniss · Intrinsic Camera Calibration — 5 Minutes with Cyrill

If the video does not play here, open it on YouTube.

In plain words

The goal of calibration: for every pixel, know the exact direction of the ray it sees. Then a pixel tells you on which line in 3-D the object lies.

Key ideas

  • Model: x = K R [I | −X0] X. K holds the intrinsics, R the viewing direction, X0 the projection centre.
  • The DLT can estimate K, R, X0 together, but needs ≥ 6 points with known 3-D coordinates. That is hard to set up.
  • Zhang's method needs only a flat checkerboard: each photo gets its own coordinate system on the board, so every corner has Z = 0 and known X, Y (you know the square size). At least 3 photos (more in practice).
  • In practice: print the board, glue it flat, do the "calibration dance" in front of the camera, feed the photos to the algorithm.
  • Lens effects such as barrel distortion are added as extra non-linear parameters and estimated by least squares; in general every pixel may get its own small correction.

✏️ Check yourself

  1. Why must the board be glued to something flat?
    Show answer

    The method assumes Z = 0 for every corner. A bent board breaks that assumption and biases K and the distortion.

  2. After calibration, what can you do that you could not before?
    Show answer

    Straighten (undistort) images so straight lines are straight, and turn pixels into exact 3-D ray directions for measuring and 3-D reconstruction.

3 · Zhang's method

Cyrill Stachniss · Camera Calibration using Zhang's Method (full lecture)

If the video does not play here, open it on YouTube.

In plain words

Each tilted photo of the flat board gives two clues about K. Three photos give six clues, enough to solve for the camera's inside numbers in closed form. Then a least-squares step polishes everything, including lens distortion.

The recipe, step by step

  1. One homography per photo. With Z = 0 the projection drops the 3rd column of R: [u v 1]ᵀ ≃ H [X Y 1]ᵀ with H = K [r1 r2 t]. Estimate H like the DLT (SVD); 4 points are enough, you have 54.
  2. Use what you know about rotations. r1 and r2 are perpendicular and have length 1. Writing ri = K⁻¹hi gives two equations per photo on the symmetric matrix B = K⁻ᵀK⁻¹:
    h1ᵀ B h2 = 0 and h1ᵀ B h1 = h2ᵀ B h2.
  3. Solve. B has 6 unknowns, known up to scale → 5 → 3 photos. Stack the equations as V b = 0 and take the SVD.
  4. Get K. B is symmetric positive-definite, so a Cholesky decomposition B = L Lᵀ gives K⁻¹ = Lᵀ.
  5. Refine. Add distortion parameters and minimise the reprojection error (detected corner − predicted corner) over everything with least squares. This is what cv2.calibrateCamera does.

Lab 6M implements steps 1–4 in about 30 lines of NumPy and recovers K within 0.1 %.

✏️ Check yourself

  1. Why does one photo give only two equations, not three?
    Show answer

    The orthonormality of r1, r2 gives exactly two constraints (perpendicular, equal length). r3 is not seen because Z = 0 removes it.

  2. What goes wrong if all photos are taken straight-on (board parallel to the sensor)?
    Show answer

    The equations become nearly dependent: you cannot tell focal length from distance. Tilted views are essential.

▶ Lab 6M in Colab View notebook Labs page DLT, homographies and Zhang's closed form in NumPy, then OpenCV's refinement on a real GoPro.

4 · Calibration explained (Kevin Wood)

Kevin Wood · Camera Calibration Explained and SIMPLE Step-by-Step Guide!

If the video does not play here, open it on YouTube.

In plain words

Chessboard pictures go in; camera parameters come out: intrinsics, extrinsics and distortion coefficients. Use them for precise measurement or to remove image distortion.

Step by step (as in the video)

  1. World frame → camera frame = extrinsics (rotation + translation). Camera frame → image frame = intrinsics (focal length, camera centre).
  2. Radial distortion: the image bulges out (barrel) or in (pincushion). Tangential: the sensor is tilted relative to the lens. Each moves a point's x, y; together they are the distortion coefficients (k's radial, p's tangential).
  3. Folder layout: a main folder → the calibration script, the 9 × 6 chessboard image, and a folder of calibration pictures (file names don't matter).
  4. Take pictures at different angles, covering different parts of the image.
  5. Install numpy opencv-python matplotlib, run the script: it draws the corners, plots all pictures, prints the parameters and writes a JSON (reprojection error, camera matrix, distortion, rvecs, tvecs).
How to improve your result (the four tips)
  1. Take more pictures.
  2. Vary the board's orientation and position more, including extreme cases.
  3. Make sure the entire board is in every picture; replace any that are cut off.
  4. Remove blurry pictures.

Lab 6K turns these four tips into an automatic calibration doctor.

5 · OpenCV Python (Kevin Wood)

Kevin Wood · OpenCV Python Camera Calibration (Intrinsic, Extrinsic, Distortion)

If the video does not play here, open it on YouTube.

Key ideas

  • x = P X: P (3 × 4) splits into the intrinsic matrix (f, cx, cy) and the extrinsic [R | t]. OpenCV stores R as a 3 × 1 rotation vector (Rodrigues).
  • Used for AR, SLAM, 3-D reconstruction, or just undistorted images.
  • Find only the inner corners, refine them with cornerSubPix, then solve a system A h = 0, using Z = 0 for the board.
  • His result: reprojection error 0.17 px, "pretty good".
  • Tips: count the corners correctly; take ≥ 10 images in different planes (otherwise a degenerate case); keep the whole board in view.
  • To remove distortion he uses getOptimalNewCameraMatrix with alpha = 1 (keep all pixels), then undistort, and draws a straight line to show the curve is gone.

The code (OpenCV 4 and 5)

import glob, cv2, numpy as np

nRows, nCols = 9, 6                                     # inner corners
termCriteria = (cv2.TERM_CRITERIA_EPS + cv2.TERM_CRITERIA_MAX_ITER, 30, 0.001)
worldPtsCur = np.zeros((nRows * nCols, 3), np.float32)
worldPtsCur[:, :2] = np.mgrid[0:nRows, 0:nCols].T.reshape(-1, 2)
worldPtsList, imgPtsList = [], []

for path in glob.glob("calibration/*.jpg"):
    gray = cv2.cvtColor(cv2.imread(path), cv2.COLOR_BGR2GRAY)
    found, corners = cv2.findChessboardCorners(gray, (nRows, nCols), None)
    if found:
        worldPtsList.append(worldPtsCur)
        corners = cv2.cornerSubPix(gray, corners.reshape(-1, 1, 2), (11, 11), (-1, -1), termCriteria)
        imgPtsList.append(corners)                       # reshape: OpenCV 5 returns (N, 2)

repError, camMatrix, distCoeff, rvecs, tvecs = cv2.calibrateCamera(
    worldPtsList, imgPtsList, gray.shape[::-1], None, None)
print("Camera Matrix:", camMatrix, "Reproj Error (pixels): {:.4f}".format(repError), sep="\n")

img = cv2.imread("distorted.jpg"); h, w = img.shape[:2]
camMatrixNew, roi = cv2.getOptimalNewCameraMatrix(camMatrix, distCoeff, (w, h), 1, (w, h))
imgUndist = cv2.undistort(img, camMatrix, distCoeff, None, camMatrixNew)
cv2.line(imgUndist, (100, 300), (w - 100, 300), (0, 0, 255), 2)   # a straight line to compare against

✏️ Check yourself

  1. What does alpha = 1 mean in getOptimalNewCameraMatrix?
    Show answer

    Keep all source pixels (you get curved black borders). alpha = 0 zooms in until every output pixel is valid.

  2. Why can 10 images all taken in the same plane be useless?
    Show answer

    They give (almost) the same equations: a degenerate case where the solver cannot separate the parameters.

▶ Lab 6K in Colab View notebook Labs page His app on 36 GoPro images: all-corners plot, JSON, P = K[R|t] by hand, straight-line test, calibration doctor.

6 · In < 5 minutes (Nicolai Nielsen)

Nicolai Nielsen · Camera Calibration in less than 5 Minutes with OpenCV

If the video does not play here, open it on YouTube.

The workflow

  1. Clone his repository (or use our copies in LAB/Lab_6) and create a folder images.
  2. Run get_images.py: the webcam opens; press s to save a picture, ESC to stop. He shows the board on an iPad; a print works too. Rotate and move it around the frame; 10–20 images are enough.
  3. Set the chessboard size and frame size in calibration.py and run it.
  4. It saves calibration.pkl, cameraMatrix.pkl and dist.pkl, prints a total error and undistorts with both undistort and remap. Both look the same; remap is faster, so use it for real-time video.
python get_images.py          # s = save, ESC = quit -> images/img0.png, img1.png, ...
python calibration.py         # edit chessboardSize and frameSize first
Watch out

A tablet screen works for a webcam, but avoid reflections and keep the screen brightness steady. Measure the square size on the screen if you need millimetres. For the best accuracy, use a print on a rigid board.

7 · The script, line by line

Nicolai Nielsen · Learn Camera Calibration in OpenCV with Python Script

If the video does not play here, open it on YouTube.

What each part of the script does

chessboardSize = (24, 17)          # inner corners of HIS board; ours is (9, 6)
frameSize = (1440, 1080)           # (width, height) of the images
criteria = (cv2.TERM_CRITERIA_EPS + cv2.TERM_CRITERIA_MAX_ITER, 30, 0.001)   # stop rule for cornerSubPix

objp = np.zeros((chessboardSize[0] * chessboardSize[1], 3), np.float32)      # ideal points (0,0,0), (1,0,0) ...
objp[:, :2] = np.mgrid[0:chessboardSize[0], 0:chessboardSize[1]].T.reshape(-1, 2)
objpoints, imgpoints = [], []      # 3-D points in the world, 2-D points in the image

for image in glob.glob("*.png"):
    img = cv2.imread(image)
    gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)          # OpenCV loads BGR; the detector wants grayscale
    ret, corners = cv2.findChessboardCorners(gray, chessboardSize, None)
    if ret:
        objpoints.append(objp)
        corners2 = cv2.cornerSubPix(gray, corners.reshape(-1, 1, 2), (11, 11), (-1, -1), criteria)
        imgpoints.append(corners2)
        cv2.drawChessboardCorners(img, chessboardSize, corners2, ret)

ret, cameraMatrix, dist, rvecs, tvecs = cv2.calibrateCamera(objpoints, imgpoints, frameSize, None, None)
  • He explains the pinhole model (3-D point → image plane through the optical centre; focal length = distance pinhole ↔ image plane) and why barrel/pincushion distortion ruins measurements such as stereo disparity.
  • From Zhang's paper: the error drops a lot from 1 to 3–5 views and only a little after that. His demo uses 5 images, which is the minimum. For your project use 15–25.
Two different error numbers

His total error divides the L2 norm of all residuals of a view by the number of corners. It is about √N times smaller than the RMS reprojection error that calibrateCamera returns. On the dash-cam: total error 0.11 vs RMS 0.85 px. Always say which one you report. In this course: RMS in pixels.

✏️ Check yourself

  1. Why do we convert to grayscale before findChessboardCorners?
    Show answer

    The corner detector works on one intensity channel; colour adds nothing for a black-and-white board.

  2. Your images are 1280 × 720 but two are 1281 × 721. What happens?
    Show answer

    calibrateCamera stops with a size error (or worse, gives a wrong K). Resize or drop them. Lab 6J has exactly this case.

▶ Lab 6J in Colab View notebook Labs page His two scripts on the dash-cam set: pickle files, undistort vs remap timing, total error vs RMS, how many views.

8 · ChArUco boards

Nicolai Nielsen · How to do Camera Calibration with ChArUco Boards

If the video does not play here, open it on YouTube.

In plain words

A ChArUco board is a chessboard with a little ArUco marker (a square barcode) in every white square. Every corner then has its own ID, so even a partly visible board is useful, which a plain chessboard is not.

What the video does

  1. Capture 10–20 pictures with a small script (s saves, ESC quits) and try to cover about half the image each time.
  2. Upload them to his calibration web app (commercial), enter the board: 18 × 11 squares, square and marker size, dictionary 4 × 4, 100 markers.
  3. Get a JSON: focal lengths, optical centre, distortion, and the rvec/tvec of each board. Then look at the reprojection error per image: remove outliers and rerun. Under ~0.2 px is very good.
  4. A 3-D view shows where each board was relative to the camera. Then undistort an image with getOptimalNewCameraMatrix + undistort and crop to the ROI.
  5. Even ten "identical" cameras each need their own calibration: every unit is slightly different.

The same thing, free, in OpenCV

aruco_dict = cv2.aruco.getPredefinedDictionary(cv2.aruco.DICT_4X4_100)
board = cv2.aruco.CharucoBoard((18, 11), 0.030, 0.022, aruco_dict)       # squares x, y; square, marker (m)
cv2.imwrite("charuco_18x11.png", board.generateImage((3600, 2200), marginSize=80))   # print it

detector = cv2.aruco.CharucoDetector(board)
all_obj, all_img = [], []
for f in sorted(glob.glob("output_images/*.png")):
    gray = cv2.imread(f, cv2.IMREAD_GRAYSCALE)
    ch_corners, ch_ids, m_corners, m_ids = detector.detectBoard(gray)
    if ch_ids is not None and len(ch_ids) >= 10:
        obj, img = board.matchImagePoints(ch_corners, ch_ids)
        all_obj.append(obj); all_img.append(img)

rms, K, D, rvecs, tvecs, _, _, per_view = cv2.calibrateCameraExtended(all_obj, all_img, gray.shape[::-1], None, None)
keep = [i for i, e in enumerate(per_view.ravel()) if e <= 2 * np.median(per_view)]    # drop outliers, recalibrate
OpenCV ≥ 4.7

The old cv2.aruco.calibrateCameraCharuco and interpolateCornersCharuco are gone. Use CharucoDetector + matchImagePoints + calibrateCamera as above.

▶ Lab 6L in Colab View notebook Labs page The video's 18 × 11 board: detect IDs, per-image errors, remove 2 shaky shots, JSON export, 3-D board view, ROI crop.

9 · Raspberry Pi camera in 15 minutes

The Drone Dojo (presented by Pankaj) · Calibrate OpenCV Camera In Under 15 Minutes

If the video does not play here, open it on YouTube.

In plain words

The video calibrates a Raspberry Pi camera for a drone that must know exactly where objects are (pose estimation). A script grabs 20 good chessboard pictures by itself while you move the board, computes the camera matrix and distortion, and then an ArUco marker and a ruler prove the calibration measures real centimetres.

What you need

  • Raspberry Pi 5 (or 4) with a Pi camera, Raspberry Pi OS Bookworm, and a desktop (monitor or VNC) because the script opens a camera window.
  • pattern.png printed on A4, glued to a flat cutting mat or board; a ruler.
  • An ArUco marker (ID 72 from the original dictionary in the video), printed and measured.

Step by step

  1. Log in. SSH for the terminal, VNC for the desktop. Check the camera works. 
    ssh pi@raspberrypi.local                # or pi@<IP address>
rpicam-hello --list-cameras             # the camera must be listed
  2. Get the scripts and the pattern. The video clones a repository with calibrate.py and pattern.png; our updated scripts are in the course repository: 
    mkdir -p ~/calib && cd ~/calib
for f in pi_calibrate_20.py aruco_pose_test.py; do
  wget https://raw.githubusercontent.com/Kazimbalti/intelligent-vision-systems/main/LAB/Lab_6/$f
done
wget https://raw.githubusercontent.com/opencv/opencv/4.x/doc/pattern.png
sudo apt install -y python3-opencv python3-picamera2   # NOT "sudo pip install" (breaks Bookworm's Python)
  3. Measure a square. In the video one square is 26 mm. Measure yours; this number turns the results into millimetres.
  4. Calibrate at the resolution you will use, 640 × 480 as in the video. Run from a terminal on the VNC desktop: 
    python3 pi_calibrate_20.py --mm 26 --width 640 --height 480
  5. Move the board. "Try to show the edges first, all the edges," then all other positions: near, far, tilted. The script only keeps a picture when the board is fully visible and in a new place (blue dots show the kept positions). After 20 it calibrates and saves calibration_files/cameraMatrix.txt, cameraDistortion.txt and camera.yaml.
  6. Test with a marker and a ruler. Make and print the marker, measure the black square edge to edge (20 cm in the video), and run the test: 
    python3 aruco_pose_test.py --make-marker --id 72        # -> marker_id72.png, print it
python3 aruco_pose_test.py --size 200 --id 72 --width 640 --height 480
  7. The ruler test. Move the marker until x ≈ 0 (centred on the camera axis). Slide it 6 cm along x with the ruler. The program should now say x ≈ 60 mm; a few mm off is human error. If it is far off, recalibrate.
What we changed from the video, and why
VideoToday
sudo pip install opencv-python numpysudo apt install python3-opencv python3-picamera2 or a venv --system-site-packages: sudo pip is blocked or harmful on Bookworm
camera read as a webcam streamPicamera2 (libcamera); --usb 0 still works for a USB webcam
chmod 777 on the scriptnot needed with python3 script.py (and 777 gives everyone write access)
cv2.aruco.estimatePoseSingleMarkerscv2.solvePnP(..., cv2.SOLVEPNP_IPPE_SQUARE); the old function was removed in OpenCV 4.7
Same resolution everywhere

The test script must use the same width and height as the calibration (both 640 × 480 in the video). K is in pixels: at 1280 × 720 every entry is wrong.

✏️ Check yourself

  1. Why does the script ignore a frame if the board is in the same place as an earlier one?
    Show answer

    Twenty copies of the same view add almost no information. Different positions, sizes and tilts are what pin down K and the distortion.

  2. The ruler says 60 mm, the program says 75 mm. Name two likely causes.
    Show answer

    Wrong marker size passed to the script (or wrong square size during calibration), or a different resolution than the calibration. Also possible: a bad calibration (blurry or too similar frames).

  3. Why does the video show the board at the edges first?
    Show answer

    Distortion is largest near the edges; without corners there the model cannot learn it.

▶ Lab 6I in Colab View notebook Labs page The whole workflow on a virtual 640 × 480 Pi camera (frame filter, calibration, 60 mm ruler test), then on your real Pi.

Cheat sheet & troubleshooting

ProblemFix
findChessboardCorners always returns FalseCount inner corners (a 10 × 7-square board is 9 × 6); show the whole board; avoid glare
RMS > 1 pxRemove blurry/outlier views (per-view error), hold the board still, glue it flat
Edges still curved after undistortAdd views with the board near the image corners; wide lens → CALIB_RATIONAL_MODEL or cv2.fisheye
Distances wrong by a constant factorWrong square size or marker size
Everything wrong after changing resolutionRecalibrate, or scale fx, fy, cx, cy by the resize factor
corners[0][0] is a number, not (x, y)OpenCV 5 returns (N, 2): use corners.reshape(-1, 2)
Pi camera not opening with VideoCapture(0)Use Picamera2 (Bookworm uses libcamera)

Back to the Lecture 6 slides · all Lab 6 notebooks.

Videos © their authors (Cyrill Stachniss, Kevin Wood, Nicolai Nielsen, The Drone Dojo), embedded from YouTube. Notes and code on this page were written for this course from the videos' content.