Photosensitive matrices in digital cameras

Matrix on the printed circuit board of a digital camera.
This term has other meanings, see Matrix.

Matrix

or
photosensitive matrix
- a specialized analog or digital-analog integrated circuit consisting of photosensitive elements - photodiodes.

• Designed to convert an optical image projected onto it into an analog electrical signal or into a digital data stream (if there is an ADC directly in the matrix).
• It is the main element of digital cameras, modern video and television cameras, cameras built into mobile phones, CCTV cameras and many other devices.
• It is used in optical detectors of the movement of computer mice, barcode scanners, flatbed and projection scanners, astro- and solar navigation systems.

Content

• 1 Design of one matrix pixel 1.1 Example of a subpixel of a CCD matrix with an n-type pocket
• 1.2 Subpixel microlens

Camera matrix

Nowadays you won’t surprise anyone with a digital photo camera, each of which is equipped with a camera matrix. What is a camera matrix, why is it called a digital camera matrix, what are its functions.

Almost two centuries have passed since the first prototype of the camera was created. The principle of operation of the camera remains the same: light flow enters through the lens and is fixed on the photosensitive element. Previously, film elements with their inherent chemical reaction were used. The new era of cameras brought us digital cameras.

The matrix of a camera, or rather the matrix of a digital camera, is an electronic circuit consisting of millions of tiny light-sensitive diodes that react to the light flux falling on them. One such digital camera matrix LED brings exactly one pixel to your image.

Now imagine a camera sensor that transmits 12 million pixels. Difficult? Not at all: 12 megapixels is the area of ​​the matrix in pixels. For example, if the aspect ratio of the matrix is ​​3:4, then the matrix of a digital camera will have 3 thousand pixels in a column and 4 thousand such columns.

What does the camera matrix look like? What is the physical size of the camera matrix?

A feature of the digital camera matrix electronics is the accumulation of an eclectic charge depending on the amount of light falling on the camera matrix. If there is an excess of energy on a pixel or group of pixels of a digital camera matrix, then this energy begins to transfer to neighboring pixels. As a result, when you photograph the sun, you get a light beam of different circumferences.

It is important to know: the higher quality and more expensive the matrix, and most importantly, the larger the physical size of the matrix of a digital camera, the greater the distance between its pixels, the less noticeable the effect of energy distribution on neighboring pixels.

The number of pixels on the matrix should increase with increasing quality and/or size of the digital camera matrix. Otherwise, the new pixels lose their effectiveness. The matrix size of a digital camera is an important characteristic!

First of all, what is it? Previously, in the era of film cameras, this was simple - instead of a matrix there was a light-sensitive film negative. The standard was 35mm (physical size 24x36mm). In a modern digital camera, instead of film, a photosensitive matrix is ​​installed - an integrated circuit consisting of photosensitive elements (photodiodes). The matrix is ​​designed to convert the optical image projected onto it into a digital data stream. The photomatrix digitizes (“cuts” into pixels) the image that is formed by the camera lens.

There are several types of matrices used in digital cameras, the main ones being CCD and CMOS. The CCD matrix provides better performance when shooting dynamic and small objects, it has a low noise level and a high fill factor. The CMOS matrix is ​​used in products for which the final cost is critical, due to its low cost and low power consumption.

So, the physical size of the matrix is ​​. It should be noted that the physical size of the matrix is ​​one of the most important characteristics of the camera that affects the quality of the resulting photographs. Physical size is its geometric size (length and width in millimeters). However, most often the sizes of photosensors are most often indicated in the form of fractional parts of an inch, for example 1/2.5″. Since this quantity is the inverse, then, accordingly, the size of the matrix is ​​larger if the number after the fraction is smaller. For example, here is the ratio of the most commonly used matrices:

It’s easier to focus not on the matrix size in inverse inch values, but on the crop factor. The crop factor is a coefficient that shows how many times the camera matrix is ​​smaller than the full format. For example, for the most common matrix size of modern point-and-shoot cameras 1/2.3″, the crop factor will be 5.62, i.e. the matrix is ​​5.62 times smaller than the full-format one.

The size of the matrix affects the amount of digital noise transmitted along with the main signal to the matrix. The presence of digital noise, in turn, gives the photograph an unnatural appearance and gives the impression that a matte film has been applied to the photograph. The larger the physical size of the matrix, the larger its area and the more light it receives, resulting in a stronger signal from the matrix and a better signal-to-noise ratio. This allows you to get a brighter, higher-quality picture with natural colors.

The device of one pixel matrix

Manufacturers have different pixel architectures. The CCD pixel architecture is shown here as an example.

Example of a CCD subpixel with an n-type pocket

Diagram of subpixels of a CCD matrix with an n-type pocket (using the example of a red photodetector)
Symbols on the diagram of a subpixel of a CCD matrix

— matrices with an n-type pocket:
1
— photons of light passing through the camera lens;
2
— subpixel microlens;
3
— R — red subpixel filter, Bayer filter fragment;
4
- transparent electrode made of polycrystalline silicon or an alloy of indium and tin oxide;
5
- silicon oxide;
6
- n-type silicon channel: carrier generation zone - internal photoelectric effect zone;
7
- potential well zone (n-type pocket), where electrons from the charge carrier generation zone are collected;
8
- p-type silicon substrate.

Subpixel microlens

Main article: Microlenses

Buffer shift registers on the CCD matrix, as well as the frame of the CMOS pixel on the CMOS matrix, “eat up” a significant part of the matrix area, as a result, each pixel gets only 30% of the photosensitive area of ​​its total surface. For a matrix with full-frame transfer, this area is 70%. This is why most modern CCDs have a microlens installed above the pixel. Such a simple optical device covers most of the area of ​​the CCD element and collects the entire fraction of photons incident on this part into a concentrated light flux, which, in turn, is directed at a fairly compact photosensitive area of ​​the pixel.

Characteristics of matrices

Light sensitivity (sensitivity in short), signal-to-noise ratio and physical pixel size are clearly interrelated (for matrices created using the same technology). The larger the physical size of the pixel, the greater the resulting signal-to-noise ratio for a given sensitivity, or the higher the sensitivity for a given signal-to-noise ratio. The physical size of the matrix and its resolution uniquely determine the pixel size. Pixel size directly determines such an important characteristic as photographic latitude.

Signal to noise ratio

Main article: Signal-to-noise ratio

Every physical quantity undergoes some fluctuations from its average state; in science this is called fluctuations. Therefore, every property of any body also changes, fluctuating within certain limits. This is also true for such a property as the photosensitivity of a photodetector, regardless of what this photodetector is. The consequence of this is that a certain quantity cannot have any specific value, but changes depending on the circumstances. If, for example, we consider such a photodetector parameter as “black level”, that is, the signal value that the photosensor will show in the absence of light, then this parameter will fluctuate in some way, including this value will change from one photosensor to another , if they form some array (matrix).

As an example, we can consider ordinary photographic film, where the photo sensors are grains of silver bromide, and their size and “quality” change uncontrollably from point to point (the manufacturer of the photographic material can only provide the average value of the parameter and the magnitude of its deviation from the average value, but not the specific values ​​themselves this value in specific positions). Due to this circumstance, film developed without exposure will show some, very small, but non-zero blackening, which is called “veil”. And the same phenomenon is observed in the photomatrix of a digital camera. In science, this phenomenon is called noise, since it interferes with the correct perception and display of information, and in order for the image to well convey the structure of the original signal, it is necessary that the signal level to some extent exceed the noise level characteristic of a given device. This is called signal-to-noise ratio.[1]

Sensitivity

Main article: Photosensitivity of photographic material

A term equivalent to “sensitivity” is applied to matrices because:

• depending on the purpose of the matrix, the formal sensitivity value can be determined in various ways according to various criteria;
• Using analog signal amplification and digital post-processing, you can change the sensitivity value of the matrix in a wide range.

For digital cameras, the equivalent sensitivity value can vary in the range of 50-102400 ISO. The maximum sensitivity used in mass-produced cameras corresponds to a signal-to-noise ratio of 2-5.

Permission

Main article: Resolution (optics) § Resolution of digital camera matrices

The photomatrix digitizes (divides into pieces - “pixels”) the image that is formed by the camera lens. But, if the lens, due to its insufficiently high resolution, transmits TWO luminous points of the object, separated by a third black one, like one luminous point on THREE consecutive pixels, then there is no need to talk about the exact resolution of the image by the camera.

In photographic optics, there is an approximate relationship [2]: if the resolution of the photodetector is expressed in lines per millimeter (or in pixels per inch), we denote it as M {\displaystyle M} , and also express the resolution of the lens (in its focal plane ), let us denote it as N {\displaystyle N} , then the resulting resolution of the lens+photodetector system, let us denote it as K {\displaystyle K} , can be found by the formula:

1 K = 1 N + 1 M {\displaystyle {\frac {1}{K}}={\frac {1}{N}}+{\frac {1}{M}}} or K = NMN + M {\displaystyle K={\frac {NM}{N+M}}} .

This ratio is maximum at N = M {\displaystyle N=M}, when the resolution is N 2 {\displaystyle {\frac {N}{2}}}, therefore it is desirable that the resolution of the lens matches the resolution of the photodetector.[ clarify

]

In modern digital photo matrices, the resolution is determined by the pixel size, which varies for different photo matrices ranging from 0.0025 mm to 0.0080 mm, and for most modern photo matrices it is 0.006 mm. Since two points will be different if there is a third (unexposed) point between them, then the resolution corresponds to a distance of two pixels, that is:

M = 1 2 p {\displaystyle M={\frac {1}{2p}}} , where p {\displaystyle p} is the pixel size.

Digital photo matrices have a resolution of 200 [ source not specified 2829 days

] lines per millimeter (for large-format digital cameras) up to 70[
source not specified 2829 days
] lines per millimeter (for web cameras and mobile phones).

Some developers of video cameras, CCDs and CMOS matrices, consider the system resolution (in lines) to be equal to the number of pixels read from the matrix, divided by 1.5. Since when assessing the resolution of a lens, measurements are taken in pairs of black and white lines of the Foucault world per mm (determining not a single peak, but a spatial frequency), the conversion factor of the matrix resolution into pairs of lines requires a correction factor of 3.0 [3].

Physical matrix size

Main article: Crop factor

Comparative matrix sizes

The physical dimensions of photosensors are determined by the size of individual pixels of the matrix, which in modern photosensors have a value of 0.005-0.006 mm. The larger the pixel, the larger its area and the amount of light it collects, therefore the higher its light sensitivity and the better its signal-to-noise ratio (in film photography, noise is called “grain” or “granularity”). The required resolution of photographic details determines the total number of pixels, which in modern photo matrices reaches tens of millions of pixels (Megapixels), and thus determines the physical dimensions of the photo matrix.

• The laws of optics determine the dependence of the depth of field on the physical size of the matrix. If you photograph the same scene with the same angle of view and the same aperture value on the lenses using three cameras with different physical matrix sizes, and study the result (a file on a computer, a printout from a printer) under the same conditions, then the depth of field on a picture taken with a camera with the smallest matrix will be the largest (more objects in the frame will be shown sharply), and a camera with the largest matrix will show the smallest depth of field (objects not in the sharpness zone will be more blurred).
• Photosensor sizes are most often referred to as a “type” in fractional parts of an inch (for example, 1/1.8″ or 2/3″), which is actually larger than the actual physical diagonal size of the sensor. These designations originate from the standard designations for television camera tube sizes in the 1950s. They do not express the diagonal size of the matrix itself, but the external size of the transmitting tube bulb. Engineers quickly determined that, for various reasons, the diagonal of the usable image area was about two-thirds the diameter of the tube. This definition has become established (although it should have been abandoned long ago). There is no clear mathematical relationship between the “type” of a sensor, expressed in inches, and its actual diagonal. However, as a rough approximation, we can assume that the diagonal is two-thirds of the standard size.

Physical dimensions of matrices

Matrix diagonals 1'', 1/2'', etc. are usually measured in Vidicon inches. The size of a vidicon inch is 16 mm (inherited from a vidicon with a diameter of 1″, the working diagonal there was exactly 16 mm). As can be seen from the table on the crop factor of a 1'' matrix with a diagonal of 16 mm, the size of a full 24x36 mm matrix has a diagonal of 43.27 mm or 2.7 Vidicon inches, 2.7''. The physical dimensions of the video camera matrix are different depending on the aspect ratio (4:3 or 16:9) and the specific manufacturer with the same diagonal. Therefore, for example, a camera on a 1/3'' matrix with an aspect ratio of 4:3 gives a larger viewing angle vertically and a smaller one horizontally than a camera on a matrix with the same diagonal but an aspect ratio of 16:9[4].

Frame aspect ratio

• The 4:3 frame format is mainly used in amateur digital cameras. Some companies, for example, Canon, allow these cameras to adjust the aspect ratio in the ranges of 4:3 and 16:9[5].
• The 3:2 frame format is used in SLR digital cameras, except those made according to the 4:3 standard.
• A small number of models with a 16:9 frame are produced.
• Olympus digital SLR cameras use a sensor with an aspect ratio of 4:3 (standard 4:3).

Pixel aspect ratio

Matrices are available with three different pixel proportions:

• For video equipment, sensors with a pixel aspect ratio of 4:3 (PAL) are available.
• or 3:4 (NTSC);
• Photographic, radiographic and astronomical equipment, as well as the now developing HDTV video equipment, usually have a square pixel.

Matrices for CCTV cameras. What to look for?

The image quality of a video camera largely depends on the light-sensitive sensor (matrix) used in it. After all, even if you install the best processor for video digitization, if a bad image is obtained on the matrix, it will no longer become good. I will try to explain in a popular way what you should pay attention to in the characteristics of a CCTV camera sensor, so that later you don’t experience excruciating pain when looking at the image... Matrix type
On the Internet you will probably find information that CCTV cameras use CCD (charge-coupled device) ) and CMOS (CMOS, complementary metal-oxide-semiconductor structure) light-sensitive matrices. Forget it! For a long time there was only CMOS, only hardcore.

CCD matrices, for all their advantages (better photosensitivity and color rendition, lower noise level) are practically no longer used in video surveillance. Because the very principle of their operation of CCD matrices - sequential charge reading across cells - is too slow to satisfy the demands of fast modern high-resolution video cameras. Well, most importantly, CCD is more expensive to produce, and in today’s highly competitive environment, every penny of profit counts. That is why all key manufacturers have focused on producing CMOS matrices.

By the way, there are not many manufacturers left. The largest companies, as of the beginning of 2021, are: ON Semiconductor Corporation (which at one time absorbed the well-known specialized company Aptina), Omnivision Technologies Inc., Samsung Electronics and Sony Corporation. In addition, matrices for their own needs are produced, for example, by Canon and Hikvision.

Young, full of enthusiasm and money, Chinese “second-tier” chipmakers, such as SOI (Silicon Optronics, Inc.) and others, are trying to create competition for old brands. It is difficult to say whether the young shoots will survive when the CMOS sensor market becomes saturated and becomes too crowded . But in any case, the emergence of new players and intensification of the struggle in this segment cannot be ruled out, because setting up the production of CMOS sensors is not a very difficult task by modern standards.

Large global brands like Hikvision or Dahua usually prefer to work with first-tier matrix manufacturers or their own.
Local ones behave differently. For example, Tecsar uses reputable matrices from ON Semiconductor, Omnivision and Sony even in inexpensive cameras. In the assortment of other “popular” brands, for example Berger, SOI sensors, etc. are widely represented. How digital camera sensors are made
Leadership qualities of CMOS

CMOS technology involves placing electronic components (capacitors, transistors) directly in each pixel of the photosensitive matrix.

Pixel structure and CMOS matrix

This reduces the useful area of ​​the photosensitive element and reduces sensitivity, plus the active elements increase the level of the matrix’s own noise. But the technology makes it possible to convert the charge of the photosensitive element into an electrical signal directly in the matrix and generate a digital image signal much faster, which is critical for video cameras. This is why CMOS is better suited for CCTV cameras that require fast frame changes.

The principle of operation of CCD and CMOS matrices
Plus the ability to randomly read CMOS matrix cells makes it possible to literally change the quality and bitrate of the resulting video “on the fly”, which is impossible for CCD. And the power consumption of CMOS solutions is lower, which is also important for compact surveillance cameras.

Let there be color

To obtain a color image, the matrix decomposes the light flux into its component colors: red, green and blue. For this, appropriate filters are used. Different manufacturers vary the placement and number of photosensitive elements of different colors, but the essence does not change.

The principle of image formation on a photosensitive matrix:

P – photosensitive element T – electronic components

You can also see how the CMOS camera sensor works and works in this video from Canon:

CMOS matrices from all manufacturers are based on the general principles described above, differing only in the details of implementation on silicon. For example, in pursuit of low cost and super-profits, chipmakers try to produce matrices as small as possible. Retribution for this is inevitable...

Why big is good

The standard size (or in other words, format) of the matrix is ​​usually measured diagonally in inches and indicated as a fraction, for example 1/4″, 1/3″, 2/3″, 1/2″, etc.

The first rule for choosing the best matrix is ​​quite simple: with the same number of pixels (resolution), the larger the physical dimensions of the sensor, the better. A larger sensor has larger pixels, which means it captures more light. The pixels of a larger matrix are located less closely, which means there is less interference and a lower level of spurious noise, which directly affects the quality of the resulting image. Finally, the larger sensor allows for greater viewing angles when using the same focal length lens!

Photosensitive matrix produced by ON Semicondactor for CCTV cameras

Photosensitive matrix installed on the video camera board

Alas, large-format matrices are now practically not used in mass surveillance cameras due to the high cost of both the matrices themselves and the lenses for them, which must have larger lenses and, accordingly, dimensions and cost. Today, cameras are mostly equipped with matrices of 1/2″ – 1/4″ sizes (these are the tiniest). When choosing a camera, you need to clearly understand that if you buy an ultra-cheap model with a 1/4″ matrix produced by SOI and a tiny lens with plastic lenses of dubious transparency, you will not be able to create a video monitoring system of acceptable quality, on which you could clearly distinguish small details of captured events, especially when shooting in low light conditions.

By choosing a camera with a Sony 1/2.8″ matrix, you will a priori get a much better result in video quality; a camera with such a matrix can already be used in a professional video surveillance system. And the sensitivity of such a camera will obviously be higher, which will allow better shooting in low light conditions: in bad weather, at dusk, in a dim room, etc. As the resolution increases with the same matrix size, the photosensitivity decreases, and this also needs to be taken into account when choosing. For a camera installed in a dark alley near the back door, it makes sense to choose a matrix with a lower resolution and higher sensitivity than an ultra-high-resolution camera with a low matrix sensitivity on which, due to noise, nothing can be clearly distinguished.

Photosensitivity

The light sensitivity of the matrix determines its ability to operate in low ambient light conditions. From a physics point of view, this looks completely banal: the less light energy is enough for the matrix to produce an image, the higher its photosensitivity. But! Let's be honest, it's not really worth pursuing high sensitivity anymore. The fact is that modern video surveillance cameras safely switch to “day/night” modes, switching the matrix into black-and-white image mode with a higher sensitivity when the illumination decreases. Plus, the automatic inclusion of infrared illumination gives the cameras the ability to take excellent pictures even in complete darkness. For example, in a closed room without windows and with the lights turned off, when there is no question of any level of external illumination. Light sensitivity remains critical for cameras without IR illumination, but using such cameras in modern video surveillance is almost bad manners. Although case models without backlight are still sold, of course.

Comparison of matrices from different manufacturers
In general, the rule is this: the higher the illumination, the better the matrix and, accordingly, the camera will shoot. Therefore, it is not recommended to place cameras in dimly lit corners, even if they have good sensitivity. Keep in mind that camera sensor specifications usually indicate the minimum level of illumination at which any image can be captured. But no one promises that this image will be of at least acceptable quality! It will be disgusting 100% of the time, it will be difficult to make out anything on it. To achieve at least a satisfactory result, it is recommended to shoot at least with illumination at least 10-20 times greater than the minimum permissible for the matrix.

Manufacturers have come up with a number of technical solutions to improve the sensitivity of CMOS sensors and reduce light loss during image capture. For this, one principle is mainly used: to bring the photosensitive element as close as possible to the microlens of the matrix that collects light. First, Sony offered its Exmor technology, which shortened the path of light through the matrix:

Then progressive manufacturers unanimously switched to the use of matrices with back illumination, which allows not only to shorten the path of light through the matrix, but also to make the usable area of ​​the photosensitive layer larger, placing it above other electronic elements in the cell:

Back-illumination technology gives the camera maximum sensitivity. Hence the conclusion - “all other things being equal” it is better to purchase a camera using a matrix with back illumination than without it.

To improve images in low light conditions for low-sensitivity, cheap sensors, camera manufacturers can use various tricks. For example, the “slow shutter” mode, or, more simply, the long shutter speed mode. However, the “smearing” of the contours of moving objects already at the stage of fixing the image with the matrix in this mode does not allow us to talk about more or less high-quality video recording, therefore this approach is completely unacceptable in security video surveillance, where details are important.

A definite breakthrough in image quality was the advent of Starlight technology, which first appeared in Bosch cameras in 2012. This technology, thanks to the combination of the enormous light sensitivity of the matrix (about 0.0001 - 0.001 lux) and very effective noise reduction technology, made it possible to obtain very high-quality color images from video cameras in low light conditions and even at night.

While the traditional method of overcoming low light - using IR illumination - makes it possible to obtain a clear image only in monochrome mode (grayscale), cameras with Starlight technology allow you to obtain a color image that is much more informative. In particular, in low light conditions, a video surveillance system with Starlight technology can easily distinguish the colors of cars, clothing, and other important features.

Here's a demo of Starlight technology in action:

Results

When choosing a CCTV camera, be sure to pay attention to the characteristics of the matrix, and not just its resolution. After all, the quality of the image, and therefore the usefulness of the camera, will largely depend on this. First of all, you should pay attention to a reliable brand, size and resolution of the matrix; photosensitivity is important only for cameras without IR illumination.

I highly recommend buying a camera with a matrix, from which you can find a sane datasheet with detailed information, and not buying a pig in a poke. For example, you can easily find specifications for matrices manufactured by ON Semiconductor, Omnivision or Sony. But more or less detailed characteristics of SOI matrices cannot be found during the day with a flashlight. There is a suspicion that the manufacturer has something to hide...

And the overall result is this: CMOS matrices have unconditionally won in video surveillance devices and are not going to give up to any competing technology in the near future.

Types of matrices according to the technology used

• CCD matrix (CCD, “Charge Coupled Device”);
• CMOS matrix (CMOS, “Contact Metal Oxide Semiconductor”);
• SIMD WRD (Wide dynamic range) matrix;
• Live MOS matrix;
• Super CCD matrix.
• QuantumFilm matrix.

For a long time, CCD matrices were practically the only mass-produced type of photosensor. The implementation of Active Pixel Sensors technology around 1993 and further development of technologies ultimately led to the fact that by 2008, CMOS matrices became practically an alternative to CCDs[6].

CCD matrix

Main article: CCD

The CCD matrix (CCD, “Charge Coupled Device”) consists of light-sensitive photodiodes, is made on the basis of silicon, and uses CCD technology - charge-coupled devices.

CMOS matrix

Main article: CMOS sensor

The CMOS matrix (CMOS, “Contact Metal Oxide Semiconductor”) is based on CMOS technology. Each pixel is equipped with a readout amplifier, and the signal from a specific pixel is sampled randomly, as in memory chips.

SIMD WRD (Wide dynamic range) matrix, also made on the basis of CMOS technology, also has an automatic system for adjusting its exposure time in the frame of each pixel, which allows you to radically increase the photographic width of the device [7].

Live MOS matrix

Main article: Live-MOS matrix

Created and used by Panasonic. It is based on MOS technology, but contains fewer connections per pixel and is powered by a lower voltage. Due to this and due to the simplified transmission of registers and control signals, it is possible to obtain a “live” image in the absence of overheating and increased noise levels, which are traditional for such an operating mode.

Super CCD matrix

Fujifilm cameras use matrices called “Super CCD”, which contain green pixels of two different sizes: large, for low light levels, and small, the same size as blue and red. This allows you to increase the photographic latitude of the matrix by up to 4 stops[8].

QuantumFilm

Matrix based on quantum dots. A promising technology, unlike traditional silicon image sensor chips, the use of quantum dots is more efficient at capturing light (capturing 90-95% of the light hitting the sensor), providing high sensitivity in low light conditions, as well as higher dynamic range. While most traditional image sensors are subject to the rolling shutter effect, which is critical for shooting video[9].

A look from the inside: digital camera matrices

Preface

Two months ago, in an article on comparing LCD and E-Ink displays, I mentioned that one of the next reviews would be a “dissection” of the matrix of a modern camera.
And I hasten to fulfill this promise! The first to be included in the “collection” of photosensitive matrices were the front and rear cameras of a smartphone from one well-known Korean manufacturer, which was kindly provided by Vasily Stolyarov. Then the darkwood user, who lives near Moscow, sent me his old, non-working Pentax camera (hereinafter I will deliberately not indicate the exact model of the device). The device was dead and this was a good reason to hand it over into my caring hands, and not throw it away, as many do. And just as I was about to cut, I received another offer from my practically classmate, Ilya. I couldn't refuse this offer. I was presented with a relatively modern Canon that had problems taking images.

Thus, there are three candidates on the red-revolutionary-May Day table: an OEM camera from a phone and cameras from Pentax (the oldest among all participants) and Canon (perhaps the youngest).

If anyone else doesn’t know why we are gathered here, then in the footer of this article there are links to previous “autopsy”. If someone has forgotten how a digital camera works or why a matrix is ​​needed, then you are welcome to Wiki or just watch this video from the Discovery Channel:

The theoretical part. CCD and CMOS

Today, matrices made using CMOS (Complementary Metal-Oxide Semiconductor) technology have conquered more than 90% of the world market, and not so long ago, the insanely popular CCD (Charge-Coupled Device) is already predicting a quick decline.
There are many reasons for this, here is not a complete list of the advantages of CMOS technology: firstly, low power consumption in a static state compared to CCD, secondly, CMOS immediately “produces” a digital signal that does not require additional conversion (more precisely, the conversion occurs on each individual subpixel), unlike CCD, which is actually an analog device; thirdly, low cost of production, especially with large matrix sizes.

You can briefly get acquainted with the principles of operation of CMOS matrices using two videos from Canon:

But all of our patients (maybe with the exception of the mobile phone camera matrix) belong to the era when CCD reigned supreme in the world, and CMOS was just gaining strength and light sensitivity to subsequently take a leading position. Therefore, I will still say a few words about how the CCD matrix works. A more detailed description can always be found on the Wiki pages.

So, a photon from the subject, passing through a Bayer filter, that is, an RGBG color filter, or an RGBW filter and a collecting microlens, hits the photosensitive semiconductor material. Being absorbed, the photon generates an electric-hole pair, which is “separated” in the cell under the influence of an external electric field, and the electron is “sent” to the piggy bank - a potential well, where it will wait for “reading”.

Diagram of the CCD matrix device (Source)

Reading the CCD matrix occurs “cell by cell”, so to speak. Let's say we have a 5 by 5 pixel array. First, we read the number of electrons, or simply the amount of electric current, from the first pixel. Then a special controller “shifts” all the cells by one, that is, the charge from the second cell flows into the first. The value is read again and so on until all 5 cells have been read. Next, another controller shifts the remaining “image” down one line and the process is repeated until the currents are measured in all 25 cells. This may seem like a long process, but for 5 million pixels it takes just a fraction of a second.

The process of reading an image from a CCD matrix (Source)

To make it completely clear, I suggest you watch the following videos:

Practical part

Usually, beautiful disassemblies are carried out by people in snow-white gloves, recently they got to cameras, but they say that for video assembly instructions you need to pay extra by sending an SMS to a short number.
Further, slightly more than completely clumsy methods will be used, so I do not recommend repeating this at home... How to disassemble a cell phone can always be seen on the pages of the previous article, so I will not present these heartbreaking shots here again.

The above-mentioned Pentax camera was provided by Monsieur DarkWood, whose heart, it seems to me, should now be bleeding, and a stingy male tear rolling down his cheek:

Pentax disassembly in photographs

Of all the variety of details, we are currently only interested in the LCD display, which will be shown to schoolchildren who come to us, on FNM, on excursions, the CCD matrix itself, glass with something suspiciously reminiscent of a polarizer or filter, and IR illumination (red light) for night shooting. It is worth noting that the matrix is ​​rigidly fixed to the camera body. Consequently, all the vibrations of your hands will be easily transmitted directly to the matrix itself, which, you see, does not in any way contribute to high-quality photography. Apparently, DarkWood has reinforced concrete nerves.

However, that didn’t stop him from “drowning” his favorite camera. Remember, when you go to warm countries at sea in the summer and try to photograph the next rolling wave, that a camera is a device in which currents can lead to corrosion.

Traces of corrosion right on the cable leading to the shutter button (unfortunately, not the only place like this)

It is immediately clear that Canon is a slightly more advanced, more modern model than Pentax. For example, the matrix is ​​spring-loaded (small springs are clearly visible in the lower left image). Such a passive image stabilization system helps to obtain higher quality and clearer pictures, unless, of course, you are a neurasthenic in an advanced stage!

"Insides" Canon

By the way, in the photo at the bottom right you can clearly see a huge capacitor responsible for the flash, because of problems with which I once had to write off my Canon digital point-and-shoot camera.

Mobile phone camera

Let's start our research with a mobile phone camera, which will not be devoted much time and words in this article due to the fact that the matrix itself has completely microscopic dimensions and is difficult to work with (sawing, grinding).
As it is not difficult to notice, in the optical micrographs below the matrix at the edge has two zones: a lighter and a darker one. I hope that everyone has already guessed: there are no diodes under the bright side, it is applied just like that, with a margin in order to maximally cover the subtle spiritual organization of the matrix...

We'll cover everything with reserve - we don't mind

Microphotographs obtained using an optical microscope differ significantly from those produced by an electron microscope. For example, what about “sphere quadrature”?

The fact is that in optics we do not see any transparent layers (even if they are simply less noticeable), while electron microscopy is primarily a method of surface analysis, that is, it may well be that round color filters are covered on top square "caps". Moreover, the dimensions of such a cubic spherical subpixel are about 2.5 micrometers.

This is how it is, the squaring of the sphere...by the way, in a vacuum...

Pentax camera matrix

Let's start our study of the CCD matrix of a Pentax camera with optical microphotographs. To my deep regret, due to steric hindrances, as chemists say, in the sample-microscope system, it was not possible to take pictures at high magnifications and examine individual subpixels.

Something is written, I wonder, is it possible to see the names of little Chinese children here somewhere?

Each landing pad for contacts is numbered, but not each one has the same contact wire.

And this is how we will soon learn to count – with the help of nanotechnology, of course...

A clear boundary between the matrix itself and the “piping”

And the following micrograph is worthy of a textbook on electron microscopy. Do you know why an electron microscope is not a measuring tool? Yes, yes, that’s why: due to the local accumulation of charge, seemingly spherical objects suddenly became ellipsoids:

But we know that these are spheres...

Next, let's take a look at what is around the photosensitive matrix. Since I am not an expert in the field of creating electronic circuits, I’m afraid to even imagine why all these complex designs and “intricacies” of conductors are needed, maybe there is someone ready to explain the purpose of the parts and components below (in the comments, of course same)?

Unshakable columns that have survived sawing and polishing...

You can get entangled in these layers and break a leg or two.

This issue “A Look from the Inside” is a landmark issue; after several years of “ordeals,” we have finally installed a new microanalysis system, so in some cases, I can not only provide beautiful pictures, but also explain what chemical elements what we see consists of.

And here is the most interesting thing - the matrix in all its glory. Under the mesh, in the cells of which microspheres-lenses are located, you can see individual photosensitive elements (or their remains, to be more precise, it’s difficult to say). Below, when discussing the Canon matrix, I will explain in detail the “cross-section” design of the matrix. For now, let us turn to the data of local chemical analysis. It turns out that the mesh consists of tungsten, and the microspheres, apparently, are silicon dioxide, which is “covered” on top with some kind of polymer material. A more detailed analysis can be found here.

The Matrix in all its intricate beauty

Returning to the first SEM image in this chapter, the contact pads are made of pure gold (oh yeah!), but the conductors inside the sensor appear to consist of aluminum, on which a thin layer of copper is sputtered, the content of which is on the verge of sensitivity of the device. Detailed information is presented here.

Canon camera matrix

We will continue our immersion into the micro- and nanoworlds, as usual, with optical microscopy. As in the case of Pentax, the matrix from the Canon camera could not be photographed at high magnification due to geometric inconsistencies. However, from the obtained microphotographs, it is possible to estimate the size of an individual subpixel - about 1.5 microns, which is much smaller than that of a mobile phone matrix.

Optical micrographs of a Canon sensor

By the way, one of the culprits for the inability to take pictures on an optical microscope at high magnifications is the “cover” glass, which covers the matrix and its “filling”:

Good shot: transmission behind glass

Of course, the most interesting things are always hidden on the chips, where the falling apart strictly ordered world gives a crack, allowing you to look into the most sacramental corners of the device:

We'll come back to the yellowish-orange areas of this photo a little later...

Already familiar to us columns of completely unclear purpose:

Like steadfast tin soldiers

Now let's take a closer look at the structure of the CCD matrix. On top, the CCD matrix is ​​covered with something resembling a polymer layer (1), which protects the photosensitive elements from the aggressive external environment. Below it are microlenses with dye (2 and 3). But since electron microscopy does not allow obtaining color images, it is not possible to say for sure whether a large or small sphere is colored. Microlenses made of silicon dioxide (the most likely material for their manufacture) are fixed in the cells of a tungsten mesh (4), under which photosensitive elements (5) are hidden. And, of course, this entire structure rests on a substrate made of the purest silicon!

Taking into account the fact that the matrix is ​​additionally protected by a “covering” glass, the photocells are protected better than the President of the Russian Federation in his limousine (if, of course, we correct for the scale factor).

Microanalysis data can be downloaded here.

Arrangement of the matrix point by point. Description in the text

But that's not all. We still have a piece of glass remaining covering the matrix, which, it seems, is a polarizer. It is somewhat rough around the edges, but almost perfectly smooth throughout the rest of the surface area. It seems that optical microscopy does not give any results: glass is like glass.

Glass with a suspected polarizer: nothing unusual

And only with the help of electron microscopy is it possible to see the chemical contrast in the image and the striped structure. The thickness of such a “film” is only 2.5 micrometers, while the sizes of individual layers are 180 and 100 nm, respectively, for darker and lighter ones. Based on microanalysis data (here), I would venture to suggest that the darker areas are enriched in titanium, and the light areas are enriched in aluminum. I think this is amazing!

It turns out that the camera has its own striped life inside!

Afterword

Such a world of the passing century of CCD matrices appears before us today.
Thanks to everyone (Vasily for the phone, Ilya and DarkWood for the cameras) who made their contribution to the creation of this article. Well done to you for supporting us in this difficult endeavor!

And the apotheosis of this article, or rather its apotheosis:

Rest in peace until we come up with a new use for you.

Bonus 1. What does a green dust storm look like in Moscow?

At first I wanted to post it as a separate post, but decided not to clutter up the space.
Just a few days ago, Moscow was covered by a yellow-green cloud, many had already begun to prepare for the coming of the apocalypse, but nothing happened... What in reality was the reason for such a strange color? The climate on this planet has been acting up lately: either it will be left without snow for the New Year, or it will be covered to the very top with snow, then spring will be like winter, then suddenly summer will come. Flowers, trees and vegetation are less adapted to this kind of disturbance, so 1.5 months of spring compressed into 1 week forced plants to reconsider their plans for reproduction...

In the morning, sitting down at my desk, I discovered a layer of dust on it, and after wiping it with a napkin, I realized that I needed to study this dust properly. No sooner said than done!

But I have two news for everyone: good and bad.

The good news is that the color of the yellow-green cloud was indeed due to a large amount of pollen (I counted at least three types):

Composition of the Moscow storm: pollen... Below right, pollen on the surface of a part of the plant

The bad news is that we also breathe this, and every day, and not during plant reproduction periods (micro- and nanoparticles that not every filter will catch):

Composition of the Moscow storm: not very pleasant dust and dirt

Bonus 2. Guess what it is?

PS

Recently, a large number of posts on the topic of copyright and “pay what you want” began to appear on HabraHabr. For example, they collect donations for ReactOS or for publishing/writing books. I thought about it and decided to “get involved” in this too, so it’s up to you, dear readers:

Yandex.Money 41001234893231 WebMoney (R296920395341 or Z333281944680)

In a week we managed to collect about 2,500 rubles - THANK YOU to everyone who responded and made their contribution!!!;)

P.P.S.

By the way, I unexpectedly discovered that some of my works for sawing certain objects encourage people to make their own discoveries.
Firstly
, a complete list of published articles on Habré:

Opening of the Nvidia 8600M GT chip, a more detailed article is given here: Modern chips - a look from the inside A look from the inside: CD and HDD A look from the inside: LED light bulbs A look from the inside: LED industry in Russia A look from the inside: Flash memory and RAM A look from the inside: the world around us A look From the inside: LCD and E-Ink displays From the inside: digital camera matrices From the inside: Plastic Logic From the inside: RFID and other tags From the inside: PhD studies at EPFL. Part 1 A look from the inside: postgraduate studies at EPFL. Part 2 A look from the inside: the world around us - 2 A look from the inside: the world around us - 3 A look from the inside: the world around us - 4 A look from the inside: 13 LED lamps and a bottle of rum. Part 1 A look from the inside: 13 LED lamps and a bottle of rum. Part 2 A look from the inside: 13 LED lamps and a bottle of rum. Part 3 A look from the inside: IKEA LED strikes back A look from the inside: are Filament lamps really that good?

and 3DNews: Microview: comparison of displays of modern smartphones

Secondly

, in addition to the blog on HabraHabr, articles and videos can be read and watched on Nanometer.ru, YouTube, and also Dirty.

Third

, if you, dear reader, liked the article or want to stimulate the writing of new ones, then act according to the following maxim:
“pay what you want”
Yandex.Money 41001234893231 WebMoney (R296920395341 or Z333281944680)
Sometimes briefly, and sometimes not very much about science and technology news You can read it on my Telegram channel - you are welcome;)

Methods for obtaining a color image

The photomatrix pixel itself is “black and white”. In order for the matrix to produce a color image, special technical techniques are used.

Three-matrix systems

Main article: 3CCD

An example of how a dichroic prism works

The light entering the camera, hitting a pair of dichroid prisms, is divided into three primary colors: red, green and blue. Each of these beams is directed to a separate matrix (CCD matrices are most often used, therefore the designation 3CCD is used in the name of the corresponding equipment).

Three-matrix systems are used in mid- and high-end video cameras.

Advantages of three matrices compared to single-matrix ones

• better rendering of color transitions, complete absence of color moiré;
• higher resolution: there is no low-pass filter necessary to eliminate moire;
• higher photosensitivity and lower noise level;
• the possibility of introducing color correction by placing additional filters in front of individual matrices, and not in front of the shooting lens, allows you to achieve significantly better color rendition with non-standard light sources.

Disadvantages of three matrices compared to single matrix

• fundamentally larger overall dimensions;
• the three-matrix system cannot be used with lenses with a small flange;
• in a three-matrix scheme there is a problem of color mixing, since such systems require precise adjustment, and the larger the matrices are used and the greater their physical resolution, the more difficult it is to achieve the required accuracy class.

Matrices with mosaic filters

Main article: Color filter array

In all such matrices, the pixels are located in the same plane, and each pixel is covered with a light filter of a certain color. The missing color information is restored through interpolation ( more details...

).

There are several ways to arrange filters. These methods differ in sensitivity and color rendition, and the higher the photosensitivity, the worse the color rendition:

• RGGB - Bayer filter, historically the earliest;
• RGBW have higher sensitivity and photographic latitude (typically a gain of 1.5-2 times in sensitivity and 1 step in photographic latitude), a special case of an RGBW matrix is ​​the CFAK matrix from Kodak;
• RGEB (red - green - emerald - blue);
• CGMY (blue - green - purple - yellow).

Matrices with full-color pixels

There are two technologies that allow you to obtain all three color coordinates from each pixel. The first is used in mass-produced cameras from Sigma, the second - as of mid-2008, exists only in prototype form.

Multilayer matrices (Foveon X3)

Main article: Foveon X3

Photodetectors of the Foveon X3 matrix are arranged in three layers - blue, green, red. The name of the sensor “X3” means its “three-layer” and “three-dimensionality”.

X3 matrices are used in Sigma digital cameras.

Nikon Full Color RGB Matrix

In Nikon full-color sensors (Nikon patent dated August 9, 2007[10]), the RGB rays of the object dots in each pixel, containing one microlens and three photodiodes, pass through the open microlens and fall on the first dichroic mirror. In this case, the blue component is transmitted by the first dichroic mirror to the blue detector, and the green and red components are reflected to the second mirror. The second dichroic mirror reflects the green component to the green detector, and transmits the red and infrared components. The third dichroic mirror reflects the red component to the detector and absorbs the infrared component[11].

Despite the fact that a prototype of the matrix has already been created (2008), this patent is unlikely to find application in the near future due to significant difficulties in the technology.

Compared to all other systems except three-matrix, this technology has a potential advantage in luminous flux efficiency compared to RGBW or Bayer filter technologies. (The exact gain depends on the transmission characteristics of the filters.)

Compared to Foveon X3, this technology wins in color rendering quality.

Compared to 3CCD systems, this type of matrix benefits from the possibility of being used in SLR devices and in the absence of the need for precise alignment of the optical system[10].

Notes

1. Signal-noise, digital devices and astrophotography original in English
2. About resolution
3. [1] Odinokikh G.A., Youth scientific and technical bulletin # 12, December 2013, UDC: 621.397.7
4. IP cameras, megapixel cameras for video surveillance via the Internet. Network cameras for home video surveillance - ip cameras
5. about 16:9 format in canon devices
6. CCD vs CMOS: facts and fictions (English)
7. description of WDR camera Pelco CCC5000 Pixim
8. Description of the Fujifilm S5 Pro camera (English)
9. A quantum dot sensor has been developed for cameras. www.membrana.ru. Retrieved January 10, 2021.
10. ↑ 12
    US Patent 7,138,663
11. about Nikon matrix