The 2,000 year-old technology (and a lot of new tech) behind your smartphone’s cameras

No invention exists in a vacuum. Every bit of technology we use stands on the shoulders of prior insights and discoveries. Bluetooth, the humble connectivity standard for your headphones and wireless keyboards, is built on more than 30,000 patents. Smartphones, on the other hand, incorporate more than 100,000 patents, and the best smartphones all have unique patents that help them stand out from the competition. And each of those patents is built on prior art that came before it. Some of that technology goes back years, some goes back decades, and some goes back more than 2,000 years.

Camera lenses and optical technology

One of the most important aspects of taking a photograph is getting the light that’s outside of your camera to your film or photosensor. But light isn’t a monolith. Light isn’t just a spectrum of colors. It’s also a spectrum of wavelengths, and getting those diverse wavelengths to diffract and rejoin in a coherent manner was the work of centuries of scientists.

The early lens

The lens, which bends the light that passes through it, can be dated back to the 5th century B.C. The earliest lenses were used for lighting fires by concentrating the sun’s energy onto a single point. However, writers in the early Roman Empire were aware of the lens’ capacity for magnification.

Glasses were invented in the late 13th century, which gave rise to the optics industry. Around 1600, someone (it’s not clear who) stumbled upon the idea of using more than one lens, which gave rise to the telescope and the microscope.

The camera obscura

Paralleling the development of the lens was the camera obscura, an enclosed box or room that only allows light in through a small hole (aperture). That light is then projected on the surface opposite the hole. It was first described over 2,000 years ago and was mentioned by Aristotle as being useful for studying eclipses without having to look directly at the sun.

By the 16th century, lenses had been incorporated into the apertures to help focus the light. For the next 200 years, it remained a tool of artists and scientists. In the 19th century, the secrets of chemical photography were unlocked by Niépce and Daguerre, which was essentially a means of capturing the lens-focused image of a camera obscura.

Camera lenses

The earliest cameras used a single lens to focus light. To overcome the optical aberrations inevitable in using one lens, lensmakers found ways to overcome some of these limitations by using two lenses made of different types of glass. As the century progressed, progress in lenses was marked by advances in understanding the shape and configuration of the lens elements.

Modern lenses started to emerge at the end of the 19th century with the development of the anastigmat lens. These lenses corrected for astigmatism, an aberration that causes areas outside the center of the shot to be out of focus. This was the first lens to correct for most optical aberrations that had plagued photographers up until that point.

In the 20th century, two developments took place that laid the groundwork for mobile photography in the 21st century. The first was aspheric lenses, which had been known about for hundreds of years but became viable for commercial use in the 1900s. The profile of most handheld camera lenses is spherical, but the more complex geometry of aspheric lenses allowed them to replace more intricate multi-element lenses.

The next major development was the rise of plastics. In the 1930s, the first optical-quality plastic lenses were developed. Kodak led the way with plastic lenses in their cameras in the 1960s, selling millions of plastic-lensed Instamatic cameras. The low cost and rapid fabrication of plastic lenses (compared to the optically superior glass lenses) led to a low-cost photography boom in the 1970s, with some cameras costing less than $10.

These two developments, plastic and aspheric lenses, directly address some of the biggest problems that early mobile phone camera makers would have to face: size and cost. Plastic lenses could be made cheaper and more rapidly, and aspheric lenses could overcome some of the optical deficiencies of plastic and meant fewer lenses would need to be used.

Image sensors and semiconductor technology

Long before the digital camera was a notion, scientists and inventors were familiar with the tendency of certain chemicals to react to light. As soon as it was discovered how to arrest the photochemical reaction in the 19th century, the first photographs became possible. By the early 20th century, the quantum nature of light and the photoelectric effect had been firmly established, opening the theoretical door for electronic pictures. Even so, it would take decades and a revolution in technology to make the cameras on our phones a possibility.

Integrated circuits

The genesis of today’s image sensors lies in the search for a way to combine multiple electronic components into a single device. By the 1950s, vacuum tubes were being replaced with transistors, but the complexity of computers was growing beyond human ability to maintain them. The problem was in the number of components that had to be maintained and connected to each other. If anything stopped working, it could take days or weeks to find the bug.

By the end of the decade, all the technologies were in place to consolidate complex circuitry on a single device and to miniaturize it as well. The first integrated circuit was invented in 1960, and the unmet need was so great that the industry exploded. As early as 1965, Gordon Moore remarked that the transistor density of integrated circuits was doubling every two years—Moore’s Law.

Charged-coupled device

In 1969, fewer than 10 years after the invention of the integrated circuit, the charge-coupled device (CCD) was invented at Bell Labs. Initially, researchers were trying to create a memory chip (DRAM wouldn’t hit the market until 1970), but its potential to capture images was noted from the very beginning.

The CCD is essentially an array of metal-oxide semiconductor (MOS) capacitors arranged in an array. The MOS capacitors act as photodetectors, converting light into stored electrical energy. In 1971, two years after the CCD was invented, researchers were able to make the theoretical practical by taking images with a CCD device. Kodak made its first digital camera in 1975, and by the 1980s, CCDs were available in consumer-grade video cameras.

Complementary metal-oxide semiconductors

Jumping back to 1963, a process for producing MOS integrated circuits called complementary MOS (CMOS) was invented, the hallmark of which was its low power consumption. By the 1980s and 1990s, CMOS was the industry standard for computer chips, but CCD was still the standard for digital imaging.

The problem with CCD image sensors was their power consumption and their speed. There had been attempts to use CMOS-based image sensors previously, but they suffered from noise problems, producing unusable images. NASA’s Jet Propulsion Laboratory set out to solve these problems to make lighter, more reliable image sensors for their spacecraft. In 1993, they hit upon the solution and rapidly licensed the technology for public use. The combination of low power consumption, small size, and integrated design lent itself perfectly to the needs of mobile phone camera manufacturers.

Picture the future

This is just the foundation for mobile photography that emerged with the new millennium. Since then, the field has grown by leaps and bounds. The earliest cellphone cameras consisted of lenses with four elements and image sensors with fewer than a million photosites. Today’s cameras have seven or more lenses and image sensors made up of over ten million photosites. And we’re not even talking about phones with more than one camera and infrared range detectors. Given how far we’ve come in the past 20 years, who knows where we’ll be in 10 more years?