Bear with me; it’ll take a bit of windup before I get to the point.
From time to time I’ve written that 4K video is a low bar. It seems that some don’t know what I mean by that, or what it implies.
The simplest form of 4K video is 3840 × 2160 pixels. That’s 8,294,400 pixels. We passed 8mp with interchangeable lens cameras almost two decades ago. Thus it’s not the number of pixels that’s an issue. Low bar.
So why didn’t we get 4K video in our, say, Nikon D3 back in 2007? That had to do with two things, compression efficiency and how fast those pixels could be moved to and fro within the camera. Bandwidth capacity has been growing just behind Moore’s Law for semiconductors for the same reasons, while compression efficiency has been addressed two ways: better compression, and faster dedicated logic doing the compression. At this point in time, dealing with all the bits that need to be dealt with and stored for 4K is well within even the lowest line of dedicated cameras.
The current “bar” for 4K video tends to be raw video. Specifically 12-bit raw. That’s a lot of bits, particularly if you want a frame rate higher than 30 fps. We need to multiply the pixel count by bits and then again by frame rate to understand how much data is going through the camera into storage. While compressed 4K is an easily achievable bar these days, raw 4K is about where the bar currently lies between consumer cameras (e.g. Z6 II) and prosumer or pro cameras (e.g. Z8). That varies a bit between makers, but there’s one thing I know: tech is relentless, and that bar will move down.
On the flip side is output. Chances are that the best video you’ve seen is some form of 4K compressed. Even in your local movie theater. There are exceptions, for instance the Sphere in Las Vegas, for example. Another might be the Apple Vision Pro, which is so immersive it really wants about 100mp in the capture. But these exceptions are rare and still uncommon.
Which brings me to this: virtually every one of our current smartphones and dedicated cameras that do 4K could be used to create a feature film and you probably wouldn’t notice that it was. In fact, that’s happening over and over again these days. The good news about that is that the huge cost bar to entry that was present back when I was studying filmmaking in the 70’s is no longer there. The bar to making a feature film today is “having something to say.” And I suppose: “taking the time to say it."
Those of you who don’t create video are probably saying, “so what, none of this applies to me.”
Actually, it does.
Early digital video (1970’s) begat white balance. More recently, 8K video begat 40mp+ image sensors. The ongoing quest for higher bitrates and frames per second has begat faster sensor offload, which in turn helped autofocus and continuous frame rates for still photography. It turns out that all the progressions made in digital video over the years have had still photography implications.
Yet most of you seem to think that image sensors are just still photography components. Nope, they’re simply a sampling tool. How you use that sampling tool is where the magic starts to happen.
I remember the early tinkering that led to the Connectix QuickCam (1994). Basically, our whole philosophy was centered around what we could do if we took this nascent TI CMOS sampling device and get its data into the Macintosh CPU where we could do something with it. The answer at the time was video, stills, survellience, video conferencing…the list went on and on. The little video camera in your laptop you use for Zoom meetings is a great grandchild of the QuickCam, but the whole of what it’s doing is basically the same as when we first got it working: move the image data over to the computer and then do something with it.
That’s what’s happening in your still camera: data comes off the image sensor and then some processor does things with it. Both still and video use moves forward in lockstep as both the image sensor and the image processor get better. If you’ve got a Nikon Z8 or Z9, for example, one reason why you can have a near infinite still image buffer—High efficiency raw—is because the still side is using a compression algorithm originally intended for the video side.
Likewise, doing subject detection and tracking for focus requires a constant stream of data (dare I say video? ;~). The more data, the more you can do and the more accurate the results.
If you told the camera makers not to do anything on the video side, the still capabilities wouldn’t iterate as well and as fast as they have. For the camera maker, being able to do both stills and video is a win win. As I sit and write this, I can think of several things that haven’t been done on the stills side that the video—or rather, continuous image data collection—would allow. I don’t want still camera designers thinking about just getting one sample!
In some ways I wish I were back in the Connectix labs tinkering, as I’ve got many more ideas now than I had then. That’s exactly what I want the camera makers doing, too. Don’t get blinders on and only look at that image sensor as a “one time” capture, but to explore what happens when you have a rapid and continuous stream of data and a great deal of computational horsepower to try doing things with it.
Update: a number of you seem to think I was being prescient in posting this article when the Nikon acquiring RED announcement came just a couple of days later. Well, yes and no. I did not know that Nikon would be announcing the RED acquisition this week. But I did know that Nikon has been actively sniffing around high-end video world recently, and that Nikon sees big cross benefits between the two.
