The Evolution of Chroma Key: Integration with the Virtual Studio

Chroma key is a type of keying technology in which a specific color from one image is made invisible, thus partially revealing a second image that is combined with the first. This technique has been employed since the age of film, and now it is used all the time in the broadcasting and TV commercial industries. The most familiar use of chroma key is for weather reports.

Chroma key for television uses the MK (mixer/keyer) functions of a switcher. There used to be common misconception that the chroma key color is always blue, but from a technical point of view any color can be "keyed out". However, black and brown colors would cause problems since these are also found on hair and skin, so blue or green are used as these are complementary to (furthest away from) human skin tones. It is true that blue was frequently used in the past, but blue shades are common in clothing and also people may have blue eyes, so these days green is also widely used.

With the evolution in digital technologies during the nineties, the virtual studio made its debut. A virtual studio is a system for compositing actual camera footage with computer graphics in real-time, allowing one to experience a three-dimensional virtual world. The system senses which way a camera is facing, how much it is zoomed, etc., and a computer then builds the required image in real time so that if the camera moves around to one side of the actual set, this movement will be duplicated in the virtual set generated by the computer; in other words, in both real and virtual worlds, you will then be looking from the same angle. It is also possible to create a sense of depth, so with a space that is really only the size of a small room, you can make your set look as spacious as a Hollywood studio.

Furthermore, as long as you have the people and some props (tables, chairs, etc.), everything else in the "set" can be built with computer graphics. This offers excellent cost-performance, as it saves on the money and time normally invested in set-building, as well as reducing personnel costs. Actually, when things are busy in the world of TV and TV commercial production, it is not unusual for a studio to be operating beyond capacity, and for staff in the art department to work for three months straight without a break. It is therefore important to be able to use a virtual studio in combination with the real one so as to alleviate some of the pressure.

I visited a few of the booths at Inter BEE 2007 that are exhibiting technologies related to chroma key.

■ FOR-A
This year FOR-A have again set up their virtual studio, an entertaining attraction for booth visitors. The company is exhibiting its digiStorm System, which can more efficiently implement a 3D virtual studio system and RCG system.

For example, with a conventional virtual studio you need to have one computer graphics processor for each camera, but with this system it is possible for one processor to handle several cameras.

Brainstorm Easyset3D is 3D real-time computer graphics software. You can employ it to build a virtual world using drag-and-drop operations to select necessary elements, like floor and walls, from several computer graphics libraries provided with the software, combining them in any way you like.
ManuTrack is a shape recognition sensor that provides real-time information about the position and movements of the fingertips of the person in front of a camera. Recognition is performed as part of the internal chroma key processing of the video signal from the camera, enabling it to detect hand position, angle, and gesture.

By combining these, it is possible to perform stunning illusions in real-time, with a performer seeming to produce objects from his fingertips like a conjuror, or even conjuring up a storm like magic. We can expect to see such effects being used in live TV programs (news, weather reports, variety shows, etc.) in future.

■ NTI
NTI exhibited DRAGON D'Fusion, a virtual studio system it has developed jointly with two other companies, Total Immersion and TV Asahi. For demonstration purposes, it was connected to a NIXUS system, developed by Hokkaido Nikko Telecommunications.

Since TV Asahi was involved with the development of NTI's virtual system, it was first used with the TV program called News Station. In this, there was a certain "wow" factor associated with the J.League rankings table that appeared in the "soccer corner" part of the program featuring Jay Kabira. While it created a very fresh impression, at that time various pieces of hardware were required, including a sensor to detect camera position, and a generator for detecting pan, tilt, and zoom values. DRAGON D'Fusion, in contrast, does not need any extra hardware, and it can even be used with a handheld camera.

Also, until now virtual studios only allowed you to move cameras, but innovative new technology has been developed for DRAGON D'Fusion. This means that if you move something like a flip board that has been composited with computer graphics, the composited image will also move in sync with the board.

For example, let's say that you use computer graphics to place a person into a composited flip board. Now, if you turn the board upside down, the person who seems to be walking and talking on the board will also be turned upside down. Using conventional chroma key technology, the inset image would not move with the flip board. Because this system is able to move both together, virtual worlds look like becoming more interesting in future.

Although this is not about chroma key, in passing I'd like to mention that the new 3D imaging software exhibited by this company is fascinating; it doesn't yet have an official name, but for the moment it's referred to as a catalog application within D'Fusion. With this, once the necessary data is installed on a PC, you can point a connected video camera at a car catalog, for example, and on the computer screen the car will appear in 3D over the catalog. If you turn the catalog, the car will turn with it. And if you put your finger on a body color sample in the catalog, the color of the 3D car on the screen will change accordingly. Similarly, if you point the camera at a picture book, the characters, plants, etc. in the book will appear in 3D and can move about the screen. The company is considering putting this system on a server, and making it available over the Internet for mobile phone applications. Actually, from March next year it is apparently going to be tested in Europe. Personally speaking, I feel that it could be put to good use in the educational field.

■ Bogen Imaging
When using chroma key in a studio, you have to be extremely careful handling the screen. As well as being free from dirt and dust, it must be perfectly flat, since any crease or fold would prevent you from keying out that part. This is why meticulous care has to be taken setting everything up before shooting can start. However, this problem can be solved in a blink with Reflecmedia's LiteRing.

The LiteRing is a circular arrangement of blue or green LEDs mounted around the camera lens. With this innovative system, keying is enabled simply by using the LiteRing to illuminate the supplied Chromatte, a fabric screen featuring millions of glass beads.

Since the light is emitted from around the camera lens, when reflected back from the screen each beam returns to the lens along the same path. As a result, the fabric that normally looks gray will appear as either a blue or green background, without any variation in color across the whole surface. And since exactly the same amount of light shines on every part of the screen seen by the camera, you can key out the background perfectly even, to take an extreme case, if the Chromatte is all rumpled. It's easy to set up, and easy to carry. What's more, the LiteRing can be fitted to any DV, SD or HD camera. It will probably be making itself useful in many different fields from now on.

[TV director]