Methods are considered for increasing the richness of the computer generated image so as to make it more appealing visually and thus more competitive with conventional animation methods. A travelling matte is derived from the foreground image which is then used to obscure the hidden portions of the background image in the optical printer. This process can be extended to provide additional levels of visibility. Colour is applied separately to each level during this process. A conversion scheme to a television raster format is also described which provides multilevel cel visibility directly using digital processing.
Considerable effort over the past years has been directed towards developing techniques for generating animated film by computer. Significant progress has been made in achieving an animation capability of sufficient flexibility that would permit a freedom of expression comparable to that in conventional methods. However, the quality of the resulting computer generated image has been to a large extent neglected. Greater consideration must be given towards increasing the richness of the computer generated image so as to make it more appealing visually and thus more competitive with conventional animation methods.
Our objective has been to develop a system of sufficient capability to permit the production of complex and visually rich animation sequences; and at the same time a system sufficiently simple to use, so as not to deter the professional animator. The system is essentially picture driven, as opposed to being language driven. The artist generates and modifies the pictures by interacting with the system through the controls available to him.
The primary technique for creating animation sequences is the method of key-frame animation [1] [2]. The animator prepares selected images at key intervals in the sequence, and on playback the in-between frames are computed by interpolating between the key frames. A key frame in a sequence may consist of up to sixteen independent cels, with an interpolation law assigned separately to each cel. Implicit in this interpolation process is the restriction imposed on the artist in the way the picture must be drawn. In order to achieve controlled animation, he must control the number of lines in the picture element, as well as the order in which these lines are drawn. This restriction, of course, has no effect on his sketch studies which he prepares on paper from his storyboard or script outline, as he is accustomed in conventional hand-drawn animation. Instead, it is imposed only at the stage of tracing the sketch-study on the graphic tablet into the computer. Furthermore, the composite drawing is usually broken down into cels that are traced on the graphic tablet separately, thus reducing the complexity of the image to which these restrictions apply.
The most direct form of the computer generated image that can be animated is a line drawing. Figure 1 illustrates the sort of detail one may readily incorporate into an animation sequence. The line drawings in the interpolated images as well as in the terminal key frames clearly reflect the style of the artist and to a much lesser extent are characterized as being computer generated. However, the overall visual quality of the image in this form falls short of the needs for commercial film production. Perhaps the most obvious step towards enriching the image is by generating areas. Without departing from the basic technique of animating line drawings, a conversion to a raster scan format may be applied on a frame by frame basis to the computed image sequence. Thus any area that is completely bounded by an outline is filled in by adjacent horizontal lines when it is recorded on film.
Figure 2 shows strips of 16 mm footage from a title sequence of a children's TV program Le Sac a Malice that was made using this technique. Four parallel sequences, one for each colour, were generated in outline form. Using the area scan conversion program, the four sequences were recorded by backing up the camera and filming through a different coloured filter on each pass. Because most of the areas were created by a pair of parallel outlines, the equivalence of coloured lines of variable width was achieved. In those instances where different coloured areas overlap, the colours simply mix since multiple exposure is completely additive.
Now suppose areas are to be filled in for an image sequence such as in Figure 1. When the key drawings are being prepared, each cel that will be filled in is drawn as an enclosed outline plus internal detail. Application of the programmed scanning process converts the outline to a filled- in area. Two difficulties immediately arise when recorded on film. First, wherever two cels overlap, the images are essentially transparent to each other. Although this effect was considered acceptable and possibly even desirable for a sequence such as in Figure 2, it is generally distracting and undesirable in character animation. Even more important the black lines surrounding and within the area can only be produced through a subtractive process.
One method that overcomes these problems was employed in the production of the NFB film "La Faim" by Peter Foldes. The composite line drawing sequence shown in the left film strip in Figure 3 consists of three parts - a foreground image, a middleground image and a background image. Each of these are filmed as separate film strips. In addition, the outlines of the foreground and middleground sequences are scan converted and filmed as areas or travelling mattes. The foreground and middleground line sequences are those portions of the composite image that correspond to the travelling mattes shown in the two center strips. Using an optical printer, the background lines are printed with colour through the two mattes which obscure the hidden portion of the background. The middleground lines are then printed (in another colour) through a reversed middleground matte plus the foreground matte. Finally, the foreground lines are printed through its reversed matte. The composite final image is shown at the right. With only three cel levels as illustrated, transparency may not be removed for all possible conditions, but the result is generally satisfactory.
It should be recognized that this method of removing transparency differs in principle from the application of a hidden-line removal algorithm. The latter is a procedure for computing the visible portions of an image using a data structure that defines the spatial relationships of the surfaces which make up the image. The concept of cel level visibility, on the other hand, mimics the conscious decision by the animator to sandwich the cel layers in a particular order as he would in conventional cel animation.
An additional source of image degradation is encountered when lines cross over or overlap. This causes blooming or bleeding at the intersections. One way to reduce this is to scan convert the line drawings as well as the areas thus preventing double exposure of any point on the film. In order to remove bleeding entirely wherever lines tend to cluster, the spot of the recording CRT must have very sharp skirts with no halo.
With only a slight extension, the raster format can be made compatible with television video format to drive a TV monitor. The scanned data is stored as a bit-pattern array where each bit indicates whether the corresponding point on the raster display is intensified or not. Using a 262 line picture with identical odd and even fields and a horizontal resolution of 256 lines, fifty seconds of animation can be stored in digital form on the moving-head disc. During playback, each image is displayed as five successive fields from a core memory refresh buffer. The buffer, in turn, is updated with a new picture during the vertical retrace interval of the fifth field, so that an effective frame rate of 12 frames per second is obtained.
Even though the resolution of the image is low, this format offers several advantages. The size of the bitpattern array describing one frame is fixed. Thus complex sequences, which are created from many separate parallel sequences, can be merged on disc into one composite sequence, and viewed at the full playback speed.
Figure 4 shows several composite frames from the film "La Faim", photographed from a TV monitor. Successive merging of line drawings produces a transparent image as in (a). If the scanned areas are used to erase the hidden detail of the background image before merging the next level, transparency is removed as in (b). Either polarity may be selected for the bit pattern during merging to produce any combination of white areas on black or black areas on white as shown in (c) and (d). Other examples are shown in (e) and (f). Three levels of visibility are illustrated in this figure, though the process may be extended to any number of levels.
Although the quality of the resulting video image is insufficient as a practical final output, the quantization steps in the data are visually appealing. This digital characteristic was exploited in the cover design for this publication. An offset was introduced between the scanned area and the line drawing to enhance the effect in this figure.
This facility is being extensively used for both the television and film media by animators from NFB and CBC. The purpose of these cooperative programs has been to establish the merits of computer animation techniques and to evaluate our system. Only through the production of the last NFB film "La Faim" have we been able to acquire a thorough appreciation of the real capabilities and limitations of the present system. In parallel with this exploitation phase, work will continue on the improvement and the extension of the system.
Without departing from our basic capabilities, the merging process using bit-pattern arrays may be extended to include the full resolution that is used for filming. Colour information may be imbedded within the bit patterns, or it may be generated from colour separations. The size of the bit arrays increases rapidly, however, and the amount of processing that is involved appears excessive for our installation compared to the optical printing technique.
An extension to provide full video bandwidth directly would appear to be a more attractive alternative. Allowing only 2 bits for colour intensity at each raster point, the bit array for merging one field will increase by 4 times. The size and speed of both the moving head digital disc and the present buffer memory become insufficient for such performance, so a video disc will have to be used. Further development of the system is expected to continue in this general direction.
1. N. Burtnyk and M. Wein Computer Generated Key-Frame Animation Jour. of SMPTE, Vol.80, Mar 1971.
12. N. Burtnyk and M. Wein A Computer Animation System for the Animator Proc. of 10th meeting of UAIDE, Oct 1971.