Okor v. Sega of America, Inc. , 193 F. Supp. 2d 269 ( 2001 )

193 F. Supp. 2d 269 (2001)

Joseph Kwame OKOR, Plaintiff,
v.
SEGA OF AMERICA, INC., Nintendo of America, Inc., Sony Computer Entertainments America, Inc., Defendants.

Nos. 98-12176, 98-12177.

United States District Court, D. Massachusetts.

March 29, 2001.

Joseph Kwame Okor, Cambridge, MA for plaintiff pro se.

Peter B. Ellis, Foley, Hoag & Eliot, Boston, MA, Richard H. Smith, Richard V. Burgujian, Finnegan, Henderson, Farabow, Garett & Dunner, LLP, Washington, DC, for Sega of America, Inc, defendant.

David K. Wanger, Donoghue, Barrett & Singal, P.C., Boston, MA, for Nintendo of America, Inc., defendant.

Gregory S. Gewirtz, Lerner, David, Littenberg, Krumholz & Mentlik, Westfield, NJ, John Egan, Posternak, Blankstein & Lund, Boston, MA, for Sony Computer Entertainments America, Inc., defendant.

John Egan, Posternak, Blankstein & Lund, Boston, MA, for Steven T. Mayer, interested party, Sony Computer Entertainments America, Inc., counter-claimant.

MEMORANDUM AND ORDER

WOODLOCK, District Judge.

In these two pro se cases, plaintiff Joseph Okor alleges infringement of his patents by several game manufacturers.

In Civil Action No. 98-12176, Okor alleges infringement of United States Patent Number 4,127,849 ("'849 patent") by Defendants Sega of America, Inc. ("Sega"), Nintendo America, Inc. ("Nintendo") and Sony Computer Entertainments America, Inc. ("SONY"). The '849 patent is for a "data converter system" that can convert input data into display output data.

In Civil Action No. 98-12177, Okor alleges infringement of United States Patent 4,126,851 ("'851 patent") by SONY.^[1] The '851 patent is for a programmable television game system.

Sega, Nintendo and SONY have moved for summary judgment as to the claims against them. Okor has cross-moved for summary judgment against all three defendants.

I. Background

A. Okor's Patents

1. '849 Patent

The '849 patent relates to a "system for converting coded data into display data." ('849 patent, Title). The system permits the user to store graphical objects, called "symbols" or "sprites," for display on a line-scan video display unit such as a television or computer monitor. The patent states that its object is to "significantly reduce the amount of memory required to store the description of symbols in a data converter," to "provide means whereby the symbols may be readily manipulated in a data converter," to "provide means in a data converter whereby special effects ... may be performed on the symbols on display," and to "provide in a data converter means whereby the resulting picture of the symbol may be distributed to different display devices." ('849 patent, 1:39-53).

Claim 1 of the patent claims

A system for converting coded data signals for presentation as display symbols on a display device comprising

(a) an input computer adapted to store three-dimensional data with respect to a symbol to be displayed and generate signals corresponding to a two-dimensional display representation of, aid symbol,

(b) symbol defining means connected to and providing input data to said input computer,

(c) a display computer connected to and receiving the output of said input computer,

(d) at least one dot generator connected to said display computer and controlled thereby, said dot generator adapted to generate a dot producing signal at the beginning of the horizontal position of each symbol,

(e) at least one symbol generator connected to said generator and to said display computer and adapted to generate symbol producing signals at locations indicated by said dot generator, and,

(f) a video combiner connected to said symbol generator and to said display computer for processing said symbol producing signals and delivering them to a selected display device,

(g) said dot generator including x and y comparators and x and y stacks operatively connected to one another and to said display computer, said x and y comparators receiving, respectively, x and y counts corresponding to the x and y addresses of said symbols.

(1849 patent 14:58-68, 15:1-19).

The '849 patent grew out of a prior patent application, serial number 758,415 ("'415 patent") filed on January 11, 1977.^[2] On November 17, 1977, the Patent Office rejected the '415 patent application because it provided "insufficient disclosure to enable one skilled in the art to make and use three dimensional to two dimensional converter without undue experimentation." In addition, the patent application was said to have provided "insufficient disclosure to enable one skilled in the art to control symbol generator as indicated by dot generator." Furthermore, the Patent Office cited three examples of prior art: Wagner (U.S.Patent No. 3,351,929), Warnock (U.S.Patent No. 3,602,702) and Hayashi (U.S.Patent No. 3,778,810).

On March 20, 1978 Okor filed an amendment to his patent application. The amendment added limitation (g) to claim 1. In supporting his amendment, Okor argued that his invention could be distinguished from the Wagner and Hayashi prior art because in his invention symbol information is stored "in a stack," while in Wagner the symbol information is stored "in the memory at the address corresponding to the location of the block on the display screen." Okor argued that both the Wagner and Hayashi prior art differed from his invention because his invention would store the symbol information in variable sized blocks of memory proportional to the number of symbols to be displayed, not in blocks of memory proportional to the resolution of the screen. Furthermore, Okor distinguished his invention from Wagner's because Wagner's prior art "does not allow for selective positioning of symbols on different display screens."

On June 13, 1978, the Patent Office allowed Okor's patent as amended and it was denominated patent number 4,127,849.

2. '851 Patent

As noted, see note 1 supra, the '851 patent for a programmable television game system was the subject of Civil Action No. 97-12418-DPW, which terminated in summary judgment against Okor on the grounds that none of Sega or Nintendo's products infringed the patent. The '851 patent was issued on November 21, 1978 and expired on November 21, 1995. A more detailed discussion of my resolution of the relevant issues will be found in the Memorandum I issued in that case on June 19, 2000.

B. Allegedly Infringing Products

Okor alleges that six products infringe claim 1 of the '849 patent. All of the allegedly infringing products are game console systems that attach to video monitors to display a video image that changes depending on input from player activated controllers attached to the game console. The video monitors display images using a technique called "raster scanning," in which a narrow illuminating beam scans across the top of the screen from left to right illuminating selected points (pixels) along the line to create one line of image. Immediately upon coming to the end of the topmost line, the beam drops to the second line and again scans across the screen left to right illuminating pixels. This scanning occurs rapidly enough to create the illusion of motion in the image on the screen.

1. Sega's Products

Okor contends that both the Sega Genesis Game Console ("Genesis") and the Sega Saturn Game Console ("Saturn") infringe the '849 patent.^[3]

a. Genesis is a home video game system that has a microprocessor (CPU) and video display processor (VDP) housed in a game console that can be attached to a television for playing. The software for individual games, including all the character image data, is contained in game cartridges. The system allows video image objects called "sprites" to be read from the software, manipulated by player-activated controllers, and displayed on the television screen. Each sprite is displayed as a matrix of dots of various colors and intensities.

Each sprite is displayed in a particular position, orientation, size and color on the screen of the monitor depending on manipulation by the player activated controllers. In order to determine where and in what orientation to display each sprite, the CPU prepares a "sprite attribute table" ("SAT") which is stored in video random access memory ("VRAM"). The SAT includes parameter data which indicates the horizontal and vertical screen coordinates at which the sprite is to appear, along with information about the color, size, and orientation of the sprite. This data is stored in the SAT in order of sprite number, not in order of vertical or horizontal display position.

The game system has the ability to assign "priority" to each sprite, which allows the system to know which sprite to display if two occupy the same display space, thus giving the illusion that one sprite is in front of the other.

The game system displays video images on the screen using the raster scan method. Immediately before the scan begins, during the "horizontal blanking interval," the "line buffer memory," which has a storage location for every pixel in the scan line, is loaded with image data for the next line. During the next scan line, the line buffer memory is read and each pixel is sequentially altered in conformance with the data stored in the line buffer memory.

This type of display memory is called "bit-mapped" because there is a storage location in the memory corresponding to each pixel on the display. It does not compare the horizontal addresses of the symbols to be displayed with the horizontal count output of the beam control counter to produce the display signals. However, this system is a line-buffer system, as opposed to a frame-buffer system, both of which are sometimes called bit-map systems. In a line-buffer system, the information is stored line by line, instead of for the whole screen.

The Genesis game console has only one output port that can be connected to only one video display unit at a time, and thus does not have the capability to send display signals to more than one display device.^[4]

Genesis does not have the capability to store three-dimensional data for graphic objects or to generate two-dimensional depictions of three-dimensional graphic objects for display.

b. Saturn is a game system that is more powerful than Genesis and has a larger memory. Rather than use a linebuffer memory system, Saturn buffers an entire frame (screen) and so it has a frame-buffer, bit-map display memory. Thus, it does not compare the horizontal address of a symbol to be displayed with a horizontal count from a beam control counter to produce the display signals. In addition (and unlike Genesis), because Saturn buffers a full frame it does not compare the vertical address of a symbol to be displayed with a vertical count from a beam control counter to produce the display signals.

Saturn also has only one display output port that can be attached to only one display, and thus does not have the capability to deliver display signals to multiple display devices.

Saturn does have the capability to store digitally three-dimensional representations of symbols and generate new two-dimensional images of those symbols.

2. Nintendo's Products

Okor alleges that three video game systems sold by Nintendo infringe the '849 patent: the Nintendo Entertainment System ("NES"), first sold in the United States in 1985, Gameboy, first sold in the United States in 1988, and the Super Nintendo Entertainment System ("SNES"), first sold in the United States in 1991. NES and SNES are home game console systems that attach to video monitors such as television sets, like Sega's Genesis and Saturn products. Gameboy is a small handheld game system with its own built-in monitor.

In NES, position data for various graphic objects ("sprites") is detected and stored in temporary random access memory ("Temporary RAM"). This information is stored in the random order it is encountered, but it is delivered to the symbol line buffers in order of priority for display of the sprites. The temporary RAM is of a fixed size and does not vary with the quantity of sprites to be displayed.^[5]

SNES and Gameboy do not use temporary RAM to store position data for graphic objects prior to displaying them on the screen. SNES and Gameboy have no separate memory for storing the horizontal positions of symbols to be displayed on a line of the screen.

NES and SNES connect to one display device (a television) and do not have the capability to display on multiple devices.^[6] Gameboy has a built-in display monitor and does not have the capability to connect to alternative or multiple display devices.

None of the Nintendo products can store three-dimensional symbol data without additional hardware or software. However, approximately ten cartridges can be used with SNES and Gameboy in order to perform a type of 3-D to 2-D conversion. In these cartridges, the 3-D objects are built up from 2-D surfaces called "polygons," rather than using a "volumetric" approach in which the 3-D object is broken up into a three-dimensional grid of elemental volumes called "voxels."

3. SONY's Products

The only SONY product that is in question in either action is the SONY Playstation video game system ("Playstation"), a computer controlled game console first sold in the United States in 1995.

Playstation displays successive images on its screen by producing a complete new frame of video data for each frame, storing the entire screen in memory before displaying it. Thus Playstation uses a "frame buffer," "bit map" approach. Each frame buffer must have one or more memory locations that correspond to each pixel of the display screen because the symbols to be displayed are stored in the frame buffer at addresses that correspond to the location of the symbols on the display screen. The amount of memory required to store images in the frame buffer is proportional to the resolution of the display screen, not to the number of symbols being stored.

Playstation connects to at least one player controller and is intended for connection to only one display screen, but has the capability to be connected to more than one display screen. If Playstation is connected to more than one display screen, the identical image will appear on all of them.

Playstation has the capability to store digitally three-dimensional representations of symbols and generate new two-dimensional images of those symbols.

As relevant to the alleged infringement of the '851 patent in No. 98-12177-DPW, the record establishes that SONY has never sold a light pen, light gun or any other light receiving or emitting device for use in connection with the Playstation console. SONY did enter into licensing agreements in 1996 and 1997 permitting several third parties to sell light guns for use with Playstation, but did not do so before October 1996. According to Anne Chen, SONY's Director of Business Development, SONY is not aware that any party sold a light pen, light gun, or any other light receiving or emitting device for use with Playstation prior to November 21, 1995 and no evidence is provided in the record that any party did so.

II. Summary Judgment Standard

Summary judgment is appropriate when "the pleadings, depositions, answers to interrogatories, and admissions on file, together with the affidavits, if any, show that there is no genuine issue of material fact and that the moving party is entitled to a judgment as a matter of law." Fed. R.Civ.P. 56(c). To defeat a motion for summary judgment, the nonmovant must show that there is a genuine issue, "that is, the evidence relevant to the issue, viewed in the light most favorable to the party opposing the motion ... must be sufficiently open-ended to permit a rational factfinder to resolve the issue in favor of either side." National Amusements, Inc. v. Town of Dedham, 43 F.3d 731, 735 (1st Cir.1995).

In this case, summary judgment is appropriate if the construction of the claims establishes that the allegedly infringing products either do—or do not—infringe the patent when viewing the evidence in the light most favorable to the nonmovant. I find that, as a matter of law, a proper construction of the claims resolves the issue of infringement in favor of the defendants, and therefore will grant the defendants' motions for summary judgment. In doing so, I have taken the evidence in the light most favorable to the plaintiff Okor.

III. Infringement Standard

Determining whether a patent is infringed is a two-step process. "The first step is claim construction, which involves ascertaining the scope and meaning of the claims at issue, while the second step involves determining whether the claims as construed read on the accused device." Streamfeeder, L.L.C. v. Sure-Feed Sys., Inc., 175 F.3d 974, 981 (Fed.Cir.1999). Claim construction is a question of law to be determined by the court. Cybor Corporation v. FAS Technologies, Inc., 138 F.3d 1448, 1454 (Fed.Cir.1998).

"To show infringement, the plaintiff must establish that the accused device includes every limitation of the claim or an equivalent of each limitation." Dolly, Inc. v. Spalding & Evenflo Companies, Inc., 16 F.3d 394, 397 (Fed.Cir.1994). The absence of even a single limitation of the claims from the accused device precludes a finding of literal infringement. Kahn v. General Motors Corp., 135 F.3d 1472, 1477 (Fed.Cir.1998).

IV. Construing The '849 Patent

In order to determine whether the Sega, Nintendo, and SONY products infringe claim 1 of the '849 patent, I must first construe the patent. I begin by construing the limitations that the defendants contend cannot be read onto their products. The defendants variously contend that limitations a, d, g, and f cannot be read onto their products because none of their products contain a "dot generator" that includes "x and y comparators and x and y stacks" (d,g), nor do they contain a "video combiner" that can deliver symbol signals to a "selected display device" (f). In addition, Sega and Nintendo contend that at least some of their products do not have the capability to "store three-dimensional data" or "generate" two-dimensional representations of three dimensional symbols (a).

"Proper claim construction entails an analysis of a patent record's intrinsic evidence—the claim language, the written description, and the prosecution history." Hockerson-Halberstadt Inc., v. Avia Group Int'l Inc., 222 F.3d F.3d 951, 955 (Fed.Cir.2000). A court should begin the analysis of a claim with the claim language itself, giving the claim terms "their ordinary and accustomed meaning as understood by one of ordinary skill in the art." Id. However, the court must also "examine a patent's specification and prosecution history to determine whether the patentee has given the term an unconventional meaning." Id. If during prosecution of the claim the inventor makes statements that limit the meaning of a claim term, the inventor is bound to those limitations during infringement litigation. Id., at 956, citing, e.g., CVI/Beta Ventures, Inc. v. Tura LP, 112 F.3d 1146, 1158 (Fed.Cir. 1997).

A. Dot Generator

Limitation (d) of claim 1 describes "at least one dot generator ... adapted to generate a dot producing signal at the beginning of the horizontal position of each symbol." Limitation (g) describes

said dot generator including x and y comparators and x and y stacks operatively connected to one another and to said display computer, said x and y comparators receiving, respectively, x and y counts corresponding to the x and y addresses of said symbols.

(1849 patent, 15:14-19)

During the pendency of this action, the Federal Circuit issued an en banc decision in Festo Corp. v. Shoketsu Kinzoku Kogyo Kabushiki Co., Ltd., 234 F.3d 558 (Fed. Cir.2000). Festo clarified the law of prosecution history estoppel and narrowed the use of the doctrine of equivalents when an inventor has amended a claim or made representations during the prosecution of the patent. Festo held, among other things, that "[w]hen a claim amendment creates prosecution history estoppel with regard to a claim element, there is no range of equivalents available for the amended claim element." Id., at 569. All three defendants submitted supplemental briefs arguing that because Okor amended his patent application by adding limitation (g), he may not argue that their products infringe the patent by having an equivalent to that limitation. Rather, under Festo, the amendment creates prosecution history estoppel precluding any argument about equivalents as to limitation (g) of claim 1.

Okor responded to the defendants' supplemental memoranda by stating that he did not assert the doctrine of equivalents, and therefore Festo is inapplicable. Rather, Okor argues that all the products at issue literally infringe every limitation of claim 1, including limitation (g). Therefore, to resolve the motions for summary judgment in favor of Okor, I must find whether any of the products incorporate limitation (g) (as well as all the other limitations of claim 1). That is, the allegedly infringing products must literally have a "dot-generator including x and y comparators and x and y stacks." To find against Okor, I must find that, even taking the evidence in the light most favorable to Okor, the products at issue do not literally incorporate limitation (g).

In doing so, I construe the terms in their ordinary sense to one skilled in the art, but I must examine the specifications and prosecution history to aid in the interpretation of the claim. A "stack" was well understood by those skilled in the art to mean "a particular type of memory structure that is accessed on an ordered (e.g., last-in-first-out) basis." Nintendo's expert, Mayer, testified that stack memory systems can be differentiated from "other memory organizations that can randomly access any of the data." The Okor invention uses a dot generator which includes an x and y stack and an x and y comparator.

The basic theory of the dot generator is as follows:

At the beginning of each line, the y address of every symbol using the dot generator 18 is compared with the y count 50. If there is a match, the x address and the symbol number of the symbol are loaded into a new file which is called the active file. For every clock pulse, the x address of each symbol in the active file is compared with the x count 48. If there is a match, one line of the matrix defining the symbol is generated. When all the lines of the matrix defining a given symbol have been generated, the symbol is deleted from the active file. The process described above is repeated until all the symbols have been generated.

('849 patent, 6:22-35; emphasis added).

During the patent prosecution, Okor explained how his memory architecture differed from prior art. In distinguishing his invention from the Wagner prior art patent, Okor stated:

Although in both systems the address of the block, the color of the symbol, and the symbol number are stored, the way in which they are stored differ. In Wagner the information is stored in the memory at the address corresponding to the location of the block on the display screen. In the applicant's system this information is stored in a stack. The difference between the two systems is that in Wagner the amount of memory required to store these parameters is proportional to the resolution of the display screen, whereas in applicant's arrangement the amount of memory required is proportional to the number of symbols to be displayed independent of the resolution of the screen.

(Okor's Amendment to patent no. 758,415, at 5. emphasis added.) Okor distinguishes his invention from the Hayashi prior art using similar terms. Thus, Okor himself made clear that the stack architecture of his invention did not include a memory architecture in which the image data is stored at the location corresponding to its location on the display screen rather than in a pre-ordered stack. In addition, to infringe the patent, a product would have to store information in a memory stack that varies in size with the number of objects stored in it, rather than a memory whose capacity is determined by the resolution of the screen.

In his deposition, Okor explicitly distinguished his memory system from bit-map systems by stating that there is no frame buffer in the '849 patent, but rather the sprite is displayed directly onto the screen without the need of the intermediate step of creating the image in the frame buffer before reading the buffer onto the screen. (Okor Dep., at I:225-226).

Using Okor's representations to the Patent Office as a guide, it appears that a memory system that writes information into a temporary RAM utilizing a line buffer system or a frame buffer system and storing information in a location corresponding to its location on the display screen, that is, a bit-map system, does not literally have an x stack and an x comparator.^[7] This interpretation is confirmed by Okor's testimony.

1. Sega Products

Neither Sega Genesis nor Saturn use an x-stack memory configuration and thus do not infringe claim (g) of the '849 patent. Both Sega systems store information in a sprite attribute table which includes horizontal and vertical screen coordinates at which each sprite is to appear. (see supra, I(B)(1)). In Genesis, the information is stored in a line buffer, in which the memory has a location for each pixel in a single line, and the process is repeated for each line. Saturn buffers the entire frame rather than each line. In both systems, this information is stored in the sprite attribute table that is organized in order of sprite number, not in order of vertical or horizontal display position. And in both systems, the memory size is in proportion to the resolution of the screen, not the number of symbols being stored.

Thus, neither Sega Genesis nor Saturn contains an x stack as that term is used in Okor's patent, and therefore neither product infringes the '849 patent.

2. Nintendo Products

Nintendo alleges that none of the Nintendo products at issue infringe the patent because none have a "dot generator" that utilizes an x-stack system for storing and retrieving information and for generating new image output. Nintendo contends that NES utilizes a Temporary Random Access Memory ("Temporary RAM") to temporarily store sprite information before displaying the information on the screen, the role that is filled by the dot generator in Okor's patent. Okor contends that NES's Temporary RAM meets the definition supplied in limitation (g) for the dot generator, and thus constitutes the x-stack memory architecture of the invention.

Nintendo argues that this Temporary RAM memory system cannot be an x-stack because 1) information in the Temporary RAM is not ordered by the position in which the symbols appear on the line as an x-stack is; 2) the Temporary RAM is fixed in size, rather than varying its size depending on the number of symbols to be displayed; 3) finally, because the Temporary RAM is random access memory, it is not a stack, which by definition accesses the units of memory it contains in a specific order, as opposed to randomly.

Okor contends that the Temporary RAM is an x stack, and he supports this contention by quoting a Nintendo document which states that

During the horizontal scanning period in the processing cycle, the vertical position data and vertical counter value (vertical scanning position) for each OBJECT are compared.... If the vertical position data is within the range described above, the data for it's [sic] OBJECT is transmitted TEMPORARY RAM ("IN-RANGE" detection) The detection is done one by one starting at the OAM's lowest address.

Okor does not explain why this description proves that NES uses an × stack memory architecture, but it appears that because the document explains that the detection is performed one by one starting at the lowest address in the OAM (Object Attribute Memory—NES' temporary memory), Okor concludes that such OAM functions as an × stack.

Nintendo contests such a characterization, arguing that only detection is done in order of lowest address, but that information is not stored in the Temporary RAM according to the × address of the objects to be displayed. Rather, "information is stored in the Temporary RAM in the random order it is encountered in the OAM and it is delivered to the symbol line buffers in the order of relative symbol display priority." Thus, Nintendo contends that since data is not stored in Temporary RAM in order of horizontal position, Temporary RAM cannot constitute an × stack.

In addition, Nintendo contends that because Temporary RAM is a fixed size, it cannot qualify as a variable sized memory stack. Because Okor distinguished his invention from prior art by describing the memory stack as varying with the number of objects stored, he is precluded from now arguing that a fixed size memory system can be read on the '849 patent.

SNES and Gameboy, unlike NES, do not utilize a Temporary RAM system to store the × positions of symbols to be displayed on a line of the screen. They have no separate memory that performs this function. Thus, they have no memory system that Okor has identified as an × stack.

3. SONY's Product

Playstation uses a frame-buffered bit-map system (see supra, I(B)(3)), like the Sega Saturn, storing information in the memory at the address corresponding to the location of the block on the display screen. The Playstation memory system uses a quantity of memory related to the resolution of the screen, not the number of symbols to be displayed.

Thus, Playstation, because it uses a frame-buffer, bit-map memory system, does not have a dot generator with x and y comparators and x and y stacks as those terms have meaning in the '849 patent.

B. Video Combiner

1. Construing the Patent

Limitation (f) provides "a video combiner connected to said symbol generator and to said display computer for processing said symbol producing signals and delivering them to a selected display device." ('849 patent, 15:10-13).

The abstract to the patent explains that "[t]he data converter system also includes a video distributor that makes it possible to display the resulting picture of the symbols on a plurality of display devices...." ('849 patent, Abstract). The background of the invention states that "[i]t is a further object of the present invention to provide a data converter means whereby the resulting picture of the symbol may be distributed to different display devices." ('849 patent, 1:49-52). The description of preferred embodiments states "[i]f there is more than one display device, it is necessary to know on which display devices to display each symbol." ('849 patent, 5:52-54).

The plaintiff and defendants dispute the proper interpretation of limitation (f). The defendants contend that the references to choosing between multiple display devices in the specification make it clear that limitation (f) requires that the device be able to select a display device among several options. Okor contends that the permissive language in the specifications ("makes it possible;" "may be distributed;" "if there is more than one") suggests that the actual device need not have the capability of being attached to multiple display devices to infringe the patent. Rather, the patent makes multiple display devices an option by patenting a method for creating, storing, and displaying independent symbols. Okor contends that limitation (f) does not demand that the device actually have the capability to be attached to multiple display devices. It can, he contends, permanently select a single display device.

I reject Okor's suggested understanding of the meaning of limitation (f) because it does not make the best sense of the claim as elaborated by the specification. The limitation demands that the patented device have a "video combiner" that delivers signals to a "selected" display device. If the patent did not demand that the device have the capability to be attached to multiple display devices, the word "selected" would be mere surplusage in the claim. A construction that renders words of the claim mere surplusage is frowned upon in patent law. See Texas Instruments Inc. v. United States Int'l Trade Comm'n, 988 F.2d 1165, 1171 (Fed. Cir.1993).

The specification makes it clear that the patented device is meant to have the capability to display symbols on multiple display devices. Despite the fact that the specification could be read to suggest that the multiple display function is optional, the best reading of those sections is that the device must have the capability of delivering images to multiple displays, but that it need not always do so. This construction is consistent with the text of the claim, which suggests that the capability to select among display units is an integral part of the invention.

Finally, in his amendment, Okor distinguished his invention from prior art by stating, "Wagner's system does not allow for the selective positioning of symbols on different display screens."

Thus, limitation (f) demands that the device have the capability to be attached to multiple display devices, and the capability to project different images on the different displays. A device that does not have the capability to be attached to multiple display devices does not infringe the patent.

2. Allegedly Infringing Products

Neither of the Sega systems have the capability to attach to multiple display devices. Genesis and Saturn, except in an early version of Genesis,^[8] each have only one display output port which can be coupled to only one display unit.

None of the Nintendo products have the capability of being attached to multiple display screens and selecting between them which symbols to project on each. Mayer in his affidavit states, without contradiction by plaintiff, that "[n]one of the accused [Nintendo] systems can select to display generated symbol signals among one or more connected video monitors." NES and SNES, again, except in an earlier version,^[9] have a single port to attach to a single television. Gameboy has a single video monitor built in to the system. It has no capability to attach to another video monitor.

SONY Playstation does have the capability to be attached to more than one display monitor at a time. However, if it is attached to more than one display device, the identical image will be projected on all monitors. Thus, SONY argues that it does not infringe the patent because it does not have the capability to select on which display device it will display various image symbols, it simply delivers a single image to whatever display device is connected to the console.

When read in such a way as to make sense of the specification, as well as the prosecution history and Okor's representations, claim (f) requires a video combiner that can select which display device should display which symbol. A device that can only display a single image to whatever display devices are connected to it does not meet the requirements of limitation (f) and therefore does not infringe the patent. Thus, limitation (f) of claim 1 cannot, for this separate reason, be read onto Playstation or the early versions of Genesis, NES and SNES.

When limitation (f) of claim 1 is read in this way, it becomes apparent that, as a matter of law, none of the products in question in this action infringe the '849 patent, because none except Playstation and the early versions of Genesis, NES and SNES have the capability to be attached to multiple display devices, and consequently do not have even the capability to deliver symbol signals to a selected display device. Moreover, none of the products meet the additional requirement of limitation (f) because none has the capability to project differentiated images to selected display devices if more than one are attached to the console. Thus, summary judgment is appropriate for all three defendants on this ground alone.

C. Storage and Conversion of 3-D Data

1. Construing the Patent

Limitation (a) requires that the invention include "an input computer adapted to store three-dimensional data with respect to a symbol to be displayed and generate signals corresponding to a two-dimensional display representation of said symbol." ('849 patent, 14:61-64.). In his briefs, Okor implies that all that is necessary to meet this limitation is that the device in question assign "priority" to 2-D sprites, allowing the computer to determine which 2-D object should be visible to the viewer if two occupy the same screen position. This priority suggests that one object is "in front" of the other, and thus gives the illusion that the sprites are interacting in a 3-D world. However, as is clear from the specification and prosecution history, limitation (a) demands more.

In addition, Okor implies in his pleadings that a device may infringe the patent if 2-D sprites are created in either of two ways. Either they may be "pre-computed" or they may be "computed." If sprites are "pre-computed," then the 2-D images to be displayed are all programmed into the computer, and depending on what view is desired, that image would be selected and displayed. If sprites are "computed," then the computer stores a 3-D model of the object to be displayed, and depending on what view is desired, the appropriate 2-D image would be generated through a computation process making use of the 3-D model. The implication of Okor's pleading is that either method of sprite creation could be read on the patent. However, it is clear from the patent itself that only the "compute" method is encompassed by limitation (a).

The claim itself demands that the input computer be adapted to "store three-dimensional data" and "generate signals corresponding to a two-dimensional display representation of said symbol." In order to construe the claim properly, I must determine what it means by "three-dimensional data" and what it means to "generate" corresponding two-dimensional display representations.

A current dictionary includes two alternate meanings of "three-dimensional" that could apply to the specification. The first is "of or involving three dimensions." Webster's Third New International Dictionary 2382 (1986). Under that definition "three-dimensional data" would have to include a description of the object such that the computer could understand the object as having height, width and depth. That is, the description would have to include data that would allow the object to be plotted on three axes (x, y and z). Only data that could be said to describe an object in these three dimensions could be considered "three-dimensional data." Thus, a computer that stored symbols that were plotted on only two axes would not qualify, even if it could assign them priority and thus make them appear to exist in a 3-D environment. Furthermore, using this definition of "three-dimensional," a computer that could store multiple images of an object plotted in two dimensions but drawn in perspective to make the object appear to exist in three dimensions could not be said to "store three-dimensional data" because the data it stores only describes the symbols or objects in terms of x and y axes. Under this definition, the fact that two-dimensional objects can be manipulated to give the illusion of a three-dimensional environment (for example, by making them progressively smaller, giving the illusion of movement away from the viewer) does not mean that three-dimensional data is stored with respect to these objects.

The dictionary also includes a second definition of "three-dimensional": "giving the illusion of depth or varying distances— used of a pictorial representation." Under this second definition, a symbol that is described with reference to only two axes may be "three-dimensional" if it is portrayed in perspective, for example, giving the illusion of three dimensions, even though the object is only plotted on the x and y axes.

The specification to the '849 patent makes it clear that, in order to conform to limitation (a), the invention must be able to actually store three-dimensional descriptions of objects in the first sense of describing the object with reference of three axes. The description of preferred embodiments states "Basically, the interest of the invention is in the 3-D description of the symbol." ('849 patent, 3:31-33). More specifically,

A symbol is defined by its 3-D dot matrix. Once the symbol has been defined in 3-D space, to get a particular view of it, it is projected over a plane. This is obtained by drawing lines from the view point to each point of the object, and the points where these lines intersect with the viewing plane, as shown in FIG.2(a), are the corresponding points. A collection of such points on the viewing plane forms a 2-D picture of the particular view of the 3-D object in space.

('849 patent, 3:47-55. emphasis added.)

The specification continues

Since the symbols are defined as a 3-D matrix, it is possible, by mathematical manipulation, to find another matrix that would transform a 3-D matrix into its projection on a viewing plane.

Once a symbol has been defined, it can be moved around and rotated. The size of the symbol changes as it moves toward or away from the viewing point. Thus, for any operation performed on the symbol, a new display ROM of its projection on the viewer must be generated.

It should also be noted that once a symbol has been defined, it is necessary only to specify the (x,y,z) coordinate of its origin and the coordinates of the other points defining the symbol could be generated by simple matrix addition.

('849 patent, 3:67 - 4:13; emphasis added.) Thus, in the context of the patent, "three-dimensional data" about an object is a description of the object in terms of three axes, such that the computer can generate two-dimensional "views" of the object from mathematically manipulating the 3-D description of the object.

If the computer could only store data about the object on two axes, it would not store three-dimensional data, as is required by the patent. Furthermore, if a device functioned in the "precompute manner" —by storing two dimensional representations that portray an object from various angles, and by selecting among the pre-computed two-dimensional images for display—then the device can neither be said to "store three dimensional data" (since it only stores multiple two-dimensional images), nor "generate" two dimensional representations of said symbol (since it only selects among pre-computed images rather than creating new images). In order to infringe the '849 patent, a device must actually store a description of the object with reference to three axes, such that it can generate new two-dimensional representations of the object by performing mathematical manipulations on the three-dimensional representation it has stored.

2. Allegedly Infringing Products

a. Sega Products

Sega Genesis does not infringe claim 1 of the '849 patent because it does not store three-dimensional data and generate two-dimensional representations of symbols as required by limitation (a). According to Seiichi Kajiwara, an employee of Sega, the Sega system receives "2-dimensional depictions of game objects" from the game cartridges. The game "can change the size of the 2-dimensional depictions of the object and it can change its directional orientation (facing up, down, left, right)." But it does not "store 3-dimensional data for the game objects" nor does it "calculate 2-dimensional depictions of game objects showing the objects at viewing angles different from those supplied by the cartridge ROM."^[10]

Sega has made no attempt to argue that Saturn does not incorporate limitation (a) to the degree that it demands that the device have the capability to store three-dimensional symbols and generate two-dimensional display representations of said symbols.

b. Nintendo Products

None of the Nintendo products can store three-dimensional symbol data without additional hardware or software. Thus, except as combined with the additional software in game cartridges, none of the systems infringe the patent.

Nintendo argues that even when used with the game cartridges, the 3-D to 2-D conversion takes place in a manner that cannot be read on limitation (a) of the patent. According to Nintendo, in the cartridges that perform a 3-D to 2-D conversion, the 3-D objects are built up from 2-D surfaces called "polygons." This method is contrasted to the "volumetric" method that Nintendo claims is specified in the Okor patent, in which the 3-D object is broken up into a three-dimensional grid of elemental volumes called "voxels." I do not find the "volumetric" approach to be specified in the patent or in Okor's representations.

c. SONY Products

Playstation does store digital three-dimensional representations of symbols and generate new two-dimensional images of said symbols. SONY has made no attempt to argue that Playstation does not incorporate limitation (a) to the degree that it demands that the device have the capability to store three-dimensional symbols and generate two-dimensional display representations of said symbols.

V. The '851 Patent

A. Construing the Patent

In Okor v. Sega of America, and Nintendo of America, Civil Action No. 97-12418-DPW (June 19, 2000), aff'd 2001 WL 125908 (Fed.Cir.2001), I construed limitation (c) of the '851 patent, which teaches a "plurality of light pens operatively connected to said multiplexer," to demand "two or more manual controls and two or more light pens ... operatively connected to the display unit." Id., at 23. I also applied the doctrine of equivalents and found that game systems "that have less than a plurality of both operatively connected manual controls and light pens are not equivalent to Okor's system." Id., at 23.

Okor has argued in this action that a manual control unit is the equivalent of a light pen and that a "game pad" is the equivalent of a plurality of light pens and manual controls. I continue to adhere to my reasoning in No. 97-12418 and accordingly reject this argument. The '851 patent demands a plurality of manual controls in limitation (b) and a plurality of light pens in limitation (c). Both limitations must be met for a product to infringe the patent. Therefore, light pens cannot be the equivalent of manual controls.

B. Allegedly Infringing Product

Okor alleges that Playstation infringes the '851 patent. However, no evidence has been introduced that any party sold any light pen, light gun or any other light receiving or emitting device prior to the expiration of the patent on November 21, 1995. Because I have rejected Okor's argument that a manual control unit is the equivalent of a light pen, the issue may be decided on summary judgment. I hold as a matter of law that the SONY Playstation does not infringe the '851 patent because it does not have multiple light pens as required by limitation (c), nor does it have a structure that can appropriately be said to be the equivalent of said light pens.

VI. Conclusion

For the reasons set forth more fully above, I ALLOW summary judgment against Okor on all claims, finding that, as a matter of law, none of the properly alleged products infringe the '849 patent and that the SONY Playstation does not infringe the '851 patent.

NOTES

[1] Last year I granted summary judgment to defendants Sega and Nintendo in Okor v. Sega of America, Inc. and Nintendo of America, Inc., Civil Action No. 97-12418-DPW (June 19, 2000), finding as a matter of law none of their products infringed the '851 patent. That judgment was affirmed without opinion by the Federal Circuit after argument of the instant motions before me. Okor v. Sega of America, 4 Fed.Appx. 939 (Fed.Cir.2001). Okor continues to press his '851 infringement claim against SONY in the separate action docketed as Civil Action No. 98-12177.

[2] According to Sega, the '415 application is actually a modification of a previous patent application, serial no. 627,960 filed on November 3, 1975. The '960 application was rejected by the Patent Office on July 28, 1976. Rather than respond to that Office Action, Okor filed the '415 application as a "continuation in part." The '960 application is considered abandoned for failure to respond to the Office Action.

[3] In a recent pleading, Okor made reference to the Sega Master System for the first time. As I indicated at the January 10, 2001 hearing in this matter, I will not permit suggestions of infringement by another device to be introduced belatedly in this litigation.

[4] An older version of the Genesis apparently had two alternative ports to allow it to connect to two types of televisions with different video input technology. While no mention of the possibility of simultaneously connecting the game console to two monitors was made in the instructions that accompanied the product, it would be physically possible to do so. Sega represents that it is not sure if the older Genesis version would be capable of simultaneously driving two monitors, but that if it were, only one display would be projected on both displays. As will appear, "capability" to drive more than one display is a necessary, but not a sufficient, condition under limitation (f) on which to ground a claim of infringement here.

[5] Nintendo describes the NES technology as "on-the-fly" as opposed to "bit-map" in Alpex Computer Corp. v. Nintendo Co. Ltd., 34 U.S.P.Q.2d 1167, 1177-78 (S.D.N.Y.1994). On the fly, apparently, means that the computer can store "sprites" as complete units, not needing to have room in the temporary memory for commands for each pixel. Rather, the sprite is stored as a "stamp" that appears on the screen as a single unit. In Alpex, the jury found that on-the-fly and bitmap technology were equivalent. In the instant case, in relation to NES, I must determine whether NES on-the-fly technology can literally be read on Okor's patent, which also employs "on-the-fly" technology, as opposed to bit-map technology. Alpex's expert, Mr. Milner, testified that NES technology could not really be considered "on-the-fly" because the images "are temporarily stored by the PPU before being displayed on the television screen." Id., at 1179. But Nintendo denied that NES functioned in that way. Milner also testified that NES's display generation system, which consists of a "counter operating a RAM" "goes through one cell after another just like the conveyor belt rolls them off." Id. From the description in Alpex, NES's technology may sound similar to Okor's patent. However, as I discuss infra, I find them distinguishable.

[6] As with the Sega Genesis, see note 4 supra, some older versions of NES and SNES have two ports. In Nintendo's instruction manual these ports were clearly identified as producing alternative, not simultaneous, means of connecting to a single monitor. However, Nintendo believes that it would be possible to use both ports to project a single display simultaneously on two monitors.

[7] Okor argues that "(1) if you write data into a random access memory (RAM), (2) order the data in the RAM, (3) read back the data sequentially, the setup will act as a stack." Thus, according to Okor, temporary RAM serves the same function as a stack memory architecture. However, the question is not whether the memory system acts like a stack, but whether it is a stack.

[8] See note 4 supra.

[9] See note 6 supra.

[10] For the sake of clarity, I note the distinction between rotating a 2-D object and a 3-D object. A 2-D object may only be rotated on the plane created by the x and y axes, whereas a 3-D object may rotate on a variety of planes defined with reference to the x, y, and z axes. Thus a 3-D object can be rotated away from the viewer, while a 2-D object can not. The Sega Genesis system only allows rotation in a parallel plane, and thus does not perform a 3-D to 2-D transformation.