Upload
others
View
1
Download
0
Embed Size (px)
Citation preview
UNITED STATES PATENT AND TRADEMARK OFFICE
BEFORE THE PATENT TRIAL AND APPEAL BOARD
MICROSOFT CORPORATION
Petitioner
v.
BRADIUM TECHNOLOGIES LLC
Patent Owner
CASE: To Be Assigned
Patent No. 7,139,794 B2
PETITION FOR INTER PARTES REVIEW OF U.S. PATENT NO. 7,139,794 B2
TABLE OF CONTENTS Page
-i-
EXHIBIT LIST ........................................................................................................ ii
I. INTRODUCTION .......................................................................................... 1
II. MANDATORY NOTICES UNDER 37 C.F.R. § 42.8(B) ............................ 2
III. REQUIREMENTS FOR INTER PARTES REVIEW ................................... 3
A. GROUND FOR STANDING ............................................................... 3
B. IDENTIFICATION OF CHALLENGE ............................................... 3
IV. OVERVIEW OF THE 794 PATENT ............................................................. 5
A. PRIORITY DATE OF THE 794 PATENT ......................................... 5
B. SUMMARY OF THE 794 PATENT ................................................... 6
C. SUMMARY OF PROSECUTION FILE HISTORY......................... 10
D. LEVEL OF ORDINARY SKILL IN THE ART ............................... 11
E. PROPOSED CLAIM CONSTRUCTION .......................................... 12
V. THERE IS A REASONABLE LIKELIHOOD THAT AT LEAST ONE CLAIM OF THE 794 PATENT IS UNPATENTABLE .................... 13
A. IDENTIFICATION OF THE REFERENCES AS PRIOR ART ....... 13
B. SUMMARY OF INVALIDITY POSITIONS ................................... 14
VI. DETAILED EXPLANATION OF GROUNDS FOR UNPATENTABILITY OF CLAIMS 1 AND 2 OF THE 794 PATENT ....................................................................................................... 15
A. GROUND 1: CLAIMS 1 AND 2 ARE UNPATENTABLE UNDER 35 U.S.C. § 103(a) AS BEING OBVIOUS OVER POTMESIL, HORNBACKER, AND LINDSTROM ........................ 15
B. GROUND 2: CLAIM 1 IS UNPATENTABLE UNDER 35 U.S.C. § 103(a) AS BEING OBVIOUS OVER RUTLEDGE IN VIEW OF LIGTENBERG AND COOPER ....................................... 41
C. GROUND 3: CLAIM 2 IS UNPATENTABLE UNDER 35 U.S.C. § 103(a) AS BEING OBVIOUS OVER RUTLEDGE IN VIEW OF LIGTENBERG, COOPER AND MIGDAL..................... 53
VII. CONCLUSION ............................................................................................. 60
Petition for Inter Partes Review of U.S. Patent 7,139,794 B2
-ii-
EXHIBIT LIST
Ex. 1001 U.S. Patent No. 7,139,794 B2 to Levanon et al. (“the 794 Patent”)
Ex. 1002 Declaration of Judea d’Arnaud, attaching the article Maps Alive: Viewing Geospatial Information on the WWW, Michael Potmesil, Computer Networks and ISDN Systems Vol. 29, issues 8-13, pp. 1327-1342 (“Potmesil”) as Exhibit A.
Ex. 1003 PCT Publication No. WO 1999/041675 by Cecil V. Hornbacker, III (“Hornbacker”)
Ex. 1004 U.S. Pat. No. 5,682,441 to Adrianus Ligtenberg et al. (“Ligtenberg”)
Ex. 1005 U.S. Pat. No. 6,650,998 to Charles Wayne Rutledge et al. (“Rutledge”)
Ex. 1006 U.S. Pat. No. 6,118,456 to David G. Cooper (“Cooper”)
Ex. 1007 U.S. Pat. No. 5,760,783 to Migdal et al. (“Migdal”)
Ex. 1008 Declaration of Prof. William R. Michalson
Ex. 1009 Six Provisional Applications from which the 794 Patent claims priorities.
Ex. 1010 EP1070290 to Cecil V. Hornbacker, III from a European national application based on PCT Publication No. WO 1999/041675 (Ex. 1003)
Ex. 1011 An Integrated Global GIS and Visual Simulation System by P. Lindstrom et al., Tech. Rep. GIT-GVU-97-07, March 1997 (“Lindstrom”)
Ex. 1012 Declaration of Dr. Peter Lindstrom (including Exhibits A, B and C) regarding the publication of the 1997 article entitled “An Integrated Global GIS and Visual Simulation System” which is Ex. 1011 (“Lindstrom”)
Petition for Inter Partes Review of U.S. Patent 7,139,794 B2
-iii-
Ex. 1013 Declaration of Mr. Charles Randall Carpenter (including Exhibits A, B, C and D) regarding the publication of 1997 article entitled “An Integrated Global GIS and Visual Simulation System” which is Ex. 1011 (“Lindstrom”)
Petition for Inter Partes Review of U.S. Pat. No. 7,139,794 B2
-1-
I. INTRODUCTION
Pursuant to 35 U.S.C. § 311 and 37 C.F.R. § 42.100, Microsoft Corporation
(“Microsoft” or “Petitioner”) petitions for inter partes review (“IPR”) of claims 1
and 2 of U.S. Pat. No. 7,139,794 B2 (“the 794 Patent,” Ex. 1001), currently owned
by Bradium Technologies LLC (“Bradium” or “Patent Owner”). This Petition is a 5
remedial measure for correcting the issuance of invalid claims in the original
examination and is necessitated by Patent Owner’s improper enforcement of the
invalid claims.
Specifically, this Petition shows there is a reasonable likelihood that
Petitioner will prevail with respect to at least one of the claims 1 and 2 challenged 10
under 35 U.S.C. § 314(a). As demonstrated by a preponderance of the evidence in
this Petition in compliance with 35 U.S.C. § 316(e), claims 1 and 2 are
unpatentable under pre-AIA 35 U.S.C. §103 based on specific grounds listed
below.
Grounds References Challenged Claims
Pre-AIA 35 U.S.C. §103(a)
Potmesil, Lindstrom, and Hornbacker Claims 1 and 2
Pre-AIA 35 U.S.C. §103(a)
Rutledge, Ligtenberg and Cooper Claim 1
Pre-AIA 35 U.S.C. §103(a)
Rutledge, Ligtenberg, Cooper and Migdal
Claim 2
Petitioner Microsoft respectfully requests the Office to institute a trial for 15
IPR and to cancel claims 1 and 2.
Petition for Inter Partes Review of U.S. Pat. No. 7,139,794 B2
-2-
II. MANDATORY NOTICES UNDER 37 C.F.R. § 42.8(B)
REAL PARTY IN INTEREST: Pursuant to 35 U.S.C. §312(a)(2) and 37
C.F.R. §42.8(b)(1), Petitioner Microsoft constitutes all real parties in interest for
this IPR proceeding.
RELATED MATTERS: Patent Owner Bradium is asserting the 794 Patent 5
and two other related patents, U.S. Patent Nos. 7,908,343 and 8,924,506, against
Petitioner in an on-going patent infringement lawsuit in Bradium Techs. LLC v.
Microsoft Corp., 1:15-cv-00031-RGA, filed in the U.S. District Court for the
District of Delaware on Jan. 9, 2015. In addition, Petitioner is pursuing IPR
petitions on 343 and 506 Patents asserted in the above litigation. 10
NOTICE OF COUNSEL AND SERVICE INFORMATION: Pursuant to 37
C.F.R. §§ 42.8(b)(3), 42.8(b)(4) and 42.10(a), Petitioner appoints Bing Ai (Reg.
No. 43,312) as lead counsel, Matthew Bernstein (pro hac vice), Vinay Sathe
(Reg. No. 55,595) and Patrick McKeever (Reg. No. 66,019) as back-up counsel.
Petitioner also requests authorization to file a motion for Mr. Bernstein to 15
appear pro hac vice, as Mr. Bernstein is an experienced patent litigation attorney, is
lead counsel for Petitioner in the district court litigation, and has an established
familiarity with the subject matter at issue in this proceeding. Petitioner intends to
file such a motion once authorization is granted. The above attorneys are all at the
mailing address of Perkins Coie LLP, 11988 El Camino Real, Suite 350, San 20
Petition for Inter Partes Review of U.S. Pat. No. 7,139,794 B2
-3-
Diego, CA 92130, contact numbers of 858-720-5700 (phone) and 858-720-5799
(fax), and the following email for service and all communications:
Pursuant to 37 C.F.R. § 42.10(b), a Power of Attorney executed by Microsoft for
appointing the above designated counsel is concurrently filed. 5
III. REQUIREMENTS FOR INTER PARTES REVIEW
This Petition complies with all statutory requirements and requirements
under 37 C.F.R. §§ 42.104, 42.105 and 42.15 and thus should be accorded a filing
date as the date of filing of this Petition pursuant to 37 C.F.R. § 42.106.
A. GROUND FOR STANDING 10
Pursuant to § 42.104(a), Petitioner hereby certifies that the 794 Patent is
available for IPR and that the Petitioner is not barred or estopped from requesting
IPR challenging claims of the 794 Patent on the grounds identified herein.
Specifically, Petitioner has the standing, or meets all requirements, to file this
Petition under 35 U.S.C. §§ 315(a)(1), 315(b), 315(e)(1) and 325(e)(1); and 37 15
C.F.R. §§ 42.73(d)(1), 42.101 and 42.102.
B. IDENTIFICATION OF CHALLENGE
Pursuant to 37 C.F.R. §§ 42.104(b) and 42.22, the precise relief requested is
that the Board institute an IPR trial on Claims 1 and 2 and cancel the claims
because they are invalid on the grounds and evidence presented in this Petition. 20
Claims Challenged: Claims 1 and 2 are challenged in this Petition.
Petition for Inter Partes Review of U.S. Pat. No. 7,139,794 B2
-4-
The Prior Art: The prior art references relied upon are 7 references listed in
the Exhibit List: (1) Potmesil (Ex. A of Ex. 1002); (2) Hornbacker (Ex. 1003); (3)
Ligtenberg (Ex. 1004); (4) Rutledge (Ex. 1005); (5) Cooper (Ex. 1006); (6) Migdal
(Ex 1007); and (7) Lindstrom (Ex. 1011 and associated Ex. 1012 and Ex. 1013).
Supporting Evidence Relied Upon For The Challenge: The Declaration of 5
Prof. William R. Michalson (Ex. 1008) supporting the invalidity grounds in this
Petition and other supporting evidence in the Exhibit List are filed herewith.
Statutory Ground(s) Of Challenge And Legal Principles: Pursuant to 37
C.F.R. § 42.104 (b)(2), the review of patentability of claims 1 and 2 requested in
this Petition is governed by pre-AIA 35 U.S.C. §§ 102 and 103 that were in effect 10
before March 16, 2013. Further, statutory provisions of 35 U.S.C. §§ 311 to 319
and 325 that took effect on September 16, 2012 govern this IPR.
Claim Construction: The 794 Patent has not expired. In IPR, the Office shall
give a claim in an unexpired patent “its broadest reasonable construction in light of
the specification of the patent in which it appears” to one of ordinary skill in the art. 15
37 C.F.R. § 42.100(b). SAP v. Versata, CBM2012-00001, Final Written Decision
at 10 (PTAB June 11, 2013), Paper No. 70 and In re Cuozzo Speed Technologies,
LLC, No. 2014-1301 at 11-19 (Fed. Cir. Feb. 4, 2015).
How Claims Are Unpatentable Under Statutory Grounds: Pursuant to 37
C.F.R. § 42.104 (b)(4), Section VI provides an explanation of how claims 1 and 2 20
Petition for Inter Partes Review of U.S. Pat. No. 7,139,794 B2
-5-
are unpatentable under pre-AIA 35 U.S.C. § 103, including the identification of
where each element of the claim is found in the cited prior art.
IV. OVERVIEW OF THE 794 PATENT
The 794 Patent is entitled “System and methods for network image delivery
with dynamic viewing frustum optimized for limited bandwidth communication 5
channels” and was granted on Nov. 21, 2006 from non-provisional App. No.
10/035,981 filed on Dec. 24, 2001. Per USPTO record, in 2009, the 794 Patent was
originally assigned by the inventors to 3DVU, Inc., which subsequently assigned
the patent to Inovo Ltd. On Jun. 17, 2013, Inovo assigned the 794 Patent to
Bradium. The App. No. 10/035,981 has no direct child applications. There has 10
been no post-issuance proceeding on the 794 Patent.
A. PRIORITY DATE OF THE 794 PATENT
The 794 Patent claims priority to six provisional applications: App. Nos.
60/258,488, 60/258,489, 60/258,465, 60/258,468, 60/258,466, and 60/258,467, all
filed on Dec. 27, 2000. No other priority claims were made. Based on the USPTO 15
record, the earliest priority date of the 794 Patent is no earlier than Dec. 27, 2000.
During original examination of the 794 Patent, the applicant submitted two
declarations of inventors in an attempt to swear behind prior art cited by the
Examiner, claiming an alleged earlier pre-filing invention date of October 1999.
For purpose of this IPR proceeding, this Petition cites and relies on references 20
Petition for Inter Partes Review of U.S. Pat. No. 7,139,794 B2
-6-
dated before October 1999. As such, it is unnecessary to examine, in this
proceeding, whether the evidence filed with the two declarations of the inventors in
the original examination is sufficient for establishing an alleged earlier invention
date before the earliest priority date of the 794 Patent.
B. SUMMARY OF THE 794 PATENT 5
The “Background of the Invention” of the 794 Patent describes a “well
recognized problem” of how to reduce the latency for transmitting full resolution
images over the Internet on an “as needed” basis, particularly for “complex images”
such as “geographic, topographic, and other highly detailed maps.” Ex. 1001 at
1:32-47. The 794 Patent admits that solutions already in existence included 10
“transmitting the image in highly compressed formats that support progressive
resolution build-up of the image within the current client field of view.” Id. at
1:48-58. The 794 Patent contends, however, that such “conventional” solutions,
like the ones described in U.S. Pat. Nos. 4,698,689 (Tzou) and 6,182,114 (Yap),
usually “presume that client systems have an excess of computing performance, 15
memory and storage” and are “generally unworkable for smaller, often dedicated
or embedded” clients. Id. at 1:48-2:55. According to the 794 Patent, the
conventional solutions do not work well under “limited network bandwidth”
situations. Id. at 3:4-29.
To address these perceived issues in the existing art, the 794 Patent purports 20
Petition for Inter Partes Review of U.S. Pat. No. 7,139,794 B2
-7-
to disclose a system capable of “optimally presenting image data on client systems
with potentially limited processing performance, resources, and communications
bandwidth.” Id. at 3:38-42.
Specifically, the 794 Patent
describes an image distribution 5
system having a network image
server and a client system, where a
client can input navigational command to adjust a 3D viewing frustum for the
image displayed on the client system. Id. at 5:23-53. High-resolution source image
data is pre-processed by the image server into a series K1-N of derivative images of 10
progressively lower image resolution. Id. at 5:54-6:6, Fig. 2. The source image is
also subdivided into a regular array of 64 by 64 pixel resolution image parcels
(a.k.a. image tiles), and each image parcel may be compressed to fit into a single
TCP/IP packet for faster transmission. Id. at 6:6-22; 7:30-49.
The client system 15
in the 794 Patent has a
“parcel request”
subsystem to request
image parcels from the
server, a “control block” 20
Petition for Inter Partes Review of U.S. Pat. No. 7,139,794 B2
-8-
that directs the transfer of received image parcels and overlay data to a local parcel
data store. Id. at 6:42-62. The control block decompresses the image parcels and
directs a “rendering engine” to render them. Id. at 6:63-65; Fig. 3.
When the viewing point is changed in response to a user navigation
command, the control block “determines the ordered priority of image parcels to be 5
requested from the server . . . to support the progressive rendering of the displayed
image.” Id. at 7:19-22. Image parcel requests are then placed in a request queue,
and are to be issued by the parcel request subsystem according to each request’s
assigned request priority. Id. at 7:22-24; 8:24-36. Although various factors may
affect the priority assigned to a parcel request, e.g., the “resolution of the client 10
display” (8:54-9:4) or whether the image parcel is “outside of the viewing frustum”
(9:26-29), generally speaking, “image parcels with lower resolution levels will
accumulate greater priority values,” so “a complete image of at least low resolution
will be available for rendering” in a fast manner (10:11-19). In addition, the control
parameter for calculating the priority can be set in a way that gives “higher priority 15
for parcels covering areas near the focal point of the viewer” to make sure that
image parcels are requested “based on the relative contribution of the image parcel
data to the total display quality of the image.” Id. at 10:20-38.
In the 794 Patent, after the needed image parcels are requested and received,
an algorithm is used to select the image parcels for rendering and display. Id. at 20
Petition for Inter Partes Review of U.S. Pat. No. 7,139,794 B2
-9-
8:37-42. Overlay data may also be added to the display if its image coordinates
matches the current image parcel location. Id. at 8:46-51. The 794 Patent discloses
that two-dimensional image parcels are displayed in a three-dimensional space
using projection transform. Id. at 5:44-53; 7:9-18; 8:37-41; 10:20-26; 10:63-67.
The 794 Patent states that its disclosed technology can achieve faster image 5
transfer by (1) dividing the source image into parcels/tiles, (id. at 6:1-16), (2)
processing the parcels/tiles into a series of progressively lower resolution
parcels/tiles, (id.) and (3) requesting and transmitting the parcels/tiles needed for a
particular viewpoint in a priority order, generally lower-resolution tiles first. Id.
3:38-4:42. 10
Claims 1 and 2 are the only claims in the 794 Patent. As shown by this
Petition, the 794 Patent is merely repetitive of the long history of prior art
publications on relevant technical features that the Patent Owner attempts to claim
as its own years later. See, e.g., Michalson’s Declaration in Ex. 1008, ¶¶ 33-77 in
“VI. TECHNOLOGY BACKGROUND OF THE 794 PATENT.” As shown in 15
Section VI, all of the features and the combinations in claims 1 and 2 were known
or predictable and/or obvious combinations of the prior art features, and were
published prior to the earliest priority date of the 794 Patent. Also see id., ¶¶ 95-
277. Besides the invalidity grounds in Section VI, Prof. Michalson opines that
claims 1 and 2 are obvious for additional grounds. Id., ¶¶ 278-385. Claims 1 and 2 20
Petition for Inter Partes Review of U.S. Pat. No. 7,139,794 B2
-10-
reflect the Patent Owner’s belated effort to “re-patent” subject matter that was
already in the public domain.
C. SUMMARY OF PROSECUTION FILE HISTORY
The 794 Patent was granted from U.S. non-provisional App. No. 10/035,981.
In the examination of App. No. 10/035,981, the Examiner initially rejected all 5
pending claims on Sept. 21, 2005 under § 102(e) as anticipated by U.S. Pat. No.
6,671,424 (“Skoll”). In response, Applicants attempted to swear behind the Skoll
reference, claiming that “Skoll’s application for his US patent 6,671,424 was filed
25 July, but the Inventors herein developed their invention in November 1999 and
had a working model in December 1999 and January 2000.” Applicants’ Response 10
at 2 (Jan. 10, 2006). Applicants submitted two declarations of inventors, dated Dec.
2005 and Jun. 2006, respectively, to support an contended earlier pre-filing
invention date of October 1999. Declaration of Inventor at 2 (“The herein
invention was first defined in October 1999, we had a working model in December
1999 and we can establish that we had the first working product on about 24 15
January 2000.”) (Dec. 27, 2005); see also Declaration of Inventor at 1 (Jun. 13,
2006). The Examiner rejected the first declaration as insufficient (Final Rejection
dated Mar. 14, 2006) and accepted the second declaration. The Examiner issued
claims 1 and 16 of the original application with Examiner’s Amendments as the
patent claims 1 and 2. Examiner’s Amendment & Reasons for Allowance (Aug. 21, 20
Petition for Inter Partes Review of U.S. Pat. No. 7,139,794 B2
-11-
2006).
As shown by the cited prior art references in this Petition, the Examiner
relied on an incomplete record of relevant prior art during the examination and thus
did not know that the subject matter of the issued claims 1 and 2 in the 794 Patent
was well known and published by others before its filing date and thus was not 5
patentable.
In addition, the file history contains no discussions on the prior art U.S.
Patent No. 6,182,114 (Yap) that is listed on the face of the 794 Patent and is
mentioned in the “Background of the Invention” of the 794 Patent. This lack of
discussion of the Yap reference with respect to claims 1 and 2 is an oversight by 10
the Examiner because the disclosure of the Yap reference is highly relevant and is
material to the patentability of claims 1 and 2. Notably, Prof. Michalson opines
that claims 1 and 2 are obvious over the Yap reference in view of additional prior
art. Ex. 1008, ¶¶ 340-384.
This Petition is a remedial measure for correcting the unfortunate outcome 15
of issuing the invalid claims 1 and 2 in the 794 Patent due to the lack of a fuller
and more complete record of relevant prior art and due to lack of adequate
consideration of relevant teaching in the cited prior art in the original examination.
D. LEVEL OF ORDINARY SKILL IN THE ART
A person of ordinary skill in the art (POSITA) for the 794 Patent should 20
Petition for Inter Partes Review of U.S. Pat. No. 7,139,794 B2
-12-
have a Master of Science or equivalent degree in electrical engineering or
computer science, or alternatively a Bachelor of Science or equivalent degree in
electrical engineering or computer science, with at least 5 years of experience in a
technical field related to geographic information system (“GIS”) or the
transmission of image data over a computer network. Ex. 1008 at ¶¶ 28-32. 5
E. PROPOSED CLAIM CONSTRUCTION
Petitioner proposes construction of certain claim terms below pursuant to the
broadest reasonable interpretation (BRI) standard for inter partes review. The
proposed BRI claim constructions are offered only to comply with 37 C.F.R.
§§ 42.100(b) and 42.104(b)(3) and for the sole purpose of this Petition, and thus do 10
not necessarily reflect appropriate claim constructions to be used in litigation and
other proceedings where a different claim construction standard applies.
The proposed BRI claim construction for the terms in claims 1 and 2 is plain
and ordinary meaning of each term in light of the 794 Patent specification.
For example, “First fixed size” and ”Second fixed size” in Claim 1 are given 15
their plain and ordinary meaning in light of the specification, e.g., Ex. 1001 6:6-22
and 7:28-46, which teach that image data parcels are delivered as 2 Kbyte
compressed data packets and are rendered as 8 Kbyte image parcels. Ex. 1008, ¶
93.For example, the term “parcel rendering subsystem” in Claim 1 is given its
plain and ordinary meaning in light of the specification, e.g., Ex. 1001 at 6:44-47, 20
Petition for Inter Partes Review of U.S. Pat. No. 7,139,794 B2
-13-
6:65-67, 7:4-8, and Fig. 3. Ex. 1008, ¶ 93. For example, the term of “parcel request
subsystem” is given its plain and ordinary meaning in light of the specification,
e.g., Ex. 1001 at 6:44-51 and Fig. 3. Ex. 1008, ¶ 93. As another example, the term
“image parcel” is given its plain and ordinary meaning in light of the Ex. 1001 at
6:1-16 and FIG. 2. As yet another example, the term “image data parcel” in claims 5
1 and 2 is given its plain and ordinary meaning. The specification of the 794 Patent
uses the terms “image data parcel” and “image parcel data” interchangeably, e.g.,
Ex. 1001 at 3:38-4:42 (“Summary of the Invention”). Alternatively, to the extent
that the term “image data parcel” needs construction under BRI, Petitioner
construes “image data parcel” to mean “data representing an image parcel” in light 10
of the 794 Patent specification.
V. THERE IS A REASONABLE LIKELIHOOD THAT AT LEAST ONE CLAIM OF THE 794 PATENT IS UNPATENTABLE
Claims 1 and 2 are unpatentable under pre-AIA 35 U.S.C. § 103(a) for
merely reciting known, predictable and/or obvious combinations of the cited prior 15
art references.
A. IDENTIFICATION OF THE REFERENCES AS PRIOR ART
All seven (7) prior art references cited in this Petition were not on record
during the original examination of the 794 patent.
Potmesil was published in September 1997 and is prior art under at least § 20
102(b). Hornbacker was filed as a PCT application No. PCT/US1998/003017 on
Petition for Inter Partes Review of U.S. Pat. No. 7,139,794 B2
-14-
Feb. 12, 1998 and published on Aug. 19, 1999. Hornbacker is prior art under at
least § 102(b). An European patent No. EP1070290 (Ex. 1010) was granted from
an European application No. 98906484.5 as a national phase application of the
PCT application No. PCT/US1998/003017. Lindstrom was published in March
1997 and is prior art under at least § 102(b). See also Ex. 1012 and Ex. 1013 5
regarding Lindstrom. Ligtenberg was filed on Nov. 8, 1995 and issued on Oct. 28,
1997. Ligtenberg is prior art under at least § 102(b). Rutledge was filed on July 28,
1997, which is a continuation-in-part of application no. 08/824,106, filed on Mar.
25, 1997, which is a continuation-in-part of application no. 08/613,307, filed on
Mar. 11, 1996. Rutledge was patented on Nov. 18, 2003. Rutledge is prior art 10
under at least § 102(e). Cooper was filed on April 2, 1998 and patented on Migdal
was filed on November 6, 1995 and issued on June 2, 1998. Migdal is prior art
under at least § 102(b).
B. SUMMARY OF INVALIDITY POSITIONS
The cited prior art references in this Petition, not previously before the 15
Office, disclose systems and methods of subdividing large images into a regular
array of tiles, compressing these tiles into a series of reduced-resolution tiles,
requesting image tiles of various resolutions in a priority order based on the user’s
viewpoint, and rendering the received image tiles for display on a client device.
This Petition uses two primary references, (1) Potmesil and (2) Rutledge, to form 20
Petition for Inter Partes Review of U.S. Pat. No. 7,139,794 B2
-15-
independent and distinctive invalidity positions against claims 1 and 2, and further
combines these references with (3) Hornbacker, (4) Lindstrom, (5) Ligtenberg, (6)
Cooper and (7) Migdal to reject claims 1 and 2. These references are selected
because of their distinctive teachings that cover different technical aspects of the
794 Patent and provide the Office and the public with a fuller view of the prior art 5
landscape prior to the filing of the 794 Patent that was not discussed or duly
considered during the original examination.
Specifically, claim 1 is rendered obvious by (1) Potmesil in view of
Hornbacker and further in view of Lindstrom, and (2) Rutledge in view of
Ligtenberg and Cooper, and claim 2 is rendered obvious by (1) Potmesil in view of 10
Hornbacker and further in view of Lindstrom and (2) Rutledge in view of
Ligtenberg, Cooper and Migdal.
VI. DETAILED EXPLANATION OF GROUNDS FOR UNPATENTABILITY OF CLAIMS 1 AND 2 OF THE 794 PATENT
A. GROUND 1: CLAIMS 1 AND 2 ARE UNPATENTABLE 15 UNDER 35 U.S.C. § 103(a) AS BEING OBVIOUS OVER POTMESIL, HORNBACKER, AND LINDSTROM
Potmesil, Hornbacker, and Lindstrom disclose similar technologies for
retrieving images from networked servers using Internet web browsers and HTTP
protocol and provide solutions to similar technical issues. 20
Potmesil teaches an online system that includes map servers and software
operating on a client computer, and that allows users to view geographic
Petition for Inter Partes Review of U.S. Pat. No. 7,139,794 B2
-16-
information over the worldwide web (WWW) using 2D or 3D map browsers and
Hypertext Transfer Protocol (HTTP) protocol. Ex. 1002 at Abstract. The Potmesil
system includes a “tile server” which stores images such as aerial images and
elevation data in a power-of-two pyramid to allow fast access and scroll and zoom
operations. Id. at Fig. 1, 1329-30. Potmesil teaches that the use of prefiltered 5
power-of-two images for texture mapping was well-known in the art, including in
the OpenGL standard used for rendering in the 3D browser. Id. at 1334, 1340. In
the OpenGL standard, such tiles are referred to as “mip-maps,” the same term used
for image tiles in the provisional applications from which the 794 Patent claims
priority. Ex. 1002 at 1329; Ex. 1009 (Provisional Application No.60/258,465) at 10
7:12-9:4. The 2D and 3D geographical map browsers implement a tile caching
process which calculates the tiles needed to render the current view and tiles likely
to be needed in the future, requests those tiles from the server, and caches those
tiles. For example, in a “flight simulator” mode, the 3D browser requests and
caches tiles from the flight path ahead of the user’s simulated viewpoint. Ex. 1002 15
at 1332-33, Fig. 2. Image tiles may be compressed using a variety of formats such
as JPEG or GIF. Id. at 1334-35. Potmesil teaches that the geographical system
outlined in the paper may be used in a variety of applications such as traditional
computers, notebook computers (NCs), Interactive TVs (ITV’s), cellular phones,
and heads-up displays on car windshields. Ex. 1002 at 1328. 20
Petition for Inter Partes Review of U.S. Pat. No. 7,139,794 B2
-17-
In addressing similar retrieving images over the Internet web browsers and
HTTP protocol as in Potmesil, Hornbacker teaches a method and a system for
displaying portions of very large images (such as digital documents) retrieved over
a network from a server. The Hornbacker system includes a web server networked
to client workstations, which use a web browser on the workstation to request 5
image views by means of Uniform Resource Locator (URL) code using the HTTP
protocol. Ex. 1003 at 5:3-8, 5:16-25. The large images to be transferred are divided
into 128X128 pixel view tiles, which are further organized into a hierarchy of tiles
at differing resolutions spaced by factors of two. The image tiles may be
compressed using GIF compression with a typical compression ratio of 4:1. 10
In the same technology field as Potmesil and Hornbacker, Lindstrom teaches
an online client/server system for viewing large-scale geographic data, e.g. terrain
elevation and imagery data, using a 3D perspective view with multiple windows.
Ex. 1004, Abstract, §§ 1, 3, 4.1, 4.2.6, Fig. 1. Lindstrom uses a pyramidal quadtree
structure to organize multi-resolution terrain and image data in a hierarchical 15
manner, processes requests for image tiles using a prioritized queue and an image
cache, and utilizes level-of-detail (LOD) management that limits texel resolution to
a defined threshold. Id., Abstract, §§2, 3, 4, 4.1, 4.2.1, 4.2.2, 4.2.3. Ex. 1008, ¶¶
100.1, 100.2.
A person of ordinary skill in the art (POSITA) would be motivated to 20
Petition for Inter Partes Review of U.S. Pat. No. 7,139,794 B2
-18-
combine teachings in Potmesil, Hornbacker, and Lindstrom because each reference
addresses the common technical issues in visualizing large amounts of data
obtained over a data network as such the Web via HTTP protocol and in using a
client viewing device with much smaller memory than the database which stores
the imagery data. Ex. 1008, ¶ 104. In this regard, Potmesil, Hornbacker, and 5
Lindstrom address similar or the same technical problems in rendering the images
on the client device from image data received over a data network (e.g. optimizing
bandwidth, prioritizing use of bandwidth, determining which portions of a larger
set of image data to request, etc.). Id. Potmesil and Lindstrom specifically disclose
technology for 2D and 3D visualization of terrain and map data. 10
A POSITA would recognize that Lindstrom teaches a substantially similar
online system to Potmesil with similar goals, and that the prioritization queueing
and level-of-detail management of Lindstrom would provide similar advantages to
speeding up processing in the system of Potmesil, particularly in view of the
related teaching in Potmesil that the tiles are stored and accessed in a power-of-two 15
pyramid and that the caching algorithm sorts requested tiles based on their
proximity to the user. Ex. 1002, 1329-30, 1332-33, Figs. 1-2, Ex. 1008, ¶¶ 100.1,
100.2. To a POSITA, the teachings of Hornbacker are readily applicable to online
mapping references because online maps represent a scenario in which a much
larger amount of geographically organized imagery must be stored on a server than 20
Petition for Inter Partes Review of U.S. Pat. No. 7,139,794 B2
-19-
can be stored at one time on a client. The European counterpart of Hornbacker,
EP1070290, specifically recognizes the relevance of teachings relating to mapping
to the disclosure of Hornbacker by citing and explaining several online mapping
references, including Potmesil, in the description of the prior art. Ex. 1010 at
[0006], [0007]. 5
Accordingly, for at least the above reasons, a POSITA would be motivated
to consider the teachings of the three references in designing a mapping application
for viewing map data over a limited bandwidth communications channel. Ex. 1008,
¶¶ 104-107.
In addition, the analyses and discussions of Potmesil, Hornbacker, and 10
Lindstrom in connection with claim limitations of claims 1 and 2 below provide
additional reasons or motivations that would cause a POSITA to combine Potmesil,
Hornbacker, and Lindstrom in the manner as suggested in this Petition.
As demonstrated below in detail, the combination of Potmesil, Hornbacker,
and Lindstrom collectively teaches or suggests all the limitations of claims 1 and 2 15
and renders each of claims 1 and 2 as a whole obvious and unpatentable. Ex. 1008,
¶¶ 95-172.
1. Claim 1 is Rendered Obvious by Potmesil, Hornbacker, and Lindstrom
Element 1.Preamble A client system for dynamic visualization of image 20
data provided through a network communications channel, said client system
Petition for Inter Partes Review of U.S. Pat. No. 7,139,794 B2
-20-
comprising:
Potmesil and Lindstrom each teach client systems for dynamic visualization
of image data (through a terrain rendering program) which request image data in
the form of image tiles from a network server. Ex. 1002 at Abstract; Ex. 1011 at
Abstract, Fig. 1. Hornbacker teaches a client system for dynamic visualization of 5
image data (through an online image viewing program) which requests image data
from a network server containing image tiles. Ex. 1003 at Abstract.
As discussed above, a POSITA would recognize the applicability of the
teachings of Hornbacker to a map application, as shown by the description of
several map display-related references in the corresponding EU patent for 10
Hornbacker.
Element 1.A a parcel request subsystem, including a parcel request queue,
operative to request discrete image data parcels in a priority order;
Potmesil teaches software as part of the client image visualization
application that produces a list of new tiles to be requested from the server. Ex. 15
1002 at Abstract, 1328, 1329-30 (§§ 2, 2.1), 1332-33 (§§ 3, 3.1), Fig. 2. Potmesil
teaches that cache allocation (i.e. priority of requests for tiles to use the limited
space of the cache) is based on parameters including x, y, z, level-of-detail, and
time. Ex. 1002 at 1332. The images are prioritized by proximity to the user
viewpoint; for example, in the “flight simulator” scenario, the browser requests 20
Petition for Inter Partes Review of U.S. Pat. No. 7,139,794 B2
-21-
tiles in a “widening wedge” in front of the direction of flight; in other words, it first
requests those tiles in the current view or very likely to be needed soon as the
highest priority, then requests other tiles that may be needed in the future. Id. at
1332-33. See also id. at 1327, 1334-35. It would further be obvious to a POSITA
that the request for tiles using the map applications would preferably request tiles 5
nearest the user viewpoint first, before requesting more distant tiles, in order to
effectively manage limited cache sizes. Ex. 1008, ¶¶ 112-113.
Lindstrom provides further detailed teachings that the client module makes
data requests to the server using a priority queue which changes priority
dynamically according to importance determined by the level of detail manager. 10
Ex. 1011, §§ 3, 4.2.1, Fig. 1.
Hornbacker teaches that tiles within a large database may be located by
requests to a server which use URLs to identify the specific tile by incorporating
identifying characteristics such as resolution, location, view angle, etc. See, e.g. Ex.
1003 at 3:10-27, 5:16-25, 6:13-19, 8:30-9:28. Hornbacker teaches that tiles may be 15
requested in a priority order in order to achieve progressive resolution
enhancement by providing lower resolution tiles first. See, e.g. Id. at Abstract,
12:24-13:10. Hornbacker also teaches that tiles that may be required by the next
view request may be “pre-computed” based on the anticipated view. Id. at 7:26-8:6,
10:24-28. 20
Petition for Inter Partes Review of U.S. Pat. No. 7,139,794 B2
-22-
A POSITA would readily recognize that these disclosures in Hornbacker
collectively teach or suggest requesting tiles in a priority order- from lower
resolutions to higher resolutions for progressive resolution enhancement and tiles
within the visible display area before “pre-caching” tiles outside the visible display
area. Ex. 1008, ¶¶ 114-115. 5
In the combination of Potmesil, Hornbacker and Lindstrom, all three
references teach methods of requesting image tiles over a network according to a
priority order. Each reference identifies a common problem with the display of
portions of very large images over a network, which is the latency and bandwidth
consumption associated with downloading an entire image, and arrive at similar 10
solutions in the form of requests for specific tiles in a priority order based on how
soon the tiles need to be displayed. The dynamic queueing function taught by
Lindstrom illustrates further details about how a POSITA could implement
prioritized tile requests in the system of Potmesil or Hornbacker.
A POSITA would readily recognize the similarities in the teachings of the 15
two references solving similar problems in closely related fields and would readily
combine such teachings when designing a display system addressing a similar
problem. Ex. 1008, ¶¶ 116-118.
Further, a POSITA would readily recognize that the specific system for
requesting tiles by URL could be advantageously utilized in the tile request process 20
Petition for Inter Partes Review of U.S. Pat. No. 7,139,794 B2
-23-
of Potmesil or Lindstrom, which each include identifying tiles likely to be needed
in the near future based on their geographic coordinates. Ex. 1008, ¶ 117.
Element 1.B to store received image data parcels in a parcel data store,
Potmesil teaches that retrieved tiles may be stored in a cache on the geographic
browser on the client both for immediate display of tiles and so that tiles covering 5
adjacent areas likely to be used may be retrieved quickly if they are needed. Ex.
1002 at 1328, 1332-33, 1334. Similarly, Hornbacker and Lindstrom teach that
retrieved tiles may be stored in a local cache. Ex. 1003 at 6:1-19, 8:1-6, 8:16-23,
10:13-28, 11:29-12:9, 12:17-23, 13:19-23; Ex. 1011 at Fig. 1, § 4.2.2.
In the combination of Potmesil, Hornbacker and Lindstrom, each reference 10
teaches that a local cache on a client display device is advantageous to enable rapid
display of pre-cached tiles and minimize delays associated with downloading. A
POSITA would readily recognize that these references teach similar solutions to
similar problems. Ex. 1008, ¶¶ 119-120
Element 1.C said parcel request subsystem being responsive to an image 15
parcel request of assigned priority to place said image parcel request in said parcel
request queue ordered in correspondence with said assigned priority;
Potmesil teaches 2D and 3D browsers which request tiles to display the
current view and cache tiles that may potentially be needed in subsequent viewing
(for example, tiles in the projected flight path in the 3D flight simulator mode). Ex. 20
Petition for Inter Partes Review of U.S. Pat. No. 7,139,794 B2
-24-
1002 at Abstract, 1332-33. It would be obvious to a POSITA that the tile caching
system described in Potmesil requests tiles in an ordered sequence based on
priority considerations: first to retrieve and cache those tiles needed immediately,
then those that are most likely to be needed immediately followed by those that
may be needed soon (e.g. the “widening wedge” in flight simulator mode). Ex. 5
1002 at Abstract, 1328, 1332-33, Fig. 2. For example, the tile caching process may
be based on parameters including location, level-of-detail, and time. Id. at 1332.
Lindstrom provides further detailed teachings that the client module makes data
requests to the server using a priority queue which changes priority dynamically
according to importance determined by the level of detail (LOD) manager. Ex. 10
1011, §§ 3, 4.2.1, Fig. 1. The shared queue responds to requests from rendering
processes for each view. Id. § 4.2.1, Fig. 1. Hornbacker teaches that tiles may be
requested according to priority to provide progressive resolution display of images
and to pre-cache tiles that may be needed. 5:16-25, 6:13-19, 7:26-8:6, 8:30-9:28,
10:24-28, 12:24-13:10. 15
In the combination of Potmesil, Hornbacker and Lindstrom, each reference
teaches the use of pre-caching tiles in order to avoid delays associated with
downloading new tiles. The references therefore address similar problems with
similar solutions. Accordingly, a POSITA would be motivated to consider the
combined teachings of these references in regard to a similar problem. Ex. 1008, ¶¶ 20
Petition for Inter Partes Review of U.S. Pat. No. 7,139,794 B2
-25-
121-124.
Element 1.D an parcel rendering subsystem coupled to said parcel data store
to selectively retrieve and render received image data parcels to a display memory,
Potmesil teaches that retrieved data tiles, which may include data such as
elevation profiles and RGB or monochrome images, are compiled and rendered by 5
a 2D or 3D browser which includes a tile caching process and a tile compositing
process. Ex. 1002 at Abstract, 1328, 1332, 1333-35, 1340. It would further be
obvious to a POSITA that the system described in Potmesil would require a frame
buffer (display memory) in order to render images for display. Ex. 1008, ¶¶ 125-
127. Lindstrom provides further details explaining that rendering is performed by 10
rendering modules and threads for each view, which selectively retrieve and render
tiles from shared and private caches. Ex. 1011 at §§ 4, 4.2.1, 4.2.5, Fig. 1. Ex. 1008,
¶ 125.
Element 1.E said parcel rendering system providing said parcel request
subsystem with said image parcel request of said assigned priority; 15
Potmesil teaches that the tile caching process (which requests tiles from the
server and caches them) “receives information about the current view from the
compositing process” (which performs rendering). Ex. 1002 at 1332-33. Lindstrom
provides a similar, related teaching that the client view modules that generate the
view provide data requests to the terrain server via a shared priority queue. Ex. 20
Petition for Inter Partes Review of U.S. Pat. No. 7,139,794 B2
-26-
1011, § 4.2.1, Fig. 1. Ex. 1008, ¶ 128.
Element 1.F wherein said parcel rendering subsystem determines said
assigned priority based on a determined optimal image resolution level;
Potmesil describes the OpenGL standard’s use of “prefiltered power-of-two
images,” aka “mip-maps,” which are used to display textures at an appropriate 5
level of detail based on the virtual distance between the viewpoint and the rendered
object. Ex. 1002, Fig. 1, 1329, 1332, 1334-35, 1340. Potmesil teaches retrieving
and caching image data to provide a user with different views or focal points
desired by the user by operating the caching algorithm to the user's current position,
velocity, and acceleration to estimate where the user is moving and allocates new 10
tiles there. Id. at 1332. Potmesil additionally teaches that the parameters used in the
cache allocation process include level-of-detail. Id.
Lindstrom further teaches that the client rendering modules, including the
LOD manager, provide data requests via the shared priority queue for data of a
desired type and resolution. Ex. 1011 at § 4.2.1, Fig. 1. The LOD manager 15
compares the level of detail (both for terrain geometry and texels) to a threshold
such as a screen space threshold. Id., § 4.2.3.
It would have been obvious to a POSITA based on well-known principles of
computer graphics that the cache allocation process described in Potmesil, based
on the reference to the OpenGL standard and the use of a level-of-detail parameter, 20
Petition for Inter Partes Review of U.S. Pat. No. 7,139,794 B2
-27-
would preferably determine which tiles to request and cache based in part on the
highest image resolution level that can be displayed. Ex. 1008, ¶¶ 129-131.
Lindstrom provides further detailed teachings on how a POSITA would implement
this feature based on the comparison of the LOD to a threshold.
Hornbacker teaches that the resolution of tiles requested is determined based 5
on the size of the view area and the scale at which the user is viewing the image.
Ex. 1003 at 7:4-25, 11:19-28, 13:4-10, 14:2-6.
A POSITA would appreciate the teachings that a tile caching system that
requests tiles based on factors including x, y, z and LOD in the 2D or 3D display of
Potmesil or Lindstrom or at a preferred resolution for a given zoom level and scale 10
as in Hornbacker would preferably request tiles at a resolution that the display was
capable of displaying but that would still produce a satisfying image. Ex. 1008, ¶
132.
Element 1.G wherein said display memory is coupled to an image display
of predetermined resolution; 15
Potmesil teaches a 2D or 3D browser for displaying a view on a client
device. Ex. 1002 at 1332-33, Fig. 2, 1340-41, Fig. 8. Lindstrom teaches that the
client system performs rendering of tiles from private and shared caches on one or
more defined view windows, which a POSITA would recognize would naturally
have a defined resolution. Ex. 1011, Fig. 1. Hornbacker teaches that its system 20
Petition for Inter Partes Review of U.S. Pat. No. 7,139,794 B2
-28-
may be preferably optimized for viewing tiles at a particular resolution, e.g. 200
pixels-per-inch (7:4-25) and that the window may have a typical view size of 896
by 512 pixels. Ex. 1003 at 14:2-6, 11:19-28 and 13:4-10. Additionally, it would be
obvious to a POSITA that a digital display would necessarily have a predetermined
resolution and that the rendering of images on that display would be limited by the 5
resolution of the screen or the defined area of the screen, i.e., the predetermined
resolution of the image display. Ex. 1008, ¶¶ 133-134.
Element 1.H wherein said determined optimal image resolution level is
based on said predetermined resolution;
Each reference teaches that tiles are displayed at an optimum resolution. In 10
Potmesil, tiles are allocated to the cache by parameters, including level-of-detail,
which may be used to request those tiles most likely to be needed. Ex. 1002 at
1332; see also Fig. 1, 1328, 1329-30, 1340-41. The optimal resolution in
Hornbacker is chosen based on the display resolution of the selected view. Ex.
1003 at 7:4-25, 14:2-6; see also 11:19-28, 13:4-10. Lindstrom further teaches that 15
the LOD manager compares terrain geometry and imagery resolution to a
resolution threshold based on screen resolution to optimize LOD. Ex. 1011, § 4.2.1,
Fig. 1.
A POSITA would readily recognize, in light of well-known principles of
computer graphics, including the principles used in the OpenGL standard 20
Petition for Inter Partes Review of U.S. Pat. No. 7,139,794 B2
-29-
referenced by Potmesil and Lindstrom, that an optimal resolution for displaying
tiles is a resolution where the number of pixels per square inch as the tile is
rendered does not exceed the number of pixels per square inch that the display can
show, because otherwise download bandwidth is being wasted. Ex. 1008, ¶¶ 135-
36. 5
Element 1.I wherein said assigned priority further reflects the proximity of
the image parcel referenced by said image parcel request to a predetermined focal
point;
Potmesil teaches that the caching algorithm requests and caches tiles based
on factors including the user’s current position, velocity, and viewpoint (e.g. 10
different views in a 3D browser). Ex. 1002 at 1332. Lindstrom teaches a system
that includes LOD management based on user viewpoint, including viewpoint-to-
texel distance, as well as distance-based clipping of requested tiles. Ex. 1011 at § 1,
4.2.1, 4.2.6. The viewpoint may be generated based on a six degree freedom of
navigation interface. Id. at 4.2.6. 15
A POSITA would readily recognize in light of well-known principles of
computer graphics that tiles in a power-of-two pyramid used for rendering (such as
in the OpenGL standard referenced by Potmesil and Lindstrom), are preferably
filtered based on the proximity of the object being rendered to the user’s view
point (the focal point of the scene), which in turn can be used to determine the 20
Petition for Inter Partes Review of U.S. Pat. No. 7,139,794 B2
-30-
highest resolution that can be rendered. Ex. 1008, ¶¶ 137-38.
Element 1.J wherein said discrete image data parcels are of a first fixed size
as received by said parcel request subsystem,
Potmesil teaches that tiles may be transmitted and received in a compressed
form such as JPEG or GIF. Ex. 1002, Fig. 1, p. 1329-30, 1334-35. Hornbacker 5
teaches that the view tiles preferably use GIF image files with a 4:1 compression
ratio, Ex. 1003 at 6:20-7:3, and that such compression reduces the transfer time
and demand on the network, id. at 14:2-16.
Potmesil and Hornbacker are both directed to related problems involving
viewing portions of very large image data sets over a network using, e.g., HTTP 10
protocol, and reach similar solutions involving the use of image tiles at a hierarchy
of resolutions to convey image data in discrete parcels. Both Potmesil and
Hornbacker teach that conserving bandwidth to transmit a sufficient amount of
data was a known technical issue for online image viewing systems and that image
compression such as GIF or JPEG compression is a known solution to this problem. 15
Hornbacker also teaches fixed data compression ratios, which a POSITA would
recognize as advantageous so that the tiles can be uniformly the same size both on
the image and as transferred, and so that single tiles can be transferred over a
network as a single packet. Ex. 1008, ¶ 140. Hornbacker further teaches that the
use of a standard compression format such as GIF or PNG is desirable due to 20
Petition for Inter Partes Review of U.S. Pat. No. 7,139,794 B2
-31-
browser compatibility.
A person of ordinary skill in the art would readily recognize that these
teachings would speed up download and reduce the bandwidth necessary in a
system like that taught by Potmesil or Lindstrom, which would be particularly
useful in the mobile applications envisioned by Potmesil. Ex. 1008, ¶¶ 139-41. 5
Element 1.K of a second fixed size as rendered by said parcel rendering
subsystem; Potmesil teaches that received tiles may be decompressed to a fixed-
size frame buffer. Ex. 1002 at 1334-35. Hornbacker teaches a view tile format with
a typical 4:1 GIF compression ratio. Ex. 1002 at 6:20-7:3, 14:2-16. It would be
obvious to a POSITA that an image tile that is received compressed would 10
advantageously be decompressed so it can be displayed quickly. Therefore, in light
of the disclosed use of image compression, the combined teaching of Potemesil
and Hornbacker discloses rendering images at fixed sizes different from image
sizes received. Ex. 1008, ¶¶ 142-45.
Element 1.L wherein said discrete image data parcels each includes a fixed-15
size array of pixel data. Potmesil and Hornbacker both teach image data tiles
storing data in fixed-size power-of-two arrays. Ex. 1002, Fig. 1, p. 1329-30. Ex.
1003 at 6:13-7:25, 8:7-15, 14:2-16. Ex. 1008, ¶ 146.
2. Claim 2 is Rendered Obvious by Potmesil, Hornbacker, and Lindstrom 20
Element 2.Preamble A method of supporting dynamic visualization of
Petition for Inter Partes Review of U.S. Pat. No. 7,139,794 B2
-32-
image data transferred through a communications channel, said method comprising
the steps of: the preamble of Claim 2 is substantially identical to the preamble of
Claim 1, therefore the same teachings apply.
Element 2.A determining, in response to user navigational commands, a
viewpoint orientation with respect to an image displayed within a three-5
dimensional space; Potmesil teaches a three-dimensional terrain visualization
application which renders views of terrain based on user navigational commands:
Ex. 1002 at Abstract, 1328-29, 1332-33, Fig. 2, 1340-41, Fig.8. Lindstrom teaches
a similar system to Potmesil which can visualize data through multiple 3D views
and navigation controlled by a six degree of freedom interface, and Lindstrom 10
provides additional details about how a 3D system of the type taught by Potmesil
could be implemented. Ex. 1011 at § 3, 4, 4.2.1, 4.2.3, 4.2.6. Ex. 1008, ¶ 149.
Element 2.B requesting, in a priority order, image parcels renderable as
corresponding regions of said image,
Potmesil teaches a client image visualization application that produces a list 15
of new tiles to be requested from the server. Ex. 1002 at Abstract, 1328, 1329-30
(§§ 2, 2.1), 1332-33 (§§ 3, 3.1), Fig. 2. Potmesil discloses that cache allocation (i.e.
priority of requests for tiles to use the limited space of the cache) is based on
parameters including x, y, z, level-of-detail, and time. Id. at 1332. The images are
prioritized by proximity to the user viewpoint; for example, in the “flight simulator” 20
Petition for Inter Partes Review of U.S. Pat. No. 7,139,794 B2
-33-
scenario, the browser requests tiles in a “widening wedge” in front of the direction
of flight; in other words, it first requests those tiles in the current view or very
likely to be needed soon as the highest priority, then requests other tiles that may
be needed in the future. Id. at 1332-33. See also id. at 1327, 1334-35. Lindstrom
provides further detailed teachings that the client module makes data requests to 5
the server using a priority queue which changes priority dynamically according to
importance determined by a LOD manager. Ex. 1011, §§ 3, 4.2.1, Fig. 1.
It would be obvious to a POSITA that the request for tiles using the map
applications would preferably request tiles nearest the user viewpoint first, before
requesting more distant tiles, in order to effectively manage limited cache sizes, 10
and that the dynamic queueing function taught by Lindstrom illustrates further
details about how a POSITA could request tiles in a priority order in the system of
Potmesil or Hornbacker. Ex. 1008, ¶¶ 150-51.
Hornbacker discloses the viewing of large images stored at a remote server
by breaking the images down into tiles representing discrete portions of the image 15
and transmitting the tiles in a priority order with lower resolution tiles first to
provide progressive regional resolution enhancement. Ex. 1003 at Abstract, 3:10-
27, 5:16-25, 6:13-19, 7:26-8:6, 8:30-9:28, 10:24-28, 12:24-13:10.
In the combination of Potmesil, Hornbacker and Lindstrom, the three
references teach methods of requesting image tiles over a network according to a 20
Petition for Inter Partes Review of U.S. Pat. No. 7,139,794 B2
-34-
priority order. Each reference identifies a common problem with the display of
portions of very large images over a network, which is the latency and bandwidth
consumption associated with downloading an entire image, and arrives at similar
solutions in the form of requests for specific tiles in a priority order based on how
soon the tiles need to be displayed. 5
Therefore, a POSITA would readily recognize that the teachings of the three
references solving similar problems and providing similar solutions in closely
related fields could be considered in combination when designing a display system
addressing a similar problem.
In addition, a POSITA would readily recognize that the specific system for 10
requesting tiles by URL as in Hornbacker could be advantageously utilized in the
system of requesting and caching tiles for use in a 2D or 3D web browser of
Potmesil or Lindstrom, which also has to identify the specific tiles being requested,
in order to provide a common request format operable over the Internet. Ex. 1008,
¶¶ 153-54. 15
Element 2.C each said image parcel having an associated resolution,
The three references each teach the use of image tiles having a fixed
resolution for each tile, including a multi-resolution “pyramid” tile structure where
each level includes tiles at an associated resolution. Ex. 1002 at Fig. 1, 1329-30,
1332; Ex. 1003 at 6:20-7:25, 8:30-9:28, 10:3-10, 11:19-28, 12:21-13:10, 13:26-20
Petition for Inter Partes Review of U.S. Pat. No. 7,139,794 B2
-35-
14:6, Ex. 1011 at § 4.1. Ex. 1008, ¶ 155.
Element 2.D wherein said priority order is determined to provide a
progressive regional resolution enhancement of said image as each said image
parcel is rendered;
Potmesil teaches that images are stored in a power-of-two pyramid to 5
provide fast access to scroll and zoom operations. Ex. 1002 at Abstract, Fig. 1,
1329-30, 1332-33, Fig. 2. Lindstrom teaches that the multi-resolution structure
provides continuous level of detail representation of map imagery and that the
LOD manager in each view module uses a priority queue to request tiles. Ex. 1011
at §§ 1, 2, 4.1, 4.2.1, 4.2.3. 10
A POSITA would readily recognize that zooming using the tile pyramid of
Potmesil or Lindstrom would provide progressive regional resolution enhancement
of the image from lower to higher resolution as the user zooms in. Additionally, it
would be obvious to a POSITA that the “flight simulator” or other 3D visualization
modes of either Lindstrom or Potmesil would provide progressive regional 15
resolution enhancement of 3D rendered terrain as the user moves toward a point,
because it was well-known in the computer graphics field (e.g. in the OpenGL
standard) to utilize lower-resolution textures for more distant objects occupying a
smaller portion of the screen and transition to higher-resolution textures for objects
closer to the simulated user viewpoint. Ex. 1008, ¶ 156. Hornbacker teaches the 20
Petition for Inter Partes Review of U.S. Pat. No. 7,139,794 B2
-36-
use of progressive resolution enhancement for display of large images or portions
of large images. Ex. 1003 at 12:24-13:10. Each reference teaches alternative
reasons to retrieve lower-resolution tiles before higher resolution tiles, and arrives
at a similar solution.
A POSITA would readily recognize the similar solution of progressive 5
regional enhancement and, based on such cognition, would consider the teachings
of these references with regard to this claim limitation to be complementary. Ex.
1008, ¶ 157.
Element 2.E receiving a plurality of image parcels through said
communications channel; The three references provide their respective and 10
independent teachings that a plurality of image tiles are received over a network.
Ex. 1002 at Abstract, 1328, 1329-30, 1332-33, 1334-35, 1340-41; Ex. 1003 at
Abstract, 3:10-27, 5:3-6:19; Ex. 1011 at § 4, 4.2.1. Ex. 1008, ¶ 158.
Element 2.F rendering said plurality of image parcels to provide said image;
This limitation is substantially similar to claim element 1.D; therefore, the 15
same teachings apply.
Element 2.G wherein said step of receiving includes the step of storing said
plurality of image parcels in an image store;
This limitation is substantially similar to claim element 1.B; therefore, the
same teachings apply. 20
Petition for Inter Partes Review of U.S. Pat. No. 7,139,794 B2
-37-
Element 2.H wherein said step of rendering provides for the selective
rendering of said plurality of image parcels having the highest associated
resolutions to the corresponding regions of said image;
As discussed in regard to claim element 1.D, Potmesil teaches downloading
tiles including terrain and imagery data and rendering a 2D or 3D view. Ex. 1002 5
at Abstract, 1332, 1333-35, 1340. It would be obvious to a POSITA based on well-
known principles of computer graphics that the cache allocation process described
in Potmesil, based on the reference to the OpenGL standard and the use of a level-
of-detail parameter, would preferably determine which tiles to request and cache
based in part on the highest image resolution level that can be displayed; i.e. that 10
the highest level of resolution would be used for those regions in close enough
proximity to the viewpoint to render them at the highest associated resolution. Ex.
1008, ¶¶ 161-62.
Lindstrom provides details explaining that rendering is performed by
rendering modules and threads for each view, which selectively retrieve and render 15
tiles from shared and private caches. Ex. 1011, §§ 4, 4.2.1, 4.2.5, Fig. 1. Lindstrom
further teaches that the client rendering modules, including the LOD manager,
provide data requests via the shared priority queue for data of a desired type and
resolution. Ex. 1011 at § 4.2.1, Fig. 1. The LOD manager compares the level of
detail (both for terrain geometry and texels) to a threshold such as a screen space 20
Petition for Inter Partes Review of U.S. Pat. No. 7,139,794 B2
-38-
threshold. Id., § 4.2.3.
Therefore, it would be obvious to a POSITA in view of the combined
teachings of these two references that the caching algorithm and rendering process
of a 3D terrain visualization tool would selectively render the highest available
resolutions at the appropriate regions of the image, e.g. those nearest the user 5
viewpoint. Ex. 1008, ¶ 162.
Element 2.I wherein said step of rendering limits the selective rendering of
said image parcels to image parcels having associated resolutions less than a
predetermined level;
As discussed above in regard to claim element 1.D, Potmesil teaches 10
downloading tiles including terrain and imagery data and rendering a 2D or 3D
view. Ex. 1002 at Abstract, 1328, 1332, 1333-35, 1340. Ex. 1008, ¶¶ 164-64.
Lindstrom further teaches that the LOD manager in the client rendering
process limits the rendering step by comparing the level of detail to a threshold
such as a screen space threshold. Ex. 1011, § 4.2.3; Ex. 1008, ¶ 163. 15
Therefore, it would be obvious to a POSITA based on well-known principles
of computer graphics that the cache allocation process described in Potmesil, based
on the reference to the OpenGL standard and the use of a level-of-detail parameter,
would preferably request tiles at a level of detail no higher than that which can be
rendered and displayed by the system for a point at a given distance from the 20
Petition for Inter Partes Review of U.S. Pat. No. 7,139,794 B2
-39-
viewpoint. Lindstrom provides further detailed teachings illustrating how a
POSITA would implement this feature based on the comparison of the LOD to a
threshold. Ex. 1008, ¶ 164.
Hornbacker teaches that the resolution of tiles requested is determined based
on the size of the view area and the scale at which the user is viewing the image. 5
7:4-25, 11:19-28, 13:4-10, 14:2-6. It would be obvious to a POSITA in light of
these combined teachings that the tiles used in a system displaying a 2D or 3D
perspective view as in Potmesil or Lindstrom or at a given zoom level and scale as
in Hornbacker would preferably request tiles at a resolution that the display was
capable of displaying, e.g. less than the predetermined resolution of the display as 10
the tiles are rendered, but that would still produce a satisfying image. Ex. 1008, ¶¶
163-66.
Element 2.J wherein said step of rendering selectively renders said plurality
of image parcels as the unique textures for the corresponding regions of said image;
Potmesil teaches that the image tiles downloaded by client from the tile 15
server may include elevations, gradients, and RGB or color images, and that the
tiles may be rendered as the corresponding portions of a 2D or 3D model. Ex. 1002
at Abstract, 1327-28, 1329-30, 1334-35, 1340-41, Fig. 2, Fig. 8. Lindstrom
likewise teaches that the tiles rendered by the rendering process correspond to
specific geographical areas and would therefore have corresponding unique 20
Petition for Inter Partes Review of U.S. Pat. No. 7,139,794 B2
-40-
textures. Ex. 1011, § 4.1, Fig. 1.
Hornbacker teaches that a large-scale image is broken up into tiles, which
when displayed are rendered within the view area as the unique corresponding
portions of the image at a particular resolution. Ex. 1003 at 6:7-19, 7:11-25, 8:7-15.
Therefore, a POSITA would readily recognize that the three references 5
contain similar teachings to the effect that they relate to large images divided into
fixed-size tiles arranged in a pyramid by resolution and that the tiles contain the
textures for a corresponding portion of the image at a particular resolution. Ex.
1008, ¶¶ 167-69.
Element 2.K wherein said priority order is re-evaluated in response to a 10
change in said viewpoint orientation.
Potmesil teaches that the tile caching process requests new tiles as the user
moves an input device or as the viewpoint moves along a path. The requests are
based on the current viewpoint and anticipated path of the viewpoint and, therefore,
the priority of requested tiles is re-evaluated in response to a change in said 15
viewpoint orientation. Ex. 1002 at Abstract, 1328, 1332-33, 1340-41, Fig. 2.
Lindstrom teaches that when the user navigates within the 3D environment,
the terrain manager thread within each client module updates the priorities of
requests within the priority queue, which is dynamically updated to adjust
priorities based on the importance of requests determined by the LOD manager. Ex. 20
Petition for Inter Partes Review of U.S. Pat. No. 7,139,794 B2
-41-
1011, § 4.2.1, 4.2.6, Fig. 1.
Hornbacker similarly teaches that the list of requested tiles is re-evaluated
whenever the user makes a request to shift the view. Ex. 1003 at 7:11-8:6, 10:24-
28, 11:9-12:9.
In light of the finite cache space available, it would be obvious to a POSITA 5
in view of the teachings of the three references to prioritize tile requests in view of
the current viewpoint orientation and to re-evaluate such tile requests as the
viewpoint changes. Ex. 1008, ¶¶ 170-71.
B. GROUND 2: CLAIM 1 IS UNPATENTABLE UNDER 35 U.S.C. § 103(a) AS BEING OBVIOUS OVER RUTLEDGE IN VIEW OF 10 LIGTENBERG AND COOPER
Rutledge, Ligtenberg and Cooper are in the same context of viewing map or
graphical images and disclose techniques related to downloading visual data from a
server to client device via a network connection such as a local area or a wide area
connection. In particular, Rutledge, Ligtenberg and Cooper deal with the common 15
issues in situations where the network connection may have limited bandwidths. A
POSITA would readily recognize from the teachings of the three references that
they could naturally be combined based on the closeness of the technical solutions
to the similar technical problems in the same technical field.
Rutledge discloses, among other things, a map database containing map tiles 20
of different zoom level and resolution. Ex. 1005 at 5:64; Fig. 3. A user terminal
Petition for Inter Partes Review of U.S. Pat. No. 7,139,794 B2
-42-
can retrieve map tiles needed for display in a hierarchical order in response to
user’s navigation commands such as zoom and pan. Id. at 7:3-8:49; 6:38-50, Fig. 5.
The user terminal access map data via either a modem or a computer
communication network such as the internet. Id. at 2:62-64. The map tiles may
include satellite imagery, digitized maps and scanned images. Id. at 4:42-47, 6:11-5
17. Rutledge also teaches that map tiles may comprise map objects, which may
include entities such as polygons that are drawn in an order. Id. at 6:18-36.
Ligtenberg discloses a file format and map data storage format that can be
used for efficiently downloading and rendering images over a network. Ex. 1004,
1:16-42. The network may include the internet, a local area or a wide-area network 10
including telephone lines. Id. at 2:37-46, 5:14-17. Ligtenberg also teaches
subdividing images into rectangular arrays of tiles and recursively compressing
these tiles into a series of reduced resolution tiles, until the resulting reduced image
tile is of a desired small size. Id. at Abstract. Ligtenberg’s file format allows for the
storage and transmission of map tiles of various resolutions, which can all have the 15
same pixel dimension, thereby allowing retrieval of desired portions of the image
data at desired resolutions by optimally using the I/O bandwidth and processor. Id.
at Appendix A; 2:25-30.
Cooper is in a similar field of downloading visual data from a server to
client device via a network connection such as a local area or a wide area 20
Petition for Inter Partes Review of U.S. Pat. No. 7,139,794 B2
-43-
connection, where the network connection may have limited bandwidths. Cooper
discloses a technique that optimizes rendered image quality based on user’s
viewpoint by assessing the importance of various objects in a 3D graphical scene,
requesting each object’s data from a server in a priority order, and recalculating the
object’s importance when the user’s viewpoint changes. Ex. 1006 at Abstract, 5
2:19-24; 2:49-54. This aspect of Cooper allows for immediate response to a
changing view position and reduces visual latency when the object’s data are
transmitted over a limited bandwidth network. Id.
A POSITA would have readily recognized that Rutledge’s map and image
browsing technique would benefit from Ligtenberg’s file format that allows for 10
selective retrieval and rendering of images at desired resolution, thereby resulting
in optimized I/O bandwidth use. Ex. 1004 at 2:37-46, 5:14-17. Ex. 1008, ¶ 193.
A POSITA would have also readily recognized that the combined technique
of Rutledge Ligtenberg would benefit from Cooper’s data requests based on
prioritization, by providing a realistic looking map browsing experience in which 15
visual latency of browsing is reduced. Ex. 1006 at 1:33-53. Ex. 1008, ¶¶ 195-200.
A POSITA would have readily found it obvious to combine Rutledge with
Ligtenberg and Cooper because the three references disclose techniques for
communicating visual data from a server to a client device over a communication
network. Ex. 1004, 1:16-42; Ex. 1005 at 2:62-64; Ex. 1006 at Abstract, 2:19-24; 20
Petition for Inter Partes Review of U.S. Pat. No. 7,139,794 B2
-44-
2:49-54. The three references all teach incrementally sending visual data from
server to client - e.g., map tiles of Rutledge that are sent based on a zoom layer, tile
blocks of Ligtenberg are sent based on a layer of a given resolution, and objects in
a scene of Cooper that are incrementally sent as polygons of the object. Ex. 1004 at
2:31-37; 3:1-6; 5:48-52; 11:34-36; Appendix A; Ex. 1005 at 3:5-10; 5:24-44; 5:53-5
58; 7:11-14; 8:38-47; Ex. 1006 at 1:54-60; 2:5-8.
All three references disclose that the visual data sent from the server to the
client device is based on an observer’s viewpoint - e.g., a zoom layer and pan
command from Rutledge’s user, or a user specified resolution in Ligtenberg or a
user’s viewpoint as in Cooper . Ex. 1004, 1:16-42; 5:25-28 Ex. 1005 at 8:4-6, 7:48-10
62; Ex. 1006 at Abstract, 2:19-24; 2:49-54; 3:9-13.
Each reference also discloses that visual data elements are made available
from server in multiple resolutions that can progressively provide a higher-
resolution image to the observer. Ligtenberg discloses a file format for storing and
communicating map tiles. Cooper discloses a priority-based scheme used by the 15
client device when requesting visual data from the server. Ex. 1006 at Abstract;
3:5-7; 4:35-37; 6:16-24; 6:62-7:11.
Therefore, a person of ordinary skill in the art would have been motivated to
combine Ligtenberg and Cooper with Rutledge to benefit from the reduced I/O and
CPU utilization of Ligtenberg and Cooper’s efficient use of network bandwidth. 20
Petition for Inter Partes Review of U.S. Pat. No. 7,139,794 B2
-45-
Ex. 1008, ¶¶ 193-200.
As demonstrated in detail below, the combination of Rutledge, Ligtenberg
and Cooper collectively teaches or suggests all the limitations of claim 1 and
renders this claim as a whole obvious and unpatentable. Ex. 1008, ¶¶ 172-241.
Element 1.Preamble A client system for dynamic visualization of image 5
data provided through a network communications channel, said client system
comprising:
Rutledge discloses a user terminal that receives map data via a
communication network. Ex. 1005, 1:22-23. Rutledge’s system downloads maps as
image tiles stored in a map database. Id. at 3:50-55, 4:41-47, 6:38-43. The maps 10
can be both raster and vector data from a wide variety of sources including satellite
imagery, digitized maps and scanned images. Id. at 4:44-47. The visualization is
dynamic because a user can pan and zoom in or out and navigate through the
image. Id. at Fig. 4D. Corresponding to the user’s viewpoint changes, map tiles are
transferred from the map database to the user terminal. Id. at 7:41-45. 15
Ligtenberg also discloses a technique in which image data that is stored as
tiles of multiple resolutions is sent from a server to a client for selective displaying
Ex. 1004 at 1:16-42. In particular, Ligtenberg discloses a file format used for
storage and transmission of map tiles with different resolutions. Id. at Appendix A.
Cooper discloses a technique for retrieving image object data from a server 20
Petition for Inter Partes Review of U.S. Pat. No. 7,139,794 B2
-46-
and rendering on a user device or a client device. The Cooper technique prioritizes
which visual data objects to retrieve based on observer’s viewpoint in order to
efficiently utilize available network bandwidth. Ex. 1006 at 6:11-24.
A POSITA would have understood that Rutledge, Ligtenberg and Cooper,
each reference discloses progressive download of visual object data, whether 5
computer graphics objects, or map tiles or image data, to allow viewpoint
dependent rendering at the client-side. Further, both Rutledge and Ligtenberg used
a format in which images are stored as tiles of certain resolutions.
A POSITA would have readily recognized that Ligtenberg’s file format
would benefit Rutledge to reduce CPU and I/O resource utilization to be 10
proportional only to data needed. Ex. 1004 at 11:67-12:2. A POSITA would have
further recognized that Cooper’s use of priorities for requesting additional visual
data would benefit the combination of Rutledge and Ligtenberg by providing a
way by which available network bandwidth can be efficiently used. Ex. 1008, ¶¶
202-206. 15
Element 1.A a parcel request subsystem, including a parcel request queue,
operative to request discrete image data parcels in a priority order:
Cooper discloses an object assessment function at a client that maintains a
list of visible objects in a priority queue in accordance with an instantaneous
viewpoint of a hypothetical viewer within a 3D environment. Ex. 1006 at 4:48-51. 20
Petition for Inter Partes Review of U.S. Pat. No. 7,139,794 B2
-47-
Cooper also discloses a streaming function that manages the request and receipt of
object data from the server by making requests in accordance with contents of the
priority queue, starting with the most important objects. Id. at Abstract, 4:61-62,
5:2-6, 5:16-19, 7:6-11.
A POSITA would have been motivated to combine Cooper’s priority queue 5
with Rutledge and Cooper’s map tile fetching to control network bandwidth
utilization while providing high quality user experience when navigating around a
map image. Ex. 1008, ¶¶ 207-209.
Element 1.B to store received image data parcels in a parcel data store:
Ligtenberg discloses that the client device downloads image portions and 10
stores them in a memory. Ex. 1004 at 3:49-51; 4:28-32, 4:55-61; 7:4-6, 9:6-11,
10:1-3. Cooper also discloses storing the received visual object data at the client
device. Ex. 1006 at 4:39-41. It would have been obvious to a POSITA that the
memory of Ligtenberg or Cooper can be combined with Rutledge’s disclosure for
storing image data received by Rutledge’s client device during reception and 15
rendering operation. Ex. 1008, ¶¶ 211-213.
Element 1.C said parcel request subsystem being responsive to an image
parcel request of assigned priority to place said image parcel request in said parcel
request queue ordered in correspondence with said assigned priority:
Cooper’s object assessment function maintains a list of visible objects in the 20
Petition for Inter Partes Review of U.S. Pat. No. 7,139,794 B2
-48-
priority queue. When a scene updates, e.g., when user’s viewpoint changes, all
objects in the queue are reprioritized is organized according to importance of
objects with respect to viewer’s perspective. Object assessment function 22
determines a priority value for each visible object among the objects in object list
20 and orders the list of objects in priority queue 24 correspondingly. Streaming 5
function sends requests to the server starting with the most important object first.
Ex. 1006 at 4:48-60, 6:27-32, 7:16-44, 9:65-10:2.
A POSITA would understand that one way by which requests can be ordered
is by placing them in an order queue corresponding to the order in which the
corresponding object data is to be received. A POSITA would be motivated to 10
organize requests for Rutledge and Ligtenberg’s map tiles at the client device using
Cooper’s prioritization based on importance of map tiles to a user’s viewpoint to
improve the efficiency of network bandwidth utilization. Ex. 1008, ¶ 215.
Element 1.D a parcel rendering subsystem coupled to said parcel data store
to selectively retrieve and render received image data parcels to a display memory: 15
Cooper discloses a display device that access data from an object data table
for each visual data object. The received data objects are stored in an object data
table and made available to a rendering program. Ex. 1006 at 6:11-10-45 and FIG.
5. The retrieval is selective because “most important objects in the scene are for the
current viewpoint are most accurately rendered.” Id. at 6:28-38. Cooper’s user 20
Petition for Inter Partes Review of U.S. Pat. No. 7,139,794 B2
-49-
device also makes a determination of which object should be rendered at what
level of detail. Id. at 5:59-65, 1:54-65, 5:6-8, 6:28-32. Ligtenberg discloses a
display that includes a display memory for storing pixels that appear on the display
device. Ex. 1004 at 5:1-8. An image available for display is sent to display memory.
Id. at 10:2-6. 5
To store display data during displaying, a POSITA would have used
Ligtenberg’s display memory with Cooper’s display device to implement the
image parcel store in Rutledge’s client device because it would have been obvious
to a POSITA that the image parcel store performed the function of storing image
parcels, which can be implemented using a memory. Ex. 1008, ¶¶ 217-18. 10
Element 1.E said parcel rendering system providing said parcel request
subsystem with said image parcel request of said assigned priority:
Cooper’s object assessment function maintains a list of visible objects in
priority queue in accordance with an instantaneous viewpoint of a hypothetical
viewer within 3-D environment. The instantaneous viewpoint is updated in 15
accordance with user commands as processed by interactive 3-D graphical display
system. Object assessment function 22 determines a priority value for each visible
object among the objects in object list 20, in accordance with an algorithm and
orders the list of objects in priority queue 24 correspondingly. Ex. 1006 at 3:44-49,
4:47-60, 6:51-59. 20
Petition for Inter Partes Review of U.S. Pat. No. 7,139,794 B2
-50-
A person of ordinary skill in the art would have combined Cooper’s
disclosure of viewpoint based priority assignment and ordering of list of object for
retrieval from the server with Rutledge and Ligtenberg’s user device to benefit
from efficient use of network bandwidth due to Cooper’s prioritization scheme. Ex.
1008, ¶ 220. 5
Element 1.F wherein said parcel rendering subsystem determines said
assigned priority based on a determined optimal image resolution level:
Cooper’s object assessment function determines visual importance of objects
based on user viewpoint and thus determines an order in which visual data is
received. Ex. 1006 at Abstract, 3:44-49, 6:51-58, 7:12-27, 10:39-42, 11:47-54. 10
Rutledge discloses that user viewpoint can be used to select a corresponding map
resolution level. For example, when the user requests a more detailed map by
zooming, the server retrieves more detailed map tiles corresponding to a currently
displayed viewpoint. Ex. 1005 at 7:38:62, 8:25-30.
A POSITA would have combined Cooper’s priority assignment with 15
Rutledges’s zoom functionality to efficiently use the available network bandwidth.
Further, a POSITA would have combined this feature with Ligtenberg to benefit
from low I/O utilization due to Ligtenberg’s I/O platform. Ex. 1008, ¶¶ 222-23.
Element 1.G wherein said display memory is coupled to an image display of
predetermined resolution: Rutledge discloses that images were displayed on a 20
Petition for Inter Partes Review of U.S. Pat. No. 7,139,794 B2
-51-
computer terminal 110. Ex. 1005 at 5:24-47. Ligtenberg discloses that display
terminal is used for displaying images that has a known resolution measured in
dots per inches (dpi). Ex 1004 at Fig. 1, 1:63-67, 5:1-8. It would have been obvious
to one of skill in the art that displays are defined based on a predetermined
resolution (e.g., an NTSC monitor or a high definition display, etc.) Ex. 1008, ¶ 5
225.
Element 1.H wherein said determined optimal image resolution level is
based on said predetermined resolution: Ligtenberg discloses that the resolution
needed for output display can be used for data retrieval, instead of resolution of the
original image, to save CPU and I/O resources. Ex. 1004 at 11:66-12:4. A POSITA 10
would have combined this feature of Ligtenberg with Rutledge’s map data viewing
device and Cooper’s priority based data retrieval to benefit from reduced CPU use
and I/O utilization. Ex. 1008, ¶¶ 227.
Element 1.I wherein said assigned priority further reflects the proximity of
the image parcel referenced by said image parcel request to a predetermined focal 15
point: Cooper discloses that visual data objects are prioritized based on 3-
dimensional location of a user’s viewpoint, including distance from a focal point of
the display. Ex. 1006 at 7:48-52, 8:23-32, 9:1-12. A POSITA would have
combined the focal point based prioritization of Rutledge’s zoom level and
Cooper's viewpoint with Ligtenberg’s file format to give a higher importance to 20
Petition for Inter Partes Review of U.S. Pat. No. 7,139,794 B2
-52-
objects which the user is most likely to be looking at, using a file format that is
CPU and I/O bandwidth efficient. Ex. 1008, ¶¶ 229-30.
Element 1.J wherein said discrete image data parcels are of a first fixed size
as received by said parcel request subsystem:
Ligtenberg discloses the use of interpolation to receive images at one 5
resolution and render them at a different resolution. Ex. 1004 at 2:31-38, 5:39-43.
The images can be received as tile blocks, which have the same length. Ex. 1004 at
6:51-56, 7:1-20. A POSITA would be motivated to combine Ligtenberg’s use of
tile blocks of same lengths with Rutledge and Cooper to provide an efficient
representation of image data without having to specify tile size for each resolution 10
layer. Ex. 1008, ¶¶ 232-34.
Element 1.K of a second fixed size as rendered by said parcel rendering
subsystem: Recognizing that the size at which images are rendered can be different
from the size at which they are received, Ligtenberg provides for an interpolation
technique using which images are expanded to a size suitable for display. Ex. 1004 15
at 8:43-59, 9:62-10:21. This teaching of the image interpolation by Ligtenberg
would cause the rendered image to have a different, second fixed size relative to
the received first fixed size. A POSITA would be motivated to combine
Ligtenberg’s use of fixed size data parcels with Rutledge and Cooper to provide an
efficient representation of image data. Ex. 1008, ¶¶ 236-37. 20
Petition for Inter Partes Review of U.S. Pat. No. 7,139,794 B2
-53-
Element 1.L wherein said discrete image data parcels each includes a fixed-
size array of pixel data: Both Rutledge and Ligtenberg disclose that image data can
be transferred from server to client in units of fixed size arrays. Ex. 1004 at 2:56-
62, 6:52-57, Abstract, 7:19-21; Ex. 1005 at 5:50-62. A POSITA would have used
Ligtenberg’s and Rutledge’s fixed size map tiles with Cooper’s priority queue to 5
provide a bandwidth-efficient representation of image data. Ex. 1008, ¶¶ 239-40.
C. GROUND 3: CLAIM 2 IS UNPATENTABLE UNDER 35 U.S.C. § 103(a) AS BEING OBVIOUS OVER RUTLEDGE IN VIEW OF LIGTENBERG, COOPER AND MIGDAL
Migdal is in the same technical context of Rutledge, Ligtenberg and Cooper 10
for downloading visual data from a server to client device via a network
connection such as a local area or a wide area connection. Specifically, Migdal
discloses a user device that downloads and displays graphics images such as
satellite image data or aerial photographs as texture maps. Ex. 1007 at :14-21,
3:36-50, 5:53-58, 6:9-19, 7:37-39, 7:62-8:4, 10:55-65, 11:4-10, 13:1-2, 14:29-34, 15
18:6-9, 19:12-14, 21:59-61, 22:20-22 and Abstract. Migdal also teaches the use of
three-dimensional texture rendering for rendering the satellite images, aerial
photographs and flight simulation images. Id. Migdal further discloses a technique
called “clip map” to efficiently reduce memory and processor requirements to
render photographic terrain textures. Id. at Abstract; 3:6-11; 10:14-24. 20
As discussed for claim 1, a POSITA would have readily combined Rutledge
Petition for Inter Partes Review of U.S. Pat. No. 7,139,794 B2
-54-
with Ligtenberg and Cooper. A POSITA would have also realized that Migdal’s
texture mapping technique would further benefit the Rutledge-Ligtenberg-Cooper
combination by further reducing memory and processor requirements of a user
device for rendering map and image data to a user. Ex. 1008 at ¶¶ 242-45.
Element 2.Preamble A method of supporting dynamic visualization of 5
image data transferred through a communications channel, said method comprising
the steps of: The preamble of Claim 2 is substantially identical to the preamble of
Claim 1, therefore the same teachings apply.
Element 2.A determining, in response to user navigational commands, a
viewpoint orientation with respect to an image displayed within a three-10
dimensional space: Rutledge discloses the determination of a viewpoint orientation
based on latitude, longitude and zoom level. Ex. 1005 at 3:5-10, 5:14-23, 4:41-47,
7:63 - 8:6, 9:11-17. Cooper discloses determining a viewpoint orientation within an
image displayed in a three-dimensional space that includes rotational changes to
the user’s viewpoint Ex. 1006 at Abstract, 1:6-10, 1:16-21, 1:23-29, 1:34-38, 3:16-15
54, 3:59-65, 4:14-19, 4:27-29, 4:39-47, 4:48-51, 5:26-36, 5:48-57; Ex. 1008 at ¶¶
249-51. Cooper discloses that during the viewing, user viewpoint orientation may
change in a 3-D space. Id. Cooper also disclosed that what’s being seen by the
observer is also displayed in a 3-D space. Id. Migdal teaches that images such as
terrains, aerial textures, flight simulations, satellite imagery, etc. can be displayed 20
Petition for Inter Partes Review of U.S. Pat. No. 7,139,794 B2
-55-
as three dimensional data using three dimensional rendering techniques. Ex. 1007
at :14-21, 3:36-50, 5:53-58, 6:9-19, 7:37-39, 7:62-8:4, 10:55-65, 11:4-10, 13:1-2,
14:29-34, 18:6-9, 19:12-14, 21:59-61, 22:20-22.
A person of ordinary skill in the art would have combined Ligtenberg’s file
format with Cooper and Rutledge’s use of a 3-dimensional viewpoint in order to 5
provide satisfactory user experience. Ex. 1008 at ¶ 252. Furthermore, a person of
ordinary skill in the art would have combined Migdal’s three dimensional texture
rendering technique with Rutledge, Ligtenberg and Cooper to provide a reduced-
memory implementation for rendering image data such as satellite images. Ex.
1008 at ¶¶ 243-45. 10
Element 2.B requesting, in a priority order, image parcels renderable as
corresponding regions of said image, Cooper discloses an object assessment
function at a client that maintains a list of visible objects in a priority queue in
accordance with an instantaneous viewpoint of a hypothetical viewer within a 3D
environment. Ex. 1006 at 4:48-58. Cooper also discloses a streaming function that 15
manages the request and receipt of object data from the server. Id. at 4:61-5:6. The
visual data objects are in a priority order and the importance of each object is
recalculated per scene update and new requests are sent out based on these values.
Ex. 1008 at ¶ 254.
A POSITA would have been motivated to combine Cooper’s requesting of 20
Petition for Inter Partes Review of U.S. Pat. No. 7,139,794 B2
-56-
visual data in a priority order with Rutledge, Cooper and Migdal to control
network bandwidth utilization while providing high quality user experience when
navigating around a map image. Since Rutledge, Ligtenberg and Migdal teach
storing image and map data as multiple tiles of different resolution such that
different tiles have different importance for rendering to a viewer based on the 5
viewpoint, the use of Cooper’s priority queue would have been a natural
modification to Rutledge, Ligtenberg and Migdal combination that is well within
the skill of a POSITA. Ex. 1008 at ¶¶ 255-57.
Element 2.C each said image parcel having an associated resolution:
Rutledge and Ligtenberg disclose that image data can be transferred from 10
server to client in units of fixed size arrays. Ex. 1004 at 2:56-62, 6:52-57, Abstract,
7:19-21; Ex. 1005 at 5:50-62; Ex. 1008 at ¶ 259. A POSITA would have used
Ligtenberg’s and Rutledge’s map tiles having an associated resolution with
Cooper’s priority queue and Migdal’s clip-map processing as an implementation
choice for a bandwidth-efficient representation of image data. Ex. 1008 at ¶ 260. 15
Element 2.D wherein said priority order is determined to provide a
progressive regional resolution enhancement of said image as each said image
parcel is rendered:
Cooper discloses that visual data objects are requested in a priority order
from a priority queue. Ex. 1006 at Abstract, 4:34-38, 4:48-58, 4:61-5:6, 6:16-24, 20
Petition for Inter Partes Review of U.S. Pat. No. 7,139,794 B2
-57-
6:27-32, 7:12-16, 9:65-10:5. Cooper discloses sending requests to server for
objects starting from most important objects and progressing to least important
objects. Id. at 6:16-21. When an object has a data deficit (i.e., object has been
received at a lower resolution), the object is inserted into the priority queue. Id. at
10:7-22. Cooper thus prioritizes requests for data so as to more completely render 5
those objects that contribute more to a scene. Id. at 7:4-11. Ex. 1008 at ¶ 262.
A POSITA would have combined Cooper’s object resolution improvement
using a priority order with Rutledge’s map data delivery, Ligtenberg’s file format,
and Migdal’s texture rendering of maps to provide high quality user experience
while efficiently using the available network bandwidth. Ex. 1008 at ¶ 263. 10
Element 2.E receiving a plurality of image parcels through said
communications channel; This limitation is substantially similar to claim element
1.B; therefore, the same teachings apply. Ex. 1008 at ¶ 264.
Element 2.F rendering said plurality of image parcels to provide said image;
This limitation is substantially similar to claim element 1.D; therefore, the 15
same teachings apply. Ex. 1008 at ¶ 265.
Element 2.G wherein said step of receiving includes the step of storing said
plurality of image parcels in an image store: Ligtenberg discloses that the client
device downloads image portions and stores them in a memory. Ex. 1005 at 3:49-
52; 4:28-32, 4:55-61; 7:4-6, 9:6-11, 10:1-15. Cooper both explicitly disclose that a 20
Petition for Inter Partes Review of U.S. Pat. No. 7,139,794 B2
-58-
memory is used at the client side to store visual data. Ex. 1006 at 4:39-41. Ex.
1008 at ¶ 266. It would have been obvious to a POSITA that the memory of
Ligtenberg or Cooper can be combined with Rutledge’s disclosure for providing
storage of image data received by Rutledge’s client device. Ex. 1008 at ¶ 266.
Element 2.H wherein said step of rendering provides for the selective 5
rendering of said plurality of image parcels having the highest associated
resolutions to the corresponding regions of said image; Rutledge discloses that a
user can navigate around the image by performing zoom and pan operations. Ex.
1005 at 7:37-67. When rendering, Cooper takes into account user’s viewpoint and
selectively renders scenes at a level proportional to its importance to the user’s 10
viewpoint. Ex. 1006 at 7:12-17, 11:47-54.
A POSITA would have combined the selective rendering of Cooper with
Rutledge’s pan and scan using Ligtenberg’s file format and Migdals’ texture
mapping to provide a rapid response to the viewer’s motion in a network
environment and provide objects being rendered according to their visual 15
importance. Ex. 1008 at ¶ 268.
Element 2.I wherein said step of rendering limits the selective rendering of
said image parcels to image parcels having associated resolutions less than a
predetermined level; Ligtenberg discloses that data is only a limited amount of tile
data is needed for generating image. Ex. 1004 at 8:37-42, 8:60-65, 9:2-3. It would 20
Petition for Inter Partes Review of U.S. Pat. No. 7,139,794 B2
-59-
have been obvious to one of skill in the art that because displays have a finite
resolution and because a display cannot display an image having a resolution
greater than that of the display, image parcel rendering would have an upper limit
of the resolution that would be used for rendering image parcels to save
computational resources and memory usage. Ex. 1008 at ¶ 270. 5
Element 2.J wherein said step of rendering selectively renders said plurality
of image parcels as the unique textures for the corresponding regions of said image:
Migdal discloses that image data such as photographic terrain images, can be
rendered as texture tiles. Ex. 1008 at ¶ 272. One of skill in the art would have
combined Migdal with the combination of Rutledge with Ligtenberg and Cooper to 10
benefit from Migdal’s reduction in implementation complexity when rendering
images or map data using Rutledge’s image rendering technique with Ligtenberg’s
file format, Cooper’s priority queue. Ex. 1008 at ¶ 273.
Element 2.K wherein said priority order is re-evaluated in response to a
change in said viewpoint orientation. Cooper recalculates importance of each 15
object per scene update. When the scene changes in response to a user-commanded
change in viewpoint, all objects are reprioritized. Ex. 1006 at Abstract, 6:27-35.
One of skill in the art would have combined Cooper’s reprioritization with
Rutledge’s use of viewpoint orientation based on latitude, longitude and zoom
level to effectively rendering an image to a user Ex. 1008 at ¶ 275. Furthermore, a 20
Petition for Inter Partes Review of U.S. Pat. No. 7,139,794 B2
-60-
POSITA would have combined Migdal’s texture rendering technique with
Rutledge, Ligtenberg and Cooper to provide a reduced-memory implementation
for rendering image data such as satellite images. Ex. 1008 at ¶ 276.
VII. CONCLUSION
This Petition has demonstrated a reasonable likelihood that Petitioner 5
Microsoft will prevail in its challenge of patentability for claims 1 and 2 of the 794
Patent. Petitioner Microsoft respectfully requests that a trial for IPR review of the
794 Patent be instituted.
In addition, this Petition has shown that, by a preponderance of the evidence,
claims 1 and 2 are invalid. Therefore, Petitioner Microsoft respectfully requests 10
claims 1 and 2 be canceled.
Cancellation of these claims will prevent Patent Owner from improperly
claiming technologies as its own that were already known in the prior art before its
patent filing and were already in the public domain, and will stop Patent Owner
from asserting invalid patent claims to exclude Petitioner Microsoft and others. 15
Moreover, cancellation of these claims will avoid the unnecessary litigation and
reduce wasteful use of judicial resources.
Petition for Inter Partes Review of U.S. Pat. No. 7,139,794 B2
-61-
Dated: June 16, 2015 PERKINS COIE LLP 11988 El Camino Real, Suite 350 San Diego, CA 92130 (858) 720-5700
Respectfully submitted,
/Bing Ai/ Lead Counsel Bing Ai, Reg. No. 43,312 Back-up Counsel Matthew Bernstein, Pro Hac Vice Vinay Sathe, Reg. No. 55,595 Patrick McKeever, Reg. No. 66,019 Attorneys for Microsoft Corporation
Petition for Inter Partes Review of U.S. Pat. No. 7,139,794 B2
-1-
CERTIFICATE OF SERVICE The undersigned hereby certifies that a true copy of the foregoing
PETITION FOR INTER PARTES REVIEW OF U.S. PATENT NO. 7,139,794 B2
and supporting materials (Exhibits 1001-1013 and Power of Attorney) have been
served in its entirety this Sixteenth Day of June, 2015, by FEDERAL EXPRESS®
on Patent Owner at the correspondence address for the attorney of record for the
794 Patent shown in USPTO PAIR:
3DVU 28 LEVY ESCHOL STREET
RANNANA 43703
and by electronic mail on the attorneys of record for Plaintiffs in the concurrent
litigation matter:
Mark A. Hannemann [email protected] KENYON & KENYON, LLP One Broadway New York, NY 10004-1007
Christopher J. Coulson [email protected] KENYON & KENYON, LLP One Broadway New York, NY 10004-1007
Michael N. Zachary [email protected] KENYON & KENYON, LLP 1801 Page Mill Road, Ste 210Palo Alto, CA 94304
John W. Bateman [email protected] KENYON & KENYON, LLP 1500 K. Street, NW Washington, DC 20005-1257
John C. Phillips, Jr. [email protected] PHILLIPS, GOLDMAN & SPENCE, P.A. 1200 North Broom Street Wilmington, DE 19806
David A. Bilson [email protected] PHILLIPS, GOLDMAN & SPENCE, P.A. 1200 North Broom Street Wilmington, DE 19806
/Bing Ai/ Lead Counsel Bing Ai, Reg. No. 43,312 Attorney for Microsoft Corporation