Patent Classifications as Patent Classifications as ‘Knowledge’‘Knowledge’

…towards a more conscious …towards a more conscious (auto)categorization of patents(auto)categorization of patents

Arcanum DevelopmentArcanum Development20132013

A usual hierarchic categorization A usual hierarchic categorization task…task…

Given a hierarchic taxonomy (classification Given a hierarchic taxonomy (classification system)system)Provide a list of taxonomy nodes (classification Provide a list of taxonomy nodes (classification symbols) for a document that best match the symbols) for a document that best match the subject matter of the taxonomy nodesubject matter of the taxonomy node““Best” is based on…Best” is based on…– For experts: understanding the subject matterFor experts: understanding the subject matter

– For computers: providing the hierarchy, and training For computers: providing the hierarchy, and training

with sample categorized documentswith sample categorized documents

However, patent classification task is somewhat However, patent classification task is somewhat more complicated…more complicated…

and considering classification rules

…and the rules?!

Typical methods of categorizationTypical methods of categorization

Roots (sections)

Non-classifying (‘preclassification’) levels

flatthe best is the winner on

each level

greedy hierarchictraversing down on

best only

… during training…

Common features of patent Common features of patent classification schemesclassification schemes

HierarchicHierarchic– covered subject matter of a higher level contains subject matter of a covered subject matter of a higher level contains subject matter of a

lower levellower level– but: may be assigned to a higher level when none of the lower levels fitbut: may be assigned to a higher level when none of the lower levels fit

Some nodes (symbols) cannot be used (in general or alone) for Some nodes (symbols) cannot be used (in general or alone) for classificationclassification– hierarchy levelshierarchy levels– indexing schemes indexing schemes

Schemes contain specific rules – Schemes contain specific rules – relationsrelations – between symbols – between symbols– Place / priority / precedence / limiting rulesPlace / priority / precedence / limiting rules– Indexing rulesIndexing rules– References to symbols to be taken in considerationReferences to symbols to be taken in consideration

Rules given in the scheme are extended by definitions / manuals of Rules given in the scheme are extended by definitions / manuals of classificationclassificationSchemes can be multilingualSchemes can be multilingualUsed by various offices, cultures (maybe slightly differently)Used by various offices, cultures (maybe slightly differently)

Using relations in patent Using relations in patent categorizationcategorization

Hierarchic, rules, references

Last place rule

Takes precedence

Why a (more) formal analysis and Why a (more) formal analysis and presentation is advantageous?presentation is advantageous?

Recently: the rules are Recently: the rules are presented as textpresented as text, in , in various master filesvarious master files, machine-readable but not , machine-readable but not machine-interpretable way (it is ‘content’ but not machine-interpretable way (it is ‘content’ but not yet ‘knowledge’)yet ‘knowledge’)Lots of complex rules spread over Lots of complex rules spread over multiple multiple sourcessources (e.g. definitions) and places (e.g. (e.g. definitions) and places (e.g. reverse references)reverse references)

Both for humans and computer programs, it Both for humans and computer programs, it causes trouble to causes trouble to collect and applycollect and apply all the rules all the rules systematicallysystematically

It is worth then to convert IPC content to It is worth then to convert IPC content to more explicit IPC ‘knowledge’…more explicit IPC ‘knowledge’…

HypothesesHypotheses

Tests were made Tests were made – to verify confusionsto verify confusions

of patent examinersof patent examinersof various autocategorizersof various autocategorizers

– ……if they are in correlation to relations given in IPCif they are in correlation to relations given in IPC

AssumptionAssumption– more references in IPC: higher overlap between more references in IPC: higher overlap between

subject matter areasubject matter area

HypothesesHypotheses– the more references in IPC between two areas,the more references in IPC between two areas,

the higher the confusion of humans and computersthe higher the confusion of humans and computers– the knowledge coded in IPC is, indeed, the knowledge coded in IPC is, indeed, usedused by by

patent categorizerspatent categorizers

Testing the hypothesesTesting the hypotheses

If patent examiners take seriously references in If patent examiners take seriously references in IPCIPC– the more references between two symbols, the higher the more references between two symbols, the higher

number of co-classificationnumber of co-classification

Practice between two offices can be differentPractice between two offices can be different– the more references in IPC, the higher likelihood of the more references in IPC, the higher likelihood of

different decisiondifferent decision

Confusion of autocategorizersConfusion of autocategorizers– more failures if subject matter area is overlappingmore failures if subject matter area is overlapping

Cocategorization vs. IPC referencesCocategorization vs. IPC references

When there are When there are references in IPC, references in IPC, patent examiners patent examiners take them seriouslytake them seriously– References mark References mark

overlapping subject overlapping subject matter areas matter areas and/or and/or

– References References propose the use of propose the use of secondary secondary (indexing) symbols(indexing) symbols

On class level, On class level, frequency of frequency of references in IPC is references in IPC is similar similar to the frequency of to the frequency of common use of common use of symbols of both symbols of both classes in patent classes in patent documentsdocuments

A61 vs C07 and C12

C07-C12

F16

G-H

B65

A47,A61

B60-B65

Differences in examiner’s practiceDifferences in examiner’s practice

When there are When there are references in IPC, references in IPC, patent examiners patent examiners may assign them may assign them differentlydifferently– References mark References mark

overlapping overlapping subject matter subject matter areas areas

On class level, On class level, frequency of frequency of references in IPC references in IPC is similar is similar to differences to differences between selected between selected first symbol first symbol (prereform (prereform practice, simulate practice, simulate preclassification)preclassification)

A61 vs C07 and C12

C07-C12

F16

G-H

B65

A47,A61

B60-B65

Confusion of autocategorizersConfusion of autocategorizers

When there are When there are references in IPC, references in IPC, autocategorizers autocategorizers fail more frequentlyfail more frequently– ““first symbol” may first symbol” may

be selected be selected differently differently

On class level, On class level, frequency of frequency of references in IPC references in IPC is similar is similar to differences to differences between selected between selected first symbolfirst symbol of an of an autocategorizer autocategorizer (2002 data, to (2002 data, to simulate simulate preclassification)preclassification)

A61 vs C07 and C12

C07-C12

F16

G-H

B65

A47,A61

B60-B65

ConclusionConclusion

1.1. Reference statistics in IPCReference statistics in IPC2.2. Co-classificationCo-classification3.3. Human classification differencesHuman classification differences4.4. Preclassification autocategorization errorsPreclassification autocategorization errors

show similar characteristicsshow similar characteristicson higher levels of IPCon higher levels of IPC

It may be even more important on lower levels, having It may be even more important on lower levels, having there more complex rulesthere more complex rules

Therefore, an easier access to the rules maybe Therefore, an easier access to the rules maybe welcome both by human and machine categorizerswelcome both by human and machine categorizers

Presentation of IPCInfoPresentation of IPCInfo

An analysis and data preparation was performed An analysis and data preparation was performed as in-house researchas in-house research– defining relevant relation types (about 15 main defining relevant relation types (about 15 main

relations and further ~20) (excerpts below)relations and further ~20) (excerpts below)– parsing IPC scheme, definitions, catchwords and RCLparsing IPC scheme, definitions, catchwords and RCL– building relation graph in RDBMS building relation graph in RDBMS

(>1.5 m relations)(>1.5 m relations)

The result is presented on a user interfaceThe result is presented on a user interface

Convertible to RDF or OWL for further use Convertible to RDF or OWL for further use

Patent taxonomy relations, samplesPatent taxonomy relations, samples

reference: reference: (transitive!)(transitive!)

A01B 1/00 Hand tools (edge trimmers for lawns A01G 3/06)A01B 1/00 Hand tools (edge trimmers for lawns A01G 3/06)A01G 3/06 Hand-held edge trimmers or shears for lawns (mowers A01G 3/06 Hand-held edge trimmers or shears for lawns (mowers combined with lawn edgers combined with lawn edgers A01D 43/16A01D 43/16))

precedence: precedence: (over 600 transitive cases, e.g A61M 3/00 (over 600 transitive cases, e.g A61M 3/00 A61M 5/00 A61M 5/00 A61M 36/00 [in definitions!])A61M 36/00 [in definitions!])

A01B 3/24 Tractor-drawn ploughs (A01B 3/04 takes precedence)A01B 3/24 Tractor-drawn ploughs (A01B 3/04 takes precedence)A01B 3/04 Animal-drawn ploughsA01B 3/04 Animal-drawn ploughs

limiting: limiting: A01N PRESERVATION OF BODIES…; BIOCIDES, e.g. AS A01N PRESERVATION OF BODIES…; BIOCIDES, e.g. AS DISINFECTANTS, AS PESTICIDES OR AS HERBICIDES; … DISINFECTANTS, AS PESTICIDES OR AS HERBICIDES; …

in Definitions for A01N subclass:in Definitions for A01N subclass:

Fungicidal, bactericidal, insecticidal, disinfecting or antiseptic paper D21HFungicidal, bactericidal, insecticidal, disinfecting or antiseptic paper D21H

Patent taxonomy relations, samplesPatent taxonomy relations, samples

indexing : indexing : guidance heading before A61K 101/00guidance heading before A61K 101/00

Indexing scheme associated with group A61K 51/00, relating to the nature of the Indexing scheme associated with group A61K 51/00, relating to the nature of the radioactive substanceradioactive substance

placerule : placerule : note before A01N 25/00, even specifying an exception…note before A01N 25/00, even specifying an exception…

In groups A01N 27/00-A01N 65/00, In groups A01N 27/00-A01N 65/00, in the absence of an indication to the contraryin the absence of an indication to the contrary, an , an active ingredient is classified in the last appropriate place.active ingredient is classified in the last appropriate place.

priorities (standardseq): priorities (standardseq): for main groups in IPC where no place rule is appliedfor main groups in IPC where no place rule is applied

cooccurrence: cooccurrence: e.g. in catchwords: also the text of IPC mentions the referencee.g. in catchwords: also the text of IPC mentions the reference

CONDITIONING harvested crops A01D 43/10, A01D 82/00CONDITIONING harvested crops A01D 43/10, A01D 82/00

A01D 43/10 with means for crushing or bruising the mown cropA01D 43/10 with means for crushing or bruising the mown crop

A01D 82/00 Crop conditioners, i.e. machines for crushing or bruising stalks (mowers A01D 82/00 Crop conditioners, i.e. machines for crushing or bruising stalks (mowers combined with means for crushing or bruising the mown crop combined with means for crushing or bruising the mown crop A01D 43/10A01D 43/10))

Presentation of IPCInfo / 2Presentation of IPCInfo / 2

Thank you…Thank you…

And keep reading if interested…And keep reading if interested…

FormalizationFormalization

With mathematical notationsWith mathematical notations

Targeted for audience not familiar with IPCTargeted for audience not familiar with IPC

The ‘patent’ The ‘patent’ (auto)categorization task(auto)categorization task

Regular multiclass hierarchic categorization taskRegular multiclass hierarchic categorization task– Given a hierarchic taxonomy (a patent classification) Given a hierarchic taxonomy (a patent classification)

with categorieswith categories– Given a set of training documents, each associated to Given a set of training documents, each associated to

multiple categoriesmultiple categories…or……or…an expert knowing both state of the art of the field and an expert knowing both state of the art of the field and the taxonomythe taxonomy

– For a document, provide a list of potential categories For a document, provide a list of potential categories (preferably with relevance)(preferably with relevance)

– Categorization level may be fixed (preclassification) or Categorization level may be fixed (preclassification) or fullfull

But…But…

The ‘patent’ The ‘patent’ (auto)categorization task, but…(auto)categorization task, but…

Really a regular multiclass hierarchic categorization task?Really a regular multiclass hierarchic categorization task?– Taxonomy: Taxonomy: texttext and and definitionsdefinitions (manuals or handbooks) and (manuals or handbooks) and revisionsrevisions, and , and

therefore:therefore:known known relationsrelations between categories (rules of classification, e.g. last place rule, takes between categories (rules of classification, e.g. last place rule, takes precedence)precedence)secondary categoriessecondary categories, non-primary categories (indexing codes, ‘not used as first , non-primary categories (indexing codes, ‘not used as first symbol’)symbol’)some some categories excluded for ‘final’categories excluded for ‘final’ categorization (top levels of the hierarchy) but categorization (top levels of the hierarchy) but required in preclassificationrequired in preclassification (where secondary categories cannot be used) (where secondary categories cannot be used)

– DocumentsDocumentscontain contain metadatametadata (priorities, inventor, applicant) (priorities, inventor, applicant)various various “fields”“fields” (title, abstract, description, claims) (title, abstract, description, claims)some fields are subject of some fields are subject of independent categorizationindependent categorization (claims), (claims), some fields may be use just globally (abstract, description)some fields may be use just globally (abstract, description)

– Changes: subject matter of a symbol, classification rules and proceduresChanges: subject matter of a symbol, classification rules and proceduresprovided categories may require revisions, provided categories may require revisions, since since taxonomy can be revisedtaxonomy can be revised in regular intervals or immediately in regular intervals or immediatelye.g. there is no more ‘main classification symbol’e.g. there is no more ‘main classification symbol’preclassification may help to reduce the scope but requires handling failurespreclassification may help to reduce the scope but requires handling failures

Notations: Hierarchic taxonomyNotations: Hierarchic taxonomy

Taxonomy: Taxonomy: TT Category: Category: CC, , supercategory: supercategory: ⊗⊗ ∉ ∉ C C Parent function: Parent function: pp: : CC→→CC ⋃⋃ ⊗⊗ function, function, describing a non-directed tree graphdescribing a non-directed tree graphAncestors: Ancestors: pp++: : CC→→CC++ ⋃ ⋃ , transitive closure of ⊗, transitive closure of ⊗ ppChild function (subcategories): Child function (subcategories): cc: : CC→→CC* = * = pp-1-1

Descendants: Descendants: cc++: : CC→→CC* transitive closure of c* transitive closure of c

Roots of taxonomy (‘sections’): Roots of taxonomy (‘sections’): CC⊗⊗ ⊂ ⊂ CC , ,CC⊗⊗ = { = {rr ∊ ∊ C C | | p(r)p(r) = = }⊗}⊗

Notations: Patent taxonomyNotations: Patent taxonomy

Level of category: Level of category: LL, , ll: : CC→→LL (e.g. ‘subclass’) (e.g. ‘subclass’)Classifying category level: Classifying category level: LLcc ⊂ ⊂ LL

Classifying category: Classifying category: CCcc ⊂ ⊂ CCCCcc = { = { cc ∊ ∊ CC: : l(c)l(c) ∊∊ LLcc } }Non-classifying category: Non-classifying category: CCcc‾ ⊂‾ ⊂ CCCCcc‾ = ‾ = CC ∖∖ CCcc

Category symbol: Category symbol: ss: : CC↔↔$$ (($$ stands for string) stands for string)Category sort relation: Category sort relation: cc11 < < cc22 ⇔ ⇔ ss((cc11) < ) < ss((cc22))also also minmin, , maxmax applicable for applicable for CC++ Category interval: [Category interval: [ff,,tt] = {] = {cc ∈ ∈ CC | | ff ≤ ≤ cc ∧∧ cc ≤ ≤ tt } }UsuallyUsually: descendants form a contiguous interval, i.e. : descendants form a contiguous interval, i.e. ∀ a ∀ a ∈ ∈ CC : : d d ∈∈ [ [ minmin((cc++(a)), (a)), maxmax((cc++(a))] (a))] ⇔⇔ d d ∈ ∈ cc++(a)(a)

Notations: category relationsNotations: category relations

Relation types: Relation types: R R ⊂⊂ ( (CC → ( → (℘℘((C) C) ∪∪ )⊗)⊗ ))All relations in a taxonomy: All relations in a taxonomy: TTRR ⊂ ⊂ CC ☓ ☓ CC ☓☓ RRAll relations for a category: All relations for a category: rr∀∀ : : CC → (→ (R R ☓☓ CC)*)*Obvious relation types in hierarchies: Obvious relation types in hierarchies: { { parentparent, , childchild, , ancestorancestor, , descendant descendant } } ⊂ ⊂ RRdefined as defined as parentparent ≈ ≈ pp, , childchild ≈ ≈ cc etc. etc.Further obvious relation: sibling (s), as child of parent (c)Further obvious relation: sibling (s), as child of parent (c)

Interval and set relations: union of the single-category form, e.g. Interval and set relations: union of the single-category form, e.g. descendantdescendant({c({c11,[c,[c22,c,c33]})]})result abbreviated as an interval or set: result abbreviated as an interval or set: descendantdescendant(a) = [(a) = [minmin(c(c++(a)),(a)),maxmax(c(c++(a))](a))]

Cpc

CC

ccscs

ccssc

}|))(({

}|{:)(sibling

Patent taxonomy relations Patent taxonomy relations on a single versionon a single version

Invertable relationsInvertable relations– Simple Simple referencereference: category ‘refers’ to another: category ‘refers’ to another– ‘‘Takes Takes precedenceprecedence’ reference’ reference– limitinglimiting references, very similar to references, very similar to precedenceprecedence– Allowed Allowed indexingindexing symbols on an interval symbols on an interval

Precedence relations on siblingsPrecedence relations on siblings– placeruleplacerule: first place rule or last place rule: first place rule or last place rule– prioritypriority: siblings prioritized by ‘standardized : siblings prioritized by ‘standardized

sequence’sequence’

cooccurrencecooccurrence of references (commutative) of references (commutative)

Patent taxonomy relations, samplesPatent taxonomy relations, samples

reference: reference: (may refer further!)(may refer further!)A01B 1/00 Hand tools (edge trimmers for lawns A01G 3/06)A01B 1/00 Hand tools (edge trimmers for lawns A01G 3/06)

A01G 3/06 Hand-held edge trimmers or shears for lawns (mowers A01G 3/06 Hand-held edge trimmers or shears for lawns (mowers combined with lawn edgers combined with lawn edgers A01D 43/16A01D 43/16))

precedence: precedence: A01B 3/24 Tractor-drawn ploughs (A01B 3/04 takes precedence)A01B 3/24 Tractor-drawn ploughs (A01B 3/04 takes precedence)

A01B 3/04 Animal-drawn ploughsA01B 3/04 Animal-drawn ploughs

limiting: limiting: A01N PRESERVATION OF BODIES…; BIOCIDES, e.g. AS A01N PRESERVATION OF BODIES…; BIOCIDES, e.g. AS DISINFECTANTS, AS PESTICIDES OR AS HERBICIDES; … DISINFECTANTS, AS PESTICIDES OR AS HERBICIDES; …

in Definitions for A01N subclass:in Definitions for A01N subclass:

Fungicidal, bactericidal, insecticidal, disinfecting or antiseptic paper D21HFungicidal, bactericidal, insecticidal, disinfecting or antiseptic paper D21H

Patent taxonomy relations, samplesPatent taxonomy relations, samples

indexing : indexing : guidance heading before A61K 101/00guidance heading before A61K 101/00

Indexing scheme associated with group A61K 51/00, relating to the nature of the Indexing scheme associated with group A61K 51/00, relating to the nature of the radioactive substanceradioactive substance

placerule : placerule : note before A01N 25/00, even specifying an exception…note before A01N 25/00, even specifying an exception…

In groups A01N 27/00-A01N 65/00, In groups A01N 27/00-A01N 65/00, in the absence of an indication to the contraryin the absence of an indication to the contrary, an , an active ingredient is classified in the last appropriate place.active ingredient is classified in the last appropriate place.

priorities (stand.seq.): priorities (stand.seq.): main groups in IPC where no place rule is appliedmain groups in IPC where no place rule is applied

cooccurrence: cooccurrence: in catchwords: also the text of IPC mentions the referencein catchwords: also the text of IPC mentions the reference

CONDITIONING harvested crops A01D 43/10, A01D 82/00CONDITIONING harvested crops A01D 43/10, A01D 82/00

A01D 43/10 with means for crushing or bruising the mown cropA01D 43/10 with means for crushing or bruising the mown crop

A01D 82/00 Crop conditioners, i.e. machines for crushing or bruising stalks (mowers A01D 82/00 Crop conditioners, i.e. machines for crushing or bruising stalks (mowers combined with means for crushing or bruising the mown crop combined with means for crushing or bruising the mown crop A01D 43/10A01D 43/10))

Patent taxonomy relations, Patent taxonomy relations, multiple versionsmultiple versions

Patent taxonomies change in timePatent taxonomies change in timeA former category (or a set) may be A former category (or a set) may be – transferred totransferred to a single or a set of new categories or, it a single or a set of new categories or, it

is recognized that the subject matter is is recognized that the subject matter is – covered bycovered by a single or set of existing categories a single or set of existing categories

In the newer version, all the categories which In the newer version, all the categories which are associated to a single or a set of former are associated to a single or a set of former categories, are in categories, are in concordanceconcordance relation relationconcordanceconcordance relation may be computed by relation may be computed by transitive traversing category changes over transitive traversing category changes over multiple versionsmultiple versions

Patent taxonomy relations:Patent taxonomy relations:concordance relation sampleconcordance relation sample

2011: B24B 49/002011: B24B 49/00

Measuring or gauging equipment for controlling the feed movement of the Measuring or gauging equipment for controlling the feed movement of the grinding tool or work; Arrangements of indicating or measuring equipment, grinding tool or work; Arrangements of indicating or measuring equipment, e.g. for indicating the start of the grinding operatione.g. for indicating the start of the grinding operation

2012: B24B 49/00 2012: B24B 49/00 B24B 37/005 - 37/015, B24B 49/00 B24B 37/005 - 37/015, B24B 49/00

B24B 37/005 . Control means for lapping machines or devicesB24B 37/005 . Control means for lapping machines or devicesB24B 37/013 . . Devices or means for detecting lapping completionB24B 37/013 . . Devices or means for detecting lapping completionB24B 37/015 . . Temperature controlB24B 37/015 . . Temperature control

B24B 49/00 Measuring or gauging equipment for controlling the feed B24B 49/00 Measuring or gauging equipment for controlling the feed movement of the grinding tool or work; Arrangements of indicating or movement of the grinding tool or work; Arrangements of indicating or measuring equipment, e.g. for indicating the start of the grinding operation measuring equipment, e.g. for indicating the start of the grinding operation ( B24B 33/06( B24B 33/06, B24B 37/005, B24B 37/005 takes precedence; if applicable to other takes precedence; if applicable to other machine tools, B23Q 15/00-B23Q 17/00 take precedence)machine tools, B23Q 15/00-B23Q 17/00 take precedence)

Effect of relations on categorizationEffect of relations on categorization

A weighted directed graph can be built between A weighted directed graph can be built between categoriescategoriesWhenever an ‘oracle’ (e.g. a flat categorizer, a Whenever an ‘oracle’ (e.g. a flat categorizer, a fielded search etc.) proposes a category, related fielded search etc.) proposes a category, related categories must be evaluated and verified, may categories must be evaluated and verified, may be, in a given order, considering also weightsbe, in a given order, considering also weightsTraining may also benefit from knowing, in Training may also benefit from knowing, in advanceadvance– order of evaluation, e.g. standardized sequences, order of evaluation, e.g. standardized sequences,

priority rulespriority rules– relations: relations:

to enhance a good hit or suppress a false hit to enhance a good hit or suppress a false hit or co-classifiyor co-classifiy