White Paper - Mine Planners Lie With Numbers

For more information;

Graham Lumley +61 412787920 [email protected]

MINE PLANNERS LIE WITH NUMBERS

By

Dr Graham Lumley BE(Min)Hons, MBA, DBA, FAUSIMM(CP), MMICA, MAICD, RPEQ

30 November 2011

Abstract

The most significant risk in developing a mine is that planner’s forecasts are not met. Cost and time

allowances are rarely met and returns on investment are lower than predicted in 80-90% of

developments. A primary input of this is budgeted output of the proposed equipment is not achieved.

Three causes are proposed for this; technical deficiencies in the planning process; planner’s optimism and

strategic misrepresentation (deliberate deception). Because the company’s balance sheet erodes every

day they operate there is pressure on the highest levels of our mining companies to convert deposits to

mines. Planners / consultants also have a vested interest in projects proceeding through the stages of

feasibility studies. It is hardly surprising that in-house planners and consultants make forecasts which

produce a result sufficient to justify the board approving the next stage of the development. This industry

in Australia uses the Valmin Code (2005) and the JORC Code (2004). The ASX also has their own rules for

listing. Each includes great detail on resource definition. However, they all pay scant regard (eg. The

Valmin Code includes four lines in a 20 page document) to equipment performance; which is one of the

primary drivers of the economics of a project, (converting a resource to a reserve). Mining companies

have tight guidelines on the resource but clearly, this is not the only potential area of error? Neither The

Valmin nor JORC Codes are protecting investors. They have simply shifted the source of error. The error

(deliberate or not) is explained and demonstrated through six case studies. A better approach is

recommended which includes better forecasting of equipment rates through benchmarking against

industry standards and more accountability. Not so many mines will be developed but investors and

shareholders will be better protected.



Introduction

The first recorded questioning of the ethics in mining or project development was penned in 1556 by

Agricola. He wrote, “A prudent owner, before he buys shares, ought to go to the mine and carefully

examine the nature of the vein, for it is very important that he should be on his guard lest fraudulent sellers

of shares should deceive him,” (Bullock, 2011,p78). 310 years later and it seemed nothing had changed.

Mark Twain was reported (unconfirmed) to have told an assembled group of miners at Red Dog,

California, in 1866, that “a mine is a hole in the ground owned by a liar,” (Bullock, 2011,p78) The alleged

comments were targeted at US gold mining interests and the speculation which followed its discovery in

1849. Blainey (1963) makes it very clear that the Australian mining industry has a questionable heritage in

the areas of honesty and integrity. He creates the picture of a burgeoning industry which needed an

increasing number of speculators and speculative money to grow. To attract those speculative dollars

directors, brokers (some of which are still listed as some of our most well-respected broking houses),

bankers, politicians, etc. needed volatility in share prices. They got that volatility occasionally through the

discovery of rich reefs but more often through a range of less ethical means.

So in 2012, 166 years after the genesis of the Australian mining industry in the copper-rich hills out from

Adelaide and the subsequent discovery of gold at Ophir in NSW in 1851, the question is posed; has

anything really changed? Is the need for money still driving less than ethical approaches to mine

developments?

It seems that Australia is not the only country and mining not the only industry grappling with this issue.

Flyvbjerg (2007) makes it clear that this problem transcends industries and countries. When referring to

mega infrastructure projects he quotes Wachs (1989), "planners lie with numbers." He states that

planners are busy not with getting forecasts right and following the appropriate Codes of Ethics but with

getting projects funded and built. According to Wachs (1989), accurate forecasts are often not an

effective means for achieving this objective. Flyvberg (2007, p.6) supports this notion when he states,

“….accurate forecasts may be counterproductive, whereas biased forecasts may be effective in competing

for funds and securing the go-ahead.”

While agreeing with the summation of outcomes, Merrow (2011) believes that project failure is usually

laid out early in a project's development through the basic business strategy and timetable for the project.

Not aligning planning properly and developing the basic technical data and analytics sufficiently leads to

not achieving forecast ROI.

Merrow, Wachs and Flyvbjerg all point to the same need. Whether poor outcomes are caused by error,

unintentional self-deception or deliberate deception, outcomes can be improved through structured

definition and disclosure in each stage in the development process.



In the 21st century, like their counterparts in the second half of the 19th century, many shareholders and

financiers of our mining companies are apparently not overly interested in the underlying resource nor the

mine; they are speculating in the volatility. To maintain volatility these companies must keep a flow of

“development” happening. From resource definition to upgrades to reserve calculations to conceptual

mine studies to prefeasibility studies to detailed feasibility studies to engineering studies, the flow of

progress must not be interrupted to maintain interest and volatility in the underlying company value. To

maintain a constant flow of funding and go-aheads to the next stage or interest in being taken over by a

big company it is essential that positive outcomes come at each stage. It is proposed that this encourages

low cost inputs and high benefit outcomes in the mining industry in project development planning. Having

said that, some of our miners do want openness, honestly, accountability and accuracy in their mine

developments to predict outcomes.

So the industry is left with two groups of miners; the profit-focussed, genuine businesses interested in

creating value through extracting value from the ground and the speculative business interested in

extracting value from people and companies with money. Both groups of miners need the money to keep

flowing.

Mining companies are producing a large number of projects which are not yielding the potential ROR

predicted by feasibility studies (once changes in commodity prices are factored in). Bullock (2011, p81)

summarises the reasons (provided by a range of authors) why mining developments are not reaching their

projected ROR into 16 areas. This paper will provide insights on two of the areas which have arguably the

most impact.

Overoptimistic mine design or productivity.

Overoptimistic mine development schedule and start-up (learning curve) time.

This paper develops the case for a standard to be applied to the use of equipment production rates in

mine plans and a system for accountability to be applied to mine developments and that this be

incorporated into the Valmin and JORC Codes and should be embraced by the ASX in their disclosure rules

for Mining Companies.

Mine Planning Process

To take a development from a resource to a reserve and finally a mine, three levels of feasibility study

(plus further engineering studies) are often required (Bullock, 2011). Each stage requires a different level

of detail and each requires the mine planner to make a range of assumptions around optimal output levels

and rates to achieve those levels. The average of 20 authors presented by Bullock (2011) is for the first

study to have an accuracy of +/- 35%; the second study +/-22% and the third to have +/-14%. There is

however, no prescribed engineering approach to move from one level to the next. Further, the author



questions how a reserve can be determined without accurate inputs into determining whether any

particular block is economic? When 80-90% of estimates of equipment production rates are more than

20% over what is actually achieved in the field (Lumley (2007), there is clearly no way the levels of

accuracy in the overall plan can be guaranteed both in output and ROI. Consequently, 80-90% of reserve

calculations are not supported by the engineering inputs being used to calculate them and should still be

called a resource.

This identifies a key problem in the mine planning process. There is no prescribed process nor standard to

change a resource to a reserve. The following is the definition of a Reserve (JORC Code, 2005, Clause 28)

“An ‘Ore Reserve’ is the economically mineable part of a Measured and/or Indicated

Mineral Resource. It includes diluting materials and allowances for losses, which

may occur when the material is mined. Appropriate assessments and studies have

been carried out, and include consideration of and modification by realistically

assumed mining, metallurgical, economic, marketing, legal, environmental, social

and governmental factors. These assessments demonstrate at the time of reporting

that extraction could reasonably be justified. Ore Reserves are sub-divided in order

of increasing confidence into Probable Ore Reserves and Proved Ore Reserves”.

Any optimisation algorithm (eg. Lerchs-Grossman) will provide no help in the determination of economic

reserves nor cutoff grades nor the final shape of the mine if inaccurate inputs are provided on equipment

rates. For example, the economics of a block of ore might change dramatically if eight trucks are needed

with a loader rather than six. Similarly, if your Liebherr 996 loader is scheduled to move 32 Mt per annum

(actual best practice for this machine in 2010 – GBI, 2011) and it actually achieves 22 Mt per annum

(actual average for this machine in 2010 – GBI, 2011), then the economics of this block of ore changes

dramatically by virtue of the substantial increase in the time taken to mine it and the number of trucks

which can be serviced by this loader in a unit of time.

The mining plan requires a range of inputs as detailed in Figure 1 (taken from Beniscelli, et al, 2000). Most

of those inputs revolve around production rates and costs and can be subjected to error from incorrect

inputs. Production rates and costs are the real keys to the DCF or ROR analysis but are normally done with

minimal input apart from the potentially subjective opinion of the person doing the planning, (Lumley and

Beckman 2009).



Figure 1. Elements of the Mining Plan (taken from Beniscelli et al 2000)

Industry standards (actual operating results) are available in great detail from mines around the world,

(GBI, 2011). The question is posed, “Why do people not use industry standards when completing mine

development plans?”

Feasibility Study Data Input

Geological information from exploration is utilised to a great extent in planning, (Lumley & Beckman,

2009) and this comes from the stringent requirements of the JORC Code and ASX listing rules. Mining

software has clear links between geologic models and planning modules. One only needs to observe the

Bowen Basin coal mines with draglines completely below the surface or the massive copper mines in Chile

to see that the geometry of excavations can be incredibly complex.

Accurate inputs are recognised as paramount and the Valmin Code, JORC Code and ASX listing rules

attempt to address this need. Much of the focus has been in tightening the specifications of resources and

reserves, ie. the geologic inputs. However, this improvement of the geological inputs is difficult and adds

significant cost to a mine development. Drilling programmes are high cost, have high lead times, and have

diminishing returns. While the codes’ requirements for geological, processing and cost inputs are high, the

same cannot be said for mining inputs, ie. the way a deposit will be mined and the rates predicted. The

codes are very quiet on this issue.

Case Study 1.

GBI (2011b) demonstrated the performance of P&H4100XPC shovels in the northern Bowen Basin. Best

practice was 17.9MBCM per annum and median 14.1MBCM per annum. The project team was under

pressure from Executive Management to budget 25 MBCM per annum because in their opinion, “That is

what that model is capable of.” The GBI database indicates the P&H4100XPC shovel is capable of moving



25MBCM per annum, however, only one machine in 40 from around the world will achieve this level and

none from the northern Bowen Basin. The use of 25MBCM in their development models will make a huge

difference in terms of predicted ROR but is most likely going to end in the company not meeting their

forecasts for the proposed development.

Neither Valmin nor JORC Codes offer protection to financiers nor shareholders in this case.

Examples of Poor Data Input

Flyvbjerg (2007) presents average errors between planned events / costs and the actual outcomes, and a

study of the reasons. The findings are unequivocal and are in line with the author’s findings for the mining

industry. 80-90% of projects will exceed budget cost and won’t deliver the benefits. He presents by way of

example the English Channel Tunnel which cost twice the budget, delivered half the benefit and has had a

negative rate of return. In other words, the British economy would have been better off if it had not been

built.

Lumley and Beckman (2009) outlined a number of sources of error in mine planning inputs; both for

production rates and costs. The following summarise the examples of what has led to overly ambitious

production rates being used;

Dig depths and face height impact on productivity not considered,

Variation in seam dip not considered,

Planning done in 2D and then merged to 3D,

Scheduling using maximum potential rate for KPI’s and productivity rather than what can be

achieved over a longer period (sustainable rate),

Scaling performance from equipment of different capacity,

Overestimating hours of work,

Not considering fleet interactions,

Not understanding operational limitations, eg. Double side loading vs single side loading

Manipulating densities and bucket fill

Assuming every truck is full

Etc. etc

Case Study 2.

A preferred contractor, quoting for overburden removal, used rates, shown by a benchmark against

industry standards, to be in the 95th percentile for all equipment of the same make and model. Selection



of a mining contractor will almost always be based on cost and rates achieved so the over-inflating of rates

is a common occurrence (and a common source of dispute between mining company and contractor).

Once the contract is underway it is difficult to terminate a contract due to dispute over causes of

underperformance and consequently, mines accept claims for higher payments or less output. In this case

the preferred contractor’s quote was subsequently rejected.

The Valmin Code (2005) & JORC Code (2004)

The impacts of not using good data inputs are many. Most serious of these impacts are the effects on

margin ranking to establish reserve estimates (transitioned from resources) and cashflow estimates to

value the mine operation. (Lumley & Beckman 2009).

The Valmin Code (2005) is described as the “Code for the Assessment and Valuation of Mineral and

Petroleum Assets and Securities for Independent Expert Reports”. The Securities Institute of Australia

(AUSIMM, 2005) describes it as, “….indicative of best practice for independent experts preparing

valuations and assessments in relation to specialist mining reports.” It is the standard for protecting

investors and financiers in mining and petroleum developments. It is however, remarkably quiet on the

issues of equipment performance and how this should be handled. The consequences on mine planning

outcomes of not having clear standards in the critical areas under address in this paper suggest a lack of

understanding within the industry of the significant implications. The code itself appears to have a heavy

bias towards geology, resources and the processing of the ore itself. Multiple sections are provided in

these areas plus other areas such as cost estimates and risk. The area of equipment rates and

performance is covered in the broadest way under one point of the six sub-sections in the Mining and

Processing Section. In Section 83, (p15), it states, “Existing and/or proposed mining and process plant

practices should be reviewed to establish the technical and economic feasibility of the operation under

consideration at its existing and/or proposed scales.” As a sub point of this the following are included;

h) labour sources, requirements and productivity;

i) operating practices;

j) equipment availability, utilisation and performance;

The JORC Code (2004) is described as, “The Australasian Code for Reporting of Exploration Results,

Mineral Resources and Ore Reserves”. It sets out a principles-based approach to minimum standards,

recommendations and guidelines for public reporting of exploration results, mineral resources and ore

reserves. The JORC Code says more than the Valmin Code on mining issues because these directly impact

the conversion of what is in the ground from a resource to a reserve. The JORC Code however, is again



heavily biased to geologic issues. Mine planning issues are mentioned but nothing which might force a

planner to be accountable for their decisions on equipment rates. The document says, “….the extraction of

the ore reserve has been demonstrated to be viable under reasonable financial assumptions.” And further,

“….appropriate studies will have been carried out prior to determination of the ore reserves. The studies

will have determined a mine plan that is technically achievable and economically viable…….”

The focus on geological and processing is demonstrated further in Table 1 under the heading “Estimation

and Reporting of Ore Reserves (pp17-18). It is the conversion of the resource to the reserve where

significant input on the proposed mining operation (called the mining factors) should be provided. The

guidelines are broad and demonstrate a lack of appreciation of the impact of factors such as equipment

rates have on the determination of economic reserves. In leaving it broad the JORC Code follows the

“principles-based” approach. There is no better demonstration of the failure of the principles-based

approach than the last explanation point under the criteria “Discussion of relative accuracy / confidence,”

(p. 18). That point states, “These statements of relative accuracy and confidence of the estimate should be

compared with production data, where available.” Projects representing 3% of the total mining projects

under development in Australia have accessed relevant operating, performance data to guide their mine

plans and the determination of economic reserves. Clearly, the requirements of this clause are being

ignored by the majority of “competent persons”.

This author would contend that the JORC Code, the Valmin Code and the ASX Listing Rules are failing those

people it is meant to support by virtue of it being principles-based rather than prescriptive. The target

audience for the competent person’s report is not well enough educated in the details of the mining

industry to recognise the avoidance of some key issues and deliberate deception in other areas.

Bullock (2011, p85) supports this assessment of under-representation in mining inputs when he asks,

“Why has such a tremendous effort been put forth to greatly improve the quality

and standards of the resource and reserve classifications, but with little or no

effort to improve the detailed definition of that which determines whether or not a

resource will move from a resource to a reserve classification? Does the industry

really believe that unethical practices of project feasibility studies can only come

through misrepresentation of resources?”

Beckman (pers. comm. 2011) advises that in equipment selections for new or existing projects a great deal

of effort is directed towards optimising the understanding of geology and costing predictions but the mine



plan (in the area of equipment performance) is taken as provided. The opinion of the planning engineer,

who might be an expert at mine planning but not in equipment rates is taken as the required standard of

input.

GBI (2007) shows that most mine planners are producing plans which simply don’t produce the outcomes

predicted and clearly under this scenario, the shortcomings of the relevant codes and rules should be

addressed.

Case Study 3.

The following is from a recent due diligence of a mine plan for a mine expansion. The mine plan used 18

Mt per annum as the output for Hitachi EX5500 hydraulic excavators. This assumption was not tested

against industry standards. It was simply accepted as input into the mine design. The median annual

output of this model was 13.5 Mt in 2010, (GBI, 2011). Consider also, that 50% of the people using this

make and model will be less than 13.5 Mt per annum. In fact only 1 in 5 users of this make and model will

achieve the specified rate. The implications of a significant shortfall for this and other site equipment

would likely be financially devastating to a range of people investing in or financing the project.

Neither Valmin nor JORC Codes offered protection in this case.

Case Study 4.

A copper mine in Zambia failed to meet forecast performance over a number of years after

commissioning. It was described as a “ramping up in performance” but when the technical documentation

is investigated the mine was planned to be running at full production rates by 2009 but continues to the

present at equipment rates well below plan. To meet budget total output they have employed additional

equipment. It has been reported that their difficulties are with operating hours and operating rate. It is

known that rates are well below worldwide average for the equipment they are using. This is partly

understandable, given the fact that the mine has used a largely unskilled workforce being trained from no

experience. In this situation, a significant ramp-up period is expected. The author (who was not involved in

the planning nor provided data to this development) can only surmise how such optimistic rates were

used in the planning process. The real issue is that the mine will exceed the input costs that were used in

the feasibility studies partly because the planning has used optimistic and probably unrealistic

assumptions. The planner (a large, well known consulting company) has never been held accountable.



The Valmin Code and JORC Code (or the South African equivalents – SAMVAL/SAMREC) did not offer

protection in this case despite being largely planned post-2004 when the requirements for reserves

reporting were upgraded.

Why are Inputs into Mine Developments Wrong

The literature provides three explanations for inaccurate inputs into mine plans; technical (inadequate or

poor data and models), psychological issues (planner’s optimism) and political-economic issues (deliberate

misrepresentation) (Flyvbrerg, 2007). Of these, technical reasons are most often proposed as the

dominant cause, ie. errors in data and forecasting methods (Vanston and Vanston, 2004, p33).

In the case of mining data this can be seen in the following areas;

Too much data

Companies are data rich and information poor. This is because the systems for storing and accessing the

data have been developed by people who understand numbers but not what numbers may be useful.

Poor quality / unreliable data

This is a significant inhibitor to business improvement. Data quality is the foundation for high value-

generating data assets.

Outdated data

Older data tends to have significant issues with accuracy and also relevance.

Poor quality analysis / forecasting

The analysis of mining performance and costing is a specialised activity and should only be done by

someone with expertise in the dual areas of data analysis and mining knowledge. This may or may not

reside within the company. An example of this is with dragline data. Because of the unique arrangement

of a bucket being controlled by two sets of ropes there are a range of dependencies between various kpi’s.

If these dependencies are not accounted for, eg. kpi’s simply averaged in a specified time vs a different

time, then the result is subject to error.

These sources of error in mining data and analysis may be reduced or eliminated by developing better

forecasting models, better data and more experienced planners (Lumley & Beckman, 2009). However, if

technical issues were the dominant or even a significant contributor, the distribution of errors would be



normal (or near normal) and the average error would be near zero. Anecdotally this is not the case. The

lack of impact of technical errors has also been proven empirically by Flyvbjerg (2008). Flyvbjerg (2008,

p27) states, “ technical explanations must be rejected because they do not fit the data.”

Psychological issues account for inaccuracy in terms of optimism bias or planner’s bias (Lovallo and

Kahneman, 2003). In the mining industry this is often called “planner’s optimism”. It is a cognitive

predisposition (unintentional self-deception) to judge future events in a more positive light than is

warranted by actual experience. It is a weakness in the way the human mind processes information and is

thought to be a universal problem (Lovallo and Kahneman, 2003).

If the primary cause of failure to meet ROR on mining projects was unintentional planner’s optimism then

this could be predicted and standards established for developments along those proposed by HM

Treasury, (2003) for large scale infrastructure projects in Britain. They recommended that adjustments be

made to a project’s cost, benefits and duration, and that the adjustments be based on data from past or

similar projects elsewhere. Again, the lack of normality and consistency in the inaccuracies in project

outcomes indicates that planner’s optimism is not a significant cause of the inaccuracy (Bullock, 2011).

The third explanation for inaccuracies in project outcomes is political-economic issues. They explain

inaccuracy in terms of strategic misrepresentation, which Wachs (1990) puts bluntly as intentional

deception. In this case, when forecasting the outcomes of projects, forecasters and planners deliberately

and strategically overestimate benefits and underestimate costs in order to increase the likelihood that it

is their project and not the competition’s, that gains approval and funding. Strategic misrepresentation

can be traced to internal organisational pressures.

Wachs, (1990), suggests that in the early stages (project approval) of a development plan there are strong

interests and incentives to emphasise benefits and de-emphasise costs and risks. Development teams and

consultants need a pipeline of future work. Mining companies need future mines. Wachs, (1990), finds

that this is not predominantly caused by non-intentional technical error nor planner’s optimism (which is

also non-intentional). He states that it is deliberate misrepresentation which can be represented by the

following Machiavellian formula.

(Underestimated costs) + (Overstated benefits) = (Project Approval)



The Machiavellian way prospers in an environment of low accountability. This environment of low

accountability is supported by the principles-based JORC Code. Consequently when the industry tightened

up the resource definitions and standards through the JORC in 2004, the deception didn’t stop; it just

shifted.

The need for the industry to access (speculative) funds is driving this problem. Companies and mines

compete for those funds. Consequently, the better a project looks the more likely will be its success in

attracting those funds. The low accountability provided by the principles-based JORC Code (2004) is

fuelling this reverse-Darwinism; “survival of the unfittest”, (Flyvbjerg, 2008) outcomes for our mining

developments.

Case Study 5.

The following is a very simple but common deception perpetrated on higher level management in Coal

Mines. The output of mining equipment in the Australian coal mines is described in terms of Bank Cubic

Metres. Equipment monitors measure tonnes and numbers of passes, loads, cycles, etc. Total output is

tonnes per cycle multiplied by the number of cycles. To convert tonnes moved to BCM’s one must divide

the tonnes by the in-situ SG. The output of a mine’s stripping fleet can and is being manipulated by the

choice of in-situ S.G.’s. The use of a smaller S.G increases reported output in BCM’s. Conversely, in the

development phase, the use of a higher SG means less tonnes to move and the required specification of

trucks and loaders (and/or numbers of pieces of equipment) can be lower. Further to this the changing of

SG over time in planning of new mines and in existing mines has been used to create an illusion of

improving outcomes. An unnamed mine in Central Queensland and its Mine Manager received kudos for

increasing mine overburden output. An investigation of the performance demonstrated over a 3 year

period the in-situ SG had reduced from 2.40 t/CuM to 2.08 t/CuM.

Case Study 5.

A new technology, implemented in the Bowen Basin, was justified on assumptions which led to an

indicated 25% increase in productivity. Many millions of dollars were invested in machine modifications

and hundreds of millions of dollars were lost through coal not available to be sold. The trial results were

presented by the marketer of the technology (someone who had a direct financial interest in the further

rollout of the technology) and the mine where the trial was held. The company which owned the mine

also had a direct financial interest in the technology through a shareholding in the company which owned

the IP. The industry was told productivity had improved by 28%. Verbal feedback provided by the

operators suggested it was not that good. The author analysed the data and found actual productivity had



dropped by 14%. Apart from filtering, clustering and normalising done in the guise of “comparing like with

like” there was also a simple 6% error in the calculations which interestingly pushed the result from below

to above the level used for justification. An observer would not find a conclusion of deliberate deception

too hard to arrive at (but maybe difficult to prove).

It needs to be emphasised that this paper is directed at a portion of the Australian Mining Industry where

deception is being perpetrated at single or multiple levels. Some Boards of Directors are complicit. Some

Executive Management are guilty. Some development teams and consultants are supporting them.

However, some mining houses do conduct rigorous procedures and insist on honesty to ensure their

projects are appropriately ranked for development. Unfortunately, they are also subject to deception by

people with interests not tied to the financial return from the resource in the ground but rather in the

development process. The result is that Boards of Directors, financiers and shareholders cannot trust

information provided to them about mine developments. There is a strong need to establish incentives

and methods that produce more reliable information for the benefit of those providing money for

developments.

A better way

It is clear that many mine developments are proceeding based on doubtful engineering and reform is

needed. This has been recognised by the ASX (2011) and JORC (2011). It is a macro issue but comprises a

series of micro problems. In this paper the focus is on one of those micro areas; the use of accurate

equipment rates in mine plans. Until the Valmin Code, JORC Code and the ASX Listing Rules address the

issue of engineering inputs they will remain fatally flawed and won’t achieve their noble aims. Less error

(deliberate or other) and more accountability are needed in the estimation of prospective mine returns.

Two key inputs are recommended to achieve this;

1. Better forecasting of equipment rates through benchmarking against industry standards, and

2. More accountability of engineering input

The first of these two issues is described in European literature as “reference class forecasting” (Flyvbjerg,

2007 & 2008). In the mining industry it is called “benchmarking against industry standards” (Lumley,

2007). This involves taking an “outside view” (Flyvbjerg, 2008) on the particular aspect, (In this case

equipment performance rates). In the mining industry a “benchmark” is based on a population of similar

machines. This similar population may reflect, make and model of equipment, commodity, geographic

location, diggability, pit layout, etc. This should not try to forecast what events will impact a particular



piece of equipment but should rather attempt to place the piece of equipment on a statistical distribution

of current outcomes for the population.

Figure 2 and Figure 3 are two examples of how this data may be presented.

Figure 2. Tabular presentation of benchmark data for a mining truck, in Queensland coal mines,

with 242 tonne nominal payload with a flat, 13km cycle distance



0

200,000

400,000

600,000

800,000

1,000,000

1,200,000

1,400,000

1,600,000

1,800,000

2,000,000

0 10 20 30 40 50 60 70 80 90 100

An

nu

al T

on

ne

s C

arri

ed

Percentile

Mining Truck 242 tonne Nominal Payload, 13km Cycle Distance

4th Quartile 3rd Quartile 2nd Quartile 1st Quartile

Be

st P

ract

ice

Figure 3. Graphical presentation of benchmark data for a Mining Truck with 242 tonne nominal

payload with a flat, 13km cycle distance

The planners, in conjunction with the owners, must develop a rationale for likely performance. It is the

author’s experience that nearly all mine people believe they can perform in the 75th percentile or above

and this is where forecasts are often targeted. Logically, only 25% of mines using a particular make and

model of equipment achieve this rate. This should not provide comfort to financiers and shareholders.

Case Study 6.

A new hydraulic excavator was being purchased for a mine. A distribution table similar to Table 3 was

produced for the make and model, in the same geographic location for the same commodity. That is, the

population was about as tight as could reasonably be achieved. The Mine Manager subsequently insisted

rates in the 90th percentile be used in planning for the new piece of equipment (because that is what he

“knew” it achieved from his experience). An analysis of the mine’s other equipment showed that they

operated in the 42nd percentile on average. The difference was in excess of 10 Mt per annum. This has a

substantial impact on how much of the commodity is available to be sold. The piece of equipment actually

performed in the 58th percentile in the first year of operation due to additional focus on it but it never

came near the 90th percentile.



If the population can be accurately defined and sufficient units are in the population to provide a

statistically relevant outcome then it is strongly recommended that no forecast above the median be

used.

On the issue of accountability it must be understood that planners, as a way of reinforcing their forward

order book or their future job security, may not be focused on “getting it right” but rather “getting it

funded”. As defined by the previous Machiavellian formula, accurate forecasts are often not an effective

means of achieving the objective, (Wachs, 1990). Accountability will only take hold when decision-makers

can change the power relationships that govern project development. Given the pressure on Boards of

Directors to replace depleting assets this may need to be forced by shareholders, stock exchanges and/or

regulators. To achieve accountability in the area of equipment production rates (and for many other areas

in the development process) the following guidelines are proposed;

Forecasts should be benchmarked against a relevant population.

Forecasts should only target median performance of the relevant population

Forecasts should be made subject to independent due diligence.

Forecasts, due diligence and benchmarking reports should be made available to all stakeholders.

Conferences should be organised where all stakeholders can scrutinise and question forecasts and

planners.

Professional penalties should be enforced for planners who consistently produce deceptive forecasts.

Planners and their organisations should share financial responsibility for covering cost overruns and

benefit shortfalls resulting from misrepresentation and bias in forecasting. Boards of Directors should

have recourse to external consultants who provide erroneous inputs which cause financial loss. How

they handle internal planners who make errors (deliberate or otherwise) is up to them.

The fiduciary responsibilities of Boards of Directors should be extended to project development.

A better future

Fortunately, pockets of improvement have recently emerged in the area of inputs into equipment rates

used in large developments and the subsequent accuracy of forecasts regarding mine development cost

and benefits.

A large coal mining company has over the last 18 months accessed benchmark data for use as

inputs into their mine plans for two new mines and two existing mine expansions. They received

data on up to 20 different makes and models for each project on own company performance,



Queensland performance and worldwide coal performance. The populations were sufficient to

provide statistical significance for the results provided. They also conduct annual benchmarking on

their draglines, as well as some loaders and trucks across their mines. Consequently, they have

data on their performance and have knowledge of their current percentile rank for each make and

model when compared to specified populations. This percentile rank can be applied to the

benchmark data to provide valid inputs for mine plans.

A large engineering company recently included a requirement in a contract with a specialist mine

planner that the rates used in the mine plan be benchmarked against similar operations. The

definition of the population was given great attention.

In addition, an iron ore miner accessed benchmark information for equipment they didn’t have

currently operating as part of their iron ore expansion in the Pilbara. Some of this data was for

internal purposes and some was provided to an external planner for use in the mine planning.

A small resource company conducting a DFS accessed actual results for a particular class of

excavator for a proposed mine.

While being a significant development in this critical area, it is only the tip of the iceberg in developments

in the Australian mining industry.

Conclusion

The Valmin Code (2005) is, “….indicative of best practice for independent experts preparing valuations and

assessments in relation to specialist mining reports.” It is the standard for protecting investors in mining

and petroleum developments. The JORC Code (2004) sets out minimum standards, recommendations and

guidelines for public reporting in Australasia of exploration results, mineral resources and ore reserves.

The ASX Listing Rules provide a more prescriptive approach for defining resources and reserves. They have

all failed to adequately protect shareholders, investors and financiers of mine developments. The source

of error (deliberate or otherwise) has shifted from the definition standards of resources to the processes

in shifting the mineral resources to ore reserves, ie. the engineering inputs used to define what is

economic. Because of the shortcomings in the Codes, regulators have been unable to keep mining

companies and planners accountable for the outcomes of their developments.



Mining industry feasibility and engineering studies must be improved. In Australia it must fall to the

Valmin and JORC Codes, and the ASX regulations to offer more than a few sentences of guidance when it

comes to anything apart from reserve definition, cost and processing issues. Why are there no specific

guidelines nor standards for issues such as equipment rates?

Failure to achieve planned ROR in mine developments have been proposed in three areas; technical

inaccuracies, planner’s optimism and strategic (deliberate) misrepresentation. The first two have been

shown to be real but not significant sources of error; while deliberate deception has been shown to be rife

across a range of industries.

This paper has presented a proposed two pronged approach to address this issue in the area of equipment

performance inputs into feasibility studies. This approach is;

1. Better forecasting of equipment rates through benchmarking against industry standards, and

2. More accountability

They are linked and proper implementation of them will address the losses which many shareholders and

financiers have incurred when investing in mining companies.

To achieve accountability in the area of equipment production rates (and for many other areas in the

development process) a range of recommendations have been made to protect shareholders and

financiers of mining projects

A series of statements should be required in all reports on developments outlining how reference class

forecasting has been implemented in the particular study. In the area of mining inputs / equipment rates a

statement along the lines of “Equipment rates used in this feasibility study conform to the XYZ standard.

Median performance is used based on (number – suggested minimum 15) other machines of the same

makes and models from the same commodity in the same geographic location”. And of course full details

must be provided as an appendix. Processing plant performance, costs, manning levels, timing, etc., even

plan outcomes (actual vs plan) should be forecast using a defined reference class and where a reference

class cannot be defined then this should be stated. Only then can shareholders and financiers increase

their confidence.



References

ASX, 2011, ASX Listing Rules Review Issues Paper: Reserves and Resources Disclosure Rules for Mining and

Oil and Gas Companies, ASX Public Consultation.

Beniscelli, J, Carrasco, P, Dowd, PPA, Ferguson, G, & Tulcanaza, E 2000, Estimation of Resources and

Conversion to Reserves — Protocols for the Assessment, Reduction and Management of Risk, In Proc.

Mass Min 2000, Brisbane, Australia.

Blainey, G, 1963, The Rush That Never Ended. A History of Australian Mining, 1st Edition, Melbourne

University Press, Parkville, Victoria.

Bullock, R, 2011, Accuracy of feasibility study evaluations would improve accountability, Jackling Lecture,

Society of Mining Engineers, April.

Flyvbjerg, B, 2007, Truth and Lies About Megaprojects. Faculty of Technology, Policy, and Management,

Delft University of Technology, September.

Flyvbjerg, B, 2008, Curbing Optimism Bias and Strategic Misrepresentation in Planning: Reference Class

Forecasting in Practice, European Planning Studies, 16:1, pp. 3 – 21

GBI Productivity Datawarehouse, 2011a

GBI, 2011b, Private report for mining company.

Henning, J, Kaiser, PK, Cotesta, L & Dasys, A, 2002, Innovations in mine planning and design utilizing

collaborative immersive virtual reality, 104th CIM Annual General Meeting, Vancouver, Canada

HM Treasury, 2003, The Green Book: Appraisal and Evaluation in Central Government, Treasury Guidance

(London: TSO)

JORC, 2004, The JORC Code, Australasian Code for Reporting of Exploration Results, Mineral Resources

and Ore Reserves

JORC, 2011, JORC Code Review, Issues Paper

Lovallo, D, & Kahneman, D, 2003, Delusions of Success: How Optimism Undermines Executives' Decisions,

Harvard Business Review, July, pp 56-63

Lumley, G, 2007, Improving Dragline Operator Selection And Support Processes, University of the Sunshine

Coast, Thesis submitted for the degree Doctor of Business Administration.

Lumley, G, & Beckman, R, 2009, Is technology helping get mine plans right? Australian Mining Technology

Conference 2009, 27 - 28 October 2009, in Brisbane, Australia

Merrow, E, 2011, Industrial Megaprojects : Concepts, Strategies, and Practices for Success, John Wiley

and Sons, Hoboken, New Jersey

Valmin Committee, 2005, The Valmin Code, Code for the Technical Assessment and Valuation of Mineral

and Petroleum Assets and Securities for Independent Expert Reports.



Vanston, J, and Vanston, K, 2004, Testing the Tea Leaves: Evaluating the Validity of Forecasts, Research-

Technology Management, vol. 47, no. 5, pp 33-39

Wachs, M, 1989, When planners lie with numbers, Journal of the American Planning Association, 55(4),

pp. 476–479.

Wachs, M, 1990, Ethics and Advocacy in Forecasting for Public Policy, Business and Professional Ethics

Journal, 9(1-2), pp 141-157

Documents

White Paper - Mine Planners Lie With Numbers