Embed Size (px)
Citation preview
Washington, DC (866) 945-4462
1650 Tysons Boulevard
Suite 720 McLean, VA 22102
New York (866) 945-4462
1501 Broadway 12th Floor
New York,
NY 10036
www.crosscountry-consulting.com
Project Background
Ok, well not really. As on-premise ERP upgrades go, this
one went fairly well. The company is a mid-sized
enterprise with a moderate level of system customization.
Compared to other upgrades (and I have done a lot of
them), this one was in the middle in terms of complexity
and risk.
In addition to the ERP software upgrade, the company
had to upgrade their database to the latest version, but
they were assured by the ERP software vendor that this
was not a big deal and that the application software
would work flawlessly with the new database version.
Since the database vendor and the ERP software vendor
were one in the same, what could possibly go wrong?
The upgrade project lasted approximately six months.
The project was well staffed with functional and technical
experts and an engaged user base. We had an active
steering committee comprised of executive stakeholders
from across the company. The project ran like an
upgrade project should as we hit key milestones for
development, testing and training. We stress tested the
application with production data and compared the
output against actual results in the old release. Overall,
the project was going well.
Four Uh-Ohs
So why did the project sponsor, who was new to the world of
on-premise ERP, say “I never want to do another ERP upgrade
project ever again”. There were four main reasons:
Mysterious Performance Issues.
We tested the application at production level volumes for
all major interfaces. In the test environment, processes
ran at or better than the production run times. We were
pleased. After go live, key processes that ran in minutes
on the old version were running for hours (and hours and
hours). What happened? It worked in test. It worked
previously, and we hadn’t changed any code. We called
our talented Database Administrator (DBA) in to help
resolve the issue. Then we waited. The DBA worked
closely with the ERP software/database vendor, as well
as the network and server administration teams. He
reviewed logs and cases. He was stuck and so were we.
We had to close the month, and we were stymied
because somewhere deep in our technology stack
something was not configured correctly. After nine days
of discovery, someone suggested that we try and switch
the Dirty Data Trust Factor Switch (ok, that was not the
real name, but it was something equally ridiculous) from
on to off. And ta-da – it worked! Why it worked, we did
not know, but we took it as a win and moved on. The
project sponsor asked naively, but insightfully, “Why
would we ever need to know what this switch does?”
Good question – it should have all just worked, right?
The Long Weekend of Angst and Hope.
The project ran for ten months. We all worked very hard
to understand the new features and how they applied to
our business. We re-engineered the chart of accounts,
An On-Premise ERP Upgrade Horror Story (and a permanent solution!).
1
2
cleaned up our vendor master and re-designed security.
We streamlined key processes. We trained our users,
and they tested the heck out of it. Then came the go live
weekend. We were all now reliant on our trusty upgrade
administrator. He is a talented fellow and worked hard
throughout the project, but now we handed off the
upgrade to him, entrusting him with 120 hours of steps,
tasks and scripts that needed to be completed. We set
up a war room and had everyone on call. Then we
waited. The 120 hour plan was going well until about
hour 68. That is when the train went off the track. The
process steps that ran in minutes were now running in
hours (sound familiar?). Our administrator tried to explain
what was happening, but there was not much we could
do, aside from wait and watch the processes creep
along. As the processes ran longer, we started to get
paranoid – the process finished, but did it work? We
started QA-ing the scripts, and it threw out tons of false
positive errors as the rest of the scripts had not yet been
run. Did we have a problem or not? The upgrade scripts
are long and complex, but what else could we do at this
point? We decided to move on and run the rest of the
scripts. We ended up finishing the upgrade at hour 114,
eating into every bit of buffer that was built into the plan,
but still going live on Monday morning. Our heroic
upgrade admin made it through the weekend but was
visibly exhausted. The project sponsor quietly noted,
“Wow. We would have been in serious trouble without
that guy.”
Good point, why does it take so much effort, specific
expertise and faith to upgrade your financial applications?
Lots of Work and No Real Value Delivered.
During the upgrade, our sponsor noted that so many of
the company’s resources were working on the project.
Some of the resources were dedicated full time. Others
had to work additional hours or defer other work to be
trained in and test every single part of the application. We
had hundreds of issues related to the upgrade scripts,
the customizations, the new features, the interfaces and
of course the performance. This was a lot of work. This
was a major allocation of time and effort at the cost of
other possible activities. About half way through the
upgrade project, our sponsor asked about the value we
were gaining from the upgrade. Although there were
some noticeable benefits in several process areas, like
collections, most processes would remain pretty much
the same. This legacy on-premise ERP had been around
for decades, and the upgrades became less and less
impactful in terms of delivering new value. The project
sponsor once again asked an innocent, but spot on
question that all CFOs and CIOs should be asking, “So
why are we doing this project?”
The answer was simple, though unsatisfying – because
we have to keep up with the ERP software vendor’s
release to stay on support.
Is this the latest version of the software?
As mentioned previously, the application spanned not
only finance, but hundreds of other casual users of the
application. We were doing one of our training sessions
for these users, and it was going well. We covered the
new process and what was changing. We did live demon-
strations. We fielded many questions and everyone
seemed content. At the end of the session, one user
raised his hand. He thanked us for the training and
apologized for the question he was about to ask – “Is this
the latest version of the software?” Everyone chuckled,
and I assured him it was. He accepted the answer and
moved on. But why did he ask the question in the first
place? He was asking the question as a consumer not a
transaction processor. He wanted the application to
look, feel and interact with him like other applications do
in his daily life. He wanted it to be easy to use and
accessible not only from his desktop but on his mobile
device. The on-premise ERP software vendors seldom
deliver ground breaking innovation in their releases. They
are typically just adding a few bells and whistles to
millions of lines of code that have been developed over
the past three decades.
The reality of on-premise old ERP is that they have a hard
time innovating. Their code is complex and built on an
“upgrade” approach that just adds fixes on top of fixes on
top of fixes. They can’t move fast enough to keep up with
the pace of technology.
And now the rest of the story . . .
After upgrading the financial system, our client had a new
challenge. The same ERP software vendor informed
them that they will need to start thinking about upgrading
their other platforms. Our sponsor wisely asked, “Is there
another way?” Thankfully, there is. Over the past five
years, the popularity of cloud applications for finance has
exploded. A 2013 Gartner study (Adoption of Cloud
ERP, 2013-2023) shows that 47% of companies plan to
move their core ERP systems into the cloud by 2018.
CFOs are moving to the cloud for many reasons;
however, some of the main benefits include (and address
our sponsor’s uh-ohs!):
With cloud applications, you subscribe to an
application and focus your efforts and energy on your
business, not the technology stack. This means you
never need to know about the Dirty Data Switch!
The cloud application provider takes care of the
technical portion of the upgrades, releasing them two to
four times a year. The upgrades take place over a few
hours and allow you peace of mind that experts are
taking care of the upgrades not only for you, but for all of
their users at once!
There is some work with cloud upgrades, but at a
fraction of the cost and effort. Your focus is on
understanding and deploying the new features,
completing regression testing, and making sure your
interfaces and reports still work (they should!). You
measure the efforts in days and hours, not months.
Innovation is constant. The cloud applications are
based on new technologies that enable innovation. They
are often built on open standards vs. proprietary
languages and third party add-ons. They are built for
how we work today (mobile), not how our parent’s
worked!
About CrossCountry
CrossCountry Consulting is an independent advisory firm
providing performance improvement, finance and
accounting, risk and technology consulting services.
CrossCountry assists leading organizations in optimizing
their finance, operations, compliance and technology
functions in response to business expansion, regulatory
mandates, and increased demand for timely and
accurate financial and management information.
CrossCountry is comprised almost exclusively of former
Big Four professionals with deep experience in project
management, business analysis, process improvement
and transformational change management. For more
information,visit
http://www.crosscountry-consulting.com.
For more information please contact:
John Hoebler
Managing Director
1650 Tysons Boulevard, Suite 720
McLean, VA 22102
Washington, DC (866) 945-4462
1650 Tysons Boulevard
Suite 720 McLean, VA 22102
New York (866) 945-4462
1501 Broadway 12th Floor
New York,
NY 10036
www.crosscountry-consulting.com
cleaned up our vendor master and re-designed security.
We streamlined key processes. We trained our users,
and they tested the heck out of it. Then came the go live
weekend. We were all now reliant on our trusty upgrade
administrator. He is a talented fellow and worked hard
throughout the project, but now we handed off the
upgrade to him, entrusting him with 120 hours of steps,
tasks and scripts that needed to be completed. We set
up a war room and had everyone on call. Then we
waited. The 120 hour plan was going well until about
hour 68. That is when the train went off the track. The
process steps that ran in minutes were now running in
hours (sound familiar?). Our administrator tried to explain
what was happening, but there was not much we could
do, aside from wait and watch the processes creep
along. As the processes ran longer, we started to get
paranoid – the process finished, but did it work? We
started QA-ing the scripts, and it threw out tons of false
positive errors as the rest of the scripts had not yet been
run. Did we have a problem or not? The upgrade scripts
are long and complex, but what else could we do at this
point? We decided to move on and run the rest of the
scripts. We ended up finishing the upgrade at hour 114,
eating into every bit of buffer that was built into the plan,
but still going live on Monday morning. Our heroic
upgrade admin made it through the weekend but was
visibly exhausted. The project sponsor quietly noted,
“Wow. We would have been in serious trouble without
that guy.”
Good point, why does it take so much effort, specific
expertise and faith to upgrade your financial applications?
Lots of Work and No Real Value Delivered.
During the upgrade, our sponsor noted that so many of
the company’s resources were working on the project.
Some of the resources were dedicated full time. Others
had to work additional hours or defer other work to be
trained in and test every single part of the application. We
had hundreds of issues related to the upgrade scripts,
the customizations, the new features, the interfaces and
of course the performance. This was a lot of work. This
was a major allocation of time and effort at the cost of
other possible activities. About half way through the
upgrade project, our sponsor asked about the value we
were gaining from the upgrade. Although there were
some noticeable benefits in several process areas, like
collections, most processes would remain pretty much
the same. This legacy on-premise ERP had been around
for decades, and the upgrades became less and less
impactful in terms of delivering new value. The project
sponsor once again asked an innocent, but spot on
question that all CFOs and CIOs should be asking, “So
why are we doing this project?”
The answer was simple, though unsatisfying – because
we have to keep up with the ERP software vendor’s
release to stay on support.
Is this the latest version of the software?
As mentioned previously, the application spanned not
only finance, but hundreds of other casual users of the
application. We were doing one of our training sessions
for these users, and it was going well. We covered the
new process and what was changing. We did live demon-
strations. We fielded many questions and everyone
seemed content. At the end of the session, one user
raised his hand. He thanked us for the training and
apologized for the question he was about to ask – “Is this
the latest version of the software?” Everyone chuckled,
and I assured him it was. He accepted the answer and
moved on. But why did he ask the question in the first
place? He was asking the question as a consumer not a
transaction processor. He wanted the application to
look, feel and interact with him like other applications do
in his daily life. He wanted it to be easy to use and
accessible not only from his desktop but on his mobile
device. The on-premise ERP software vendors seldom
deliver ground breaking innovation in their releases. They
are typically just adding a few bells and whistles to
millions of lines of code that have been developed over
the past three decades.
The reality of on-premise old ERP is that they have a hard
time innovating. Their code is complex and built on an
“upgrade” approach that just adds fixes on top of fixes on
top of fixes. They can’t move fast enough to keep up with
the pace of technology.
And now the rest of the story . . .
After upgrading the financial system, our client had a new
challenge. The same ERP software vendor informed
them that they will need to start thinking about upgrading
their other platforms. Our sponsor wisely asked, “Is there
another way?” Thankfully, there is. Over the past five
years, the popularity of cloud applications for finance has
exploded. A 2013 Gartner study (Adoption of Cloud
ERP, 2013-2023) shows that 47% of companies plan to
move their core ERP systems into the cloud by 2018.
CFOs are moving to the cloud for many reasons;
however, some of the main benefits include (and address
our sponsor’s uh-ohs!):
With cloud applications, you subscribe to an
application and focus your efforts and energy on your
business, not the technology stack. This means you
never need to know about the Dirty Data Switch!
The cloud application provider takes care of the
technical portion of the upgrades, releasing them two to
four times a year. The upgrades take place over a few
hours and allow you peace of mind that experts are
taking care of the upgrades not only for you, but for all of
their users at once!
There is some work with cloud upgrades, but at a
fraction of the cost and effort. Your focus is on
understanding and deploying the new features,
completing regression testing, and making sure your
interfaces and reports still work (they should!). You
measure the efforts in days and hours, not months.
Innovation is constant. The cloud applications are
based on new technologies that enable innovation. They
are often built on open standards vs. proprietary
languages and third party add-ons. They are built for
how we work today (mobile), not how our parent’s
worked!
About CrossCountry
CrossCountry Consulting is an independent advisory firm
providing performance improvement, finance and
accounting, risk and technology consulting services.
CrossCountry assists leading organizations in optimizing
their finance, operations, compliance and technology
functions in response to business expansion, regulatory
mandates, and increased demand for timely and
accurate financial and management information.
CrossCountry is comprised almost exclusively of former
Big Four professionals with deep experience in project
management, business analysis, process improvement
and transformational change management. For more
information,visit
http://www.crosscountry-consulting.com.
For more information please contact:
John Hoebler
Managing Director
1650 Tysons Boulevard, Suite 720
McLean, VA 22102
3
4
cleaned up our vendor master and re-designed security.
We streamlined key processes. We trained our users,
and they tested the heck out of it. Then came the go live
weekend. We were all now reliant on our trusty upgrade
administrator. He is a talented fellow and worked hard
throughout the project, but now we handed off the
upgrade to him, entrusting him with 120 hours of steps,
tasks and scripts that needed to be completed. We set
up a war room and had everyone on call. Then we
waited. The 120 hour plan was going well until about
hour 68. That is when the train went off the track. The
process steps that ran in minutes were now running in
hours (sound familiar?). Our administrator tried to explain
what was happening, but there was not much we could
do, aside from wait and watch the processes creep
along. As the processes ran longer, we started to get
paranoid – the process finished, but did it work? We
started QA-ing the scripts, and it threw out tons of false
positive errors as the rest of the scripts had not yet been
run. Did we have a problem or not? The upgrade scripts
are long and complex, but what else could we do at this
point? We decided to move on and run the rest of the
scripts. We ended up finishing the upgrade at hour 114,
eating into every bit of buffer that was built into the plan,
but still going live on Monday morning. Our heroic
upgrade admin made it through the weekend but was
visibly exhausted. The project sponsor quietly noted,
“Wow. We would have been in serious trouble without
that guy.”
Good point, why does it take so much effort, specific
expertise and faith to upgrade your financial applications?
Lots of Work and No Real Value Delivered.
During the upgrade, our sponsor noted that so many of
the company’s resources were working on the project.
Some of the resources were dedicated full time. Others
had to work additional hours or defer other work to be
trained in and test every single part of the application. We
had hundreds of issues related to the upgrade scripts,
the customizations, the new features, the interfaces and
of course the performance. This was a lot of work. This
was a major allocation of time and effort at the cost of
other possible activities. About half way through the
upgrade project, our sponsor asked about the value we
were gaining from the upgrade. Although there were
some noticeable benefits in several process areas, like
collections, most processes would remain pretty much
the same. This legacy on-premise ERP had been around
for decades, and the upgrades became less and less
impactful in terms of delivering new value. The project
sponsor once again asked an innocent, but spot on
question that all CFOs and CIOs should be asking, “So
why are we doing this project?”
The answer was simple, though unsatisfying – because
we have to keep up with the ERP software vendor’s
release to stay on support.
Is this the latest version of the software?
As mentioned previously, the application spanned not
only finance, but hundreds of other casual users of the
application. We were doing one of our training sessions
for these users, and it was going well. We covered the
new process and what was changing. We did live demon-
strations. We fielded many questions and everyone
seemed content. At the end of the session, one user
raised his hand. He thanked us for the training and
apologized for the question he was about to ask – “Is this
the latest version of the software?” Everyone chuckled,
and I assured him it was. He accepted the answer and
moved on. But why did he ask the question in the first
place? He was asking the question as a consumer not a
transaction processor. He wanted the application to
look, feel and interact with him like other applications do
in his daily life. He wanted it to be easy to use and
accessible not only from his desktop but on his mobile
device. The on-premise ERP software vendors seldom
deliver ground breaking innovation in their releases. They
are typically just adding a few bells and whistles to
millions of lines of code that have been developed over
the past three decades.
The reality of on-premise old ERP is that they have a hard
time innovating. Their code is complex and built on an
“upgrade” approach that just adds fixes on top of fixes on
top of fixes. They can’t move fast enough to keep up with
the pace of technology.
Washington, DC (866) 945-4462
1650 Tysons Boulevard
Suite 720 McLean, VA 22102
New York (866) 945-4462
1501 Broadway 12th Floor
New York,
NY 10036
www.crosscountry-consulting.com
And now the rest of the story . . .
After upgrading the financial system, our client had a new
challenge. The same ERP software vendor informed
them that they will need to start thinking about upgrading
their other platforms. Our sponsor wisely asked, “Is there
another way?” Thankfully, there is. Over the past five
years, the popularity of cloud applications for finance has
exploded. A 2013 Gartner study (Adoption of Cloud
ERP, 2013-2023) shows that 47% of companies plan to
move their core ERP systems into the cloud by 2018.
CFOs are moving to the cloud for many reasons;
however, some of the main benefits include (and address
our sponsor’s uh-ohs!):
With cloud applications, you subscribe to an
application and focus your efforts and energy on your
business, not the technology stack. This means you
never need to know about the Dirty Data Switch!
The cloud application provider takes care of the
technical portion of the upgrades, releasing them two to
four times a year. The upgrades take place over a few
hours and allow you peace of mind that experts are
taking care of the upgrades not only for you, but for all of
their users at once!
There is some work with cloud upgrades, but at a
fraction of the cost and effort. Your focus is on
understanding and deploying the new features,
completing regression testing, and making sure your
interfaces and reports still work (they should!). You
measure the efforts in days and hours, not months.
Innovation is constant. The cloud applications are
based on new technologies that enable innovation. They
are often built on open standards vs. proprietary
languages and third party add-ons. They are built for
how we work today (mobile), not how our parent’s
worked!
About CrossCountry
CrossCountry Consulting is an independent advisory firm
providing performance improvement, finance and
accounting, risk and technology consulting services.
CrossCountry assists leading organizations in optimizing
their finance, operations, compliance and technology
functions in response to business expansion, regulatory
mandates, and increased demand for timely and
accurate financial and management information.
CrossCountry is comprised almost exclusively of former
Big Four professionals with deep experience in project
management, business analysis, process improvement
and transformational change management. For more
information,visit
http://www.crosscountry-consulting.com.
For more information please contact:
John Hoebler
Managing Director
1650 Tysons Boulevard, Suite 720
McLean, VA 22102
1
2
3
4
