Predictive Maintenance Pilot: Business Goals and Constraints Defined
The ACME Tool Co. operations management team now knows that:
- The company faces an existential threat from reduction in operating profit margin
- there is $190,843 of revenue per week that is not realized --- due to unplanned downtime
- they have a rough idea on how many machines per week will fail (66), causing unplanned downtime
- they don't have the resources to make a major overhaul of how they are operating (e.g., changing the maintenance patterns completely)
- predictive models have the potential to build an ordered list of the most likely to fail machines
After internal discussion, the line of business and operations management team conclude that their operations likely will not change over night "magically". They are aware, however, that predictive maintenance has the potential to rank the list of machines by which are most likely to fail based on their historical data. They are also aware that data science can be tricky to "get right" and that there will be some growing pains if they choose the path of predictive maintenance.
ACME Tool decides to investigate the prospect of a predictive maintenance pilot program but under tight yet realistic constraints. They realize that machine learning is not "magic" and it will not just "tell them the machines that will fail with zero error". They do, however, want to understand "how good the predictive model can be" and "what are the tolerance of error" --- because if they invest in this path they are betting the financial future of the company on getting close to the expected results from the pilot program.
The executive team commits to funding a small team of technicians to do overnight maintenance on 18 machines each night on the machines that are likely to fail the next day. These 18 machines will be selected based on the data science team's model's top 18 most likely to fail machines.
Pilot Financial Goals
The finance team has calculated that if they drop the price of the product from $20 to $18.50 (dropping annual revenue to $17,203,662) then this would drop the operating profit margin of the company from 12% (decent) to
4.86% (not good) because (in theory) cost of goods sold and operational expenses would stay the same (if they sold the same number of tools).
Based on financial analysis ACME realizes that an operating profit margin of
4.86% does not bode well for long term operations of the company.
PM Program Cost
The finance and operations team came up with the following metrics and costs that it would take to perform preventative maintenance on 18 machines each night:
- Hours per Shift: 9 hours
- Time required per PM Fix: 1 hour
- Fixes per PM Shift: 9 machines fixed
- Required PM Technicians: 2
- Cost per Year per PM Technician: $64,800
- Total PM Program Cost: $129,600
With this overhead in mind, let's take a look at our goals for the pilot.
Minimum Viable Goal
The finance team has determined that an operations margin of around
9.6% is a realistic minimal viable number to shoot for if the management team wants to remain minimally competitive in the market (e.g., being able to raise capital, etc).
In the best case, the finance team would like to get closer to the original operating profit margin (
12%), but they realize this could be a difficult goal under the current circumstances. Regardless, the team decides to set that as the stretch goal for the project.
The team has calculated that if they can detect 11 failures (out of 18 predictions, or 61% accuracy) per day, they could reach a margin of
9.6% with PM Program costs included. This seems like a safe and reasonable minimum viable goal, but given the stakes, the team needs goals they have a high chance of clearing.
The stretch goal is to get back to the
12% operating margin, however, and if they can detect 14 out of 18 (78% on the top 18 predictions) then this would get them back to an
10.9% operating profit margin.
If they could hit this rate 95% of the time, this would reduce daily failures from 66 to 52 (detecting and preventing 21% of the failures).
However, with the lowered price but also using predictive maintenance to reduce downtime, the company summarized that:
- they could operate the same hours and produce 1,027,084 (+10.45%) tools
- operational expenses would hold constant
- cost of goods sold would increase slightly (labor would hold constant, but more products produced costs +10.45% more materials)
The finance team concluded that with the reduced downtime and lower price that ACME would sell
in product as well, with an operating profit margin of
Needless to say, the executive team was excited at the prospect of being able to combat the price drop and potentially salvage their operating profit margin.
With this information in hand, the ACME Tool Co operations team sets up a meeting with the data science team to map out the best path to produce a pilot project.
Summary of Pilot Goals
To implement the new pilot system, ACME Tool Co. does not want to make major changes to their process, so they keep the reactive maintenance process in place during the normal daily operational hours.
Based off past experimentation, both the executive team and the data science team know that predictive models are not perfect and you have to set expectations to have any shot at being successful with a machine learning project. With this in mind, the executive team works with the data science team to set the following pilot parameters:
- the company is willing to pay for a small team of technicians to do overnight maintenance on
18 machines that are likely to fail the next day
- minimum viable goal: at least
11 out of 18 correct (
61% out of top 18) 95% of the time
- stretch goal:
14 out of 18 predictions correct (
78% out of top 18) 95% of the time
The IT team started collecting machine sensor data about 2 years ago. They currently continue to daily collect the sensor data on how each machine on the line is used (torque, minutes, etc). The manufacturer of the machines also provides data on 10k known devices across multiple companies in a data share agreement to help with maintenance modeling. So ACME Tool Co. has some data to work with, but it remains to be seen how valuable the data really is in the pursuit of predicting machine failure.
Operational Contract Between Line of Business and Data Science Team
The line of business doesn't need to understand everything that goes on in data science land, but there should be some sync points along with an start state and then acceptance criteria for completion.
The data science team is provided the 10k machines worth of historical data to begin their analysis with.
At this point, the data science team has a starting point (data, resources, ROI targets), and they commit to delivering a model and an analysis of the expected performance of the model under the supplied conditions (first deliverable). Together, the data science team and the business team will analyze how the model's performance impacts the business in financial terms.
The (second) deliverable from the data science team to the operations management team is a simple report of the top 18 "most likely to fail machines" every day based on the data up to that day. The data science team doesn't have to build any fancy apps, just a simple text report via email or SQL view.
With the 2 clear established goals, parameters for the pilot, and a collaborative framework, the data science team now sets off to write a plan to achieve the business team's target goals for ROI with a predictive model.