Spotting Process Bottlenecks With Process Mining and ML

The process on the wall is not the process

Most businesses have a process diagram somewhere. Order received, credit check, pick, pack, dispatch, invoice. It is tidy. It is also wrong, in ways nobody running the process would disagree with once they see the evidence.

Real orders loop back for credit re-checks, wait two days for a manager's approval that the diagram does not mention, get split, get amended, get cancelled and reinstated. Process mining shows all of that, because it is built from what systems recorded rather than what people remember.

What process mining needs: an event log

The only input is an event log with three things per row. Most ERP, CRM, ticketing and workflow systems can produce one.

The work is in extraction. Events live in different tables, sometimes different systems, and some steps are not recorded at all. An approval given by email leaves no timestamp. Those gaps are themselves a finding, since an unrecorded step is often exactly where the delay is.

What the process map reveals

• Waiting time between steps, which almost always dwarfs the time spent doing the steps
• Rework loops, such as orders returning to credit check or invoices re-issued
• Variants: how many different paths cases actually take, and which are slowest
• Handovers between teams and the delay each one adds
• Conformance gaps, where cases skip required steps or do them out of order
• Batching effects, like approvals that only happen on Thursday afternoons

Take a distributor processing 1,200 orders a week with a promised three-day turnaround. The map might show that picking and packing take a few hours, while orders from new accounts wait a median of a day and a half for a manual credit review, and orders amended by the customer go back to the start of the queue. Neither is visible on a dashboard of average turnaround. Both are obvious on a process map filtered by variant.

Where machine learning adds to process mining

Process mining describes the past. Machine learning makes it forward-looking.

1. Late case prediction. For each open case, predict the probability it will miss its deadline based on its path so far, its attributes and current workload.
2. Remaining time prediction. Estimate when an open case will finish, which feeds customer updates and planning.
3. Next activity prediction. Spot cases likely to enter a rework loop so the cause can be fixed before it happens.
4. Root cause analysis. Models trained on case attributes show which factors, such as customer type, product or team, are associated with slow variants.

A daily list of the 30 open orders most likely to miss the promised date, with the reason, is a very practical output. The operations team deals with those first instead of chasing whoever phones loudest.

Tools, and doing it on a small budget

Commercial process mining platforms are capable and priced for enterprises. For a single process in an SME, they are rarely necessary.

• Open-source libraries such as PM4Py in Python do discovery, variant analysis and conformance checking
• A well-structured SQL extract plus a few charts answers many questions without any process mining library
• Some ERP and workflow tools include basic process analytics worth checking first

The discipline matters more than the tool. Choose one process, one clear question such as why do orders miss three days, and one reliable event log. Our write-up on mapping business processes before automating covers the interview side, which complements the data very well.

When process mining will disappoint

If key steps happen in email, spreadsheets or conversations, the event log will show long gaps with no explanation. You will know where the time goes missing but not why. That is still useful, but set expectations.

It also disappoints when timestamps are unreliable: records updated in batches at the end of a day, statuses changed retrospectively, or system clocks recording when data was synced rather than when work was done. And it is a poor fit for processes with very few cases, where interviewing the three people involved is faster.

Process mining is very good at showing where the waiting is. It is rarely able to tell you, on its own, why people are waiting.

How SpiderHunts would run it

At SpiderHunts we would start with one process and a two to three week discovery: extract the event log, build the map, and review it with the people who run the process. That session is where the why gets answered, and it usually produces a shortlist of fixes, several of which need no software at all.

If late cases are a recurring cost, we add a prediction model and a daily exceptions list. Where the fix is automating a manual step, such as the credit review, that becomes process automation work. For the non-data side of the same question, see how we find the bottleneck in your operation.

Frequently asked questions