Automation woes widen the email expectations gap

By David Walker (Google profile)

Research from consultancy Jupiter suggests many organisations have created an "email expectations gap" - promising fast and effective response to vistors' email feedback, but then failing to deliver it. The same Jupiter work shows managers mistakenly believe that all is well. And the expectations gap is only likely to worsen as site managers come under pressure to cut costs.

And now Jupiter has released another report - "Realizing the Promise of Email Automation" - which spells out just how hard it is to close the email expectations gap.

Jupiter and many other consultants have previously suggested organisations should use software to automate large parts of the email response process. Such software typically matches customer queries to answers pulled from a pre-filled database. The firms that sell that this software, firms like eGain, Kana, Firepond/Brightware and dozens of others, all say it gets more feedback answered more effectively in less time.

Not true, says Jupiter. The typical email automation package doesn't deliver what it promises. "Most sites that have implemented email automation technologies realize little or no cost savings and often alienate customers as well", states Jupiter's report. In the normally polite world of technology evaluation and forecasting, this is like saying the vendors are stealing your jewelry and kidnapping your children.

Why so negative? In part, because the systems produce measurably rotten results. Jupiter sent a "basic customer service inquiry" to 225 different sites. Just 26 per cent of those sites used email automation software, but of that 26 per cent, "less than two per cent successfully answered the inquiry with an automated response". Presumably many of the automated responses answered the wrong question, or answered the right question incompletely.

When Jupiter dug into email automation software, it found another problem: email automation software mostly just shifts work. Front-line customer service staff do a little less, but a new group of costly staff must update that answer database.

Jupiter does like one software solution, called natural-language processing or NLP. NLP software watches the front-line staff's feedback replies and learns from them. (It's now being integrated into some of the major automation packages.) Jupiter claims US bank Wells Fargo has been able to answer four-fifths of its email feedback using an NLP system from Banter. The bad news: such systems only start to become cost effective when they're processing more than 12,000 messages per month.

Why do people keep falling for solutions like email automation before analysing the results? In part because the analysis is hard to do - which is why firms like Jupiter can sell reports. In part, too, because the culture is now saturated with the artificial intelligence dream - the dream that computers can think like people. Today's technology managers spent time reading Isaac Asimov's Robot series and watching 2001's HAL as teenagers, and perhaps even studying the artificial works of Alan Turing and Marvin Minsky at university. They want artificial intelligence to work. It usually doesn't. HAL isn't here yet.

So that familiar email expectations gap remains, with customers creating messages that organisations cannot respond to satisfactorily and cost-effectively.

But one piece of Jupiter's report raises still deeper questions about the management of feedback email. While it was digging into the costs of running automated email systems, Jupiter found that some email automation solutions left customer service staff working more slowly with email than with old-fashioned telephone calls.

I've never seen research comparing the cost of email and telephone responses to an identical customer problem. But such research might make interesting reading. Most managers assume they'll cut costs by moving interactions out of the call centre and onto email. Jupiter's work hints that this might be a myth. My own experience with email systems suggests a substantial slice of site visitor feedback is best dealt with by phone.

In the past half-decade businesses have seized on email and associated technologies as a means of making customers happier for less. Now is a good time to start figuring out just when email is really the right tool for the job.