Price Waterhouse Coopers report on technology was expensive and is flawed
This report from Price Waterhouse Coopers was produced for the DoH in January.
One line caught my eye a couple of months ago which I knew was wrong – reference 3.1- and the correspondence with PWC is noted below. It made me wonder just how much of this report – which the DoH have told me cost over 75K – is based on fact.
Below are notes on the quality of the evidence they have used to produce what they call evidence which the Department of Health say “confirmed the potential for resources in the order of billions of pounds each year to be reutilised for better care by the NHS from making better use of information and technology.” I think the report did nothing of the sort.
The next time it’s claimed that this report showed how much technology could ‘save’ the NHS, I’d suggest asking for evidence and checking the references.
PWC quotes from “A review of the potential benefits from the better use of information and technology in Health and Social Care” (link above)
Page 23
“A more recent report in 2008 estimated the cost of ADR related admission to be c.£1.9bn p.a. This report used the same assumptions as a 2004 study
(ref 1.1) Adverse drug reactions as cause of admission to hospital: prospective analysis of 18 820
patients, British Medical Journal, July 2004
and assumed that 6.5% of hospital admissions are the result of adverse reactions, resulting in an average stay of eight days). N(ref 1,2)” Compass study, 2008 – also quoted in the Guardian Thursday 3 April 2008
The first study (ref 1.1) did not examine how many ADR is was possible to reduce while still having benefit from the drugs, ie “many of the implicated drugs have proven benefit”. There is no doubt that many prescribed drugs have side effects. But the study did not claim that these can all be negated is wrong. Reference 1.2 was referenced to a Guardian story. http://www.guardian.co.uk/society/2008/apr/03/nhs.drugsandalcohol
This in turn references a report from the ‘thinktank’ Compass, which has a press release on it’s website on the same date. http://www.compassonline.org.uk/news/item.asp?n=1551. This states that it is running an ongoing investigation which it said would be published in the Autumn. It appears therefore that PWC used only the Guardian story and the press release on the Compass website (which is unreferenced) to support their claim.
Based on the assumption that an ePrescribing service could reduce preventable ADRs by c.60% (based on case studies c£1.9bn, of which 50% are preventable (ref 1.3),
http://www.plosone.org/article/info%3Adoi%2F10.1371%2Fjournal.pone.0033236
This study was a high quality meta analysis. However, it did not examine people who were outpatients or had an emergency visit to hospital. Additionally, they used the criteria of a preventable ADR when “the drug event was due to a drug treatment procedure inconsistent with present-day knowledge of good medical practice or was clearly unrealistic, taking the known circumstances into account” . 2 of the 8 studies included in the analysis took place in the UK, where the NHS issues guidelines on treatments via NICE and SIGN. These two studies were
http://onlinelibrary.wiley.com/doi/10.1111/j.1365-2710.2006.00744.x/abstract
and
http://www.plosone.org/article/info%3Adoi%2F10.1371%2Fjournal.pone.0004439
The first study was a pilot for the second study. The second study noted that “half of ADR were definitely or possibly avoidable”. It did not attempt interventions to test this hypothesis. There has been no evidence offered that it is possible to reduce the burden of adverse drug reactions so far.
The introduction of an electronic prescribing solution by Doncaster Royal Infirmary was found to have the potential to result in a 60% reduction in ADRs (ref 1.4)
This is referenced to “JAC Medicines Management Case Study” which had taken place in Doncaster. JAC are a company claiming to the “The UK’s medicines management specialists”. There is no other reference cited. Google comes up with nothing. After trawling through their website for some time I found this article
http://ccgi.jacpharmacy.plus.com/joomla/Doncaster%20Royal%20Infirmary.pdf
and
which says that “ A further audit of the effect of electronic prescribing and associated clinical decision support has shown that the system can reduce potential adverse drug events by up to 60 per cent” yet does not demonstrate this with any data or publication of results.
an ePrescribing service could result in cost benefits of c.£570m p.a.
This assumes that it works; we haven’t been shown any evidence of this.
• As a result, we conservatively estimate potential NHS total gross benefits of c.£285m p.a
• Comparison of these costs with the cost benefits of c.£285m p.a. described above, suggests a year 1 net benefit of c.£208m (as occurs in year 1) and net benefits of c.£270m p.a. from year 2 onwards.
This is quite amazing. There is incredible leapfrogging of imagination over non-knowledge to create a figure based on supposition. This conclusion is plainly daft.
Page 27
By providing patients, family members and social care workers with access to an online portal with increased educational information throughout the post-operative process, there could be significant post-discharge benefits for patients. These benefits can be allocated to key post-discharge objectives that indicate a successful end to care:
1. That the level of medical support provided, the number of follow up appointments and readmission rates fall within thresholds
• One of the major goals of the NHS is to reduce the rate of emergency readmissions as they are both costly and impact patient recovery
• According to NHS statistics there was 561k emergency readmissions to the NHS within 28 days of discharge in 2010/11
• A Department of Health funded report by the RAND corporation found evidence that suggested 15-20% of total emergency admissions within 28 or 30 days of discharge from hospital may be avoidable (ref 2.2)
RAND are a “non profit institution that helps improve policy and decisionmaking through research and analysis”. The cited report
http://www.rand.org/pubs/technical_reports/TR1198.html is a “rapid review of systematic reviews”. This paper states that most of the analysed studies take place outwith the NHS, and states “prospective studies are needed to assess the proportion of readmissions that are avoidable in the contemporary NHS”. It also notes that readmissions are more common in people who are sicker. Some of the criteria are plainly unrealistic. For example, any readmission for the same diagnosis for a patient initially admitted with heart failure is treated as ‘avoidable’, yet heart failure is a chronic and usually worsening condition where future exacerbations are perhaps reducable but unavoidable. It is ironic that this section (about undertreatment) follows the pages about overtreatment (complications of prescription medication.) Specifically, social problems or a lack of community care was mentioned frequently as a cause of potentially avoidable admissions. There was no mention of ‘technology’ underuse as a cause for readmissions.
• Whilst unplanned readmission may be a result of a range of factors (infections, complications, choice of surgical approach an lack of patient education as a reason for readmission. A 2012 US study on readmission reasons post pancreaticoduodenectomy found that poor discharge education was a key factor in high rates of readmission, specifically for dehydration and malnutrition (ref 2.3)
http://ons.metapress.com/content/f387578115952008/?genre=article&id=doi%3a10.1188%2f12.ONF.408-412
Pancreaticoduodenectomy is an uncommon and very complex surgical procedure which is relatively high risk for complications. This US study cited looked at 62 patients with pancreatic cancer who underwent this operation and documentation about ‘self care advice’ in the notes. Critically, this study did not trial the effect of discharge education – it’s easy to see that this could be a confounding factor rather than a causative factor (ie a busy ward where the nurses don’t tick the box that advice was given may also not do other tasks which could impact on the person’s health.) PWC have not provided any RCT evidence that increased ‘education’ can reduce unnecessary re-admissions.
• It is estimates that by increasing the availability of educational material for patients to assist self management of post-operative care, there could be a significant reduction in emergency admission rates
PWC have provided absolutely no evidence to justify this statement.
Page 30
“While some / many are necessary, it is estimated that between 20% and 30% (ref 3.1) of short stay emergency admissions could be treated either at the A&E stage or even in primary care , and admissions are often made to avoid breaches of the 4 hour A&E waiting time threshold.”
Ref 3:1: Primary Care and Emergency Departments, Report from the Primary Care Foundation, March 2010
This was the flaw I first spotted in this report on the 20/1/13. On the 4/2/13 I had a reply which insisted it was correct:
“You raised some queries re the PwC report for the Department of Health on use of technology, re references used. Sorry for the delay in getting back to you. The team have looked into this – here’s the details.
The reference 4.3 to the report on Primary Care and Emergency Departments (page 35) is correct. The PwC report refers to “up to 30%” of non – elective admissions” – reflecting the Primary Care report’s finding that “between 10%-30%” of emergency cases “could be classified as primary care” – ie “cases of a type that are regularly seen in general practice.”
One of the ways of achieving this would be to provide community teams with access to comprehensive patient data and clinical decision support including diagnostic services as highlighted in the report. While the report estimated potential financial benefits of various actions on IT, the overall financial estimates for potential savings did not include this specific action”
In other words, not just insisting that it was correct, but also inferring that more technology could sort out the ‘problem’. On the 20/2/13, following a further email from me, they agreed that
“On page 33 of the report the word “admission” is incorrectly used instead of “attendance.” This is being corrected and the report should have read:
“Evidence suggests that certain non elective attendances could be avoided by managing patients in community care settings without compromising outcomes. One of the ways of achieving this would be to provide community teams with access to comprehensive patient data and clinical decision support including diagnostic services. Looking after patients at home is cheaper, mostly because there is no need for 24/7 medical and nursing, and where an overnight sitting service is required, this is substantially cheaper than a hospital ward.”
So PWC confused admission – where people go into hospital to stay – with attendance – where people are seen by a healthcare professional as outpatients. Misleading, but they say they will correct it and send the DOH a corrected version. But they are still wrong. PWC have not provided data that says that attendances can be reduced – the study they cite is about where, or who, sees people who are sick. There is no data contained in this report that suggests that there is work that doesn’t need to be done. The question is who does they work and with what qualifications – in other words, emergency department staff versus general practitioner staff. The paper is about aligning the two, which is being done in many places already (two thirds, according to this report). PWC are wrong and in fact their statement makes no sense – these patients didn’t need ongoing management in the community, they needed a consultation with a healthcare professional out of routine hours.
Additionally, PWC are failing to support their assertion that there are no adverse effects of community care – this is unreferenced.
“Bringing all short stay emergency admission rates down to the current national average would result in over 170,000 fewer emergency admissions with initial savings to commissioners amounting to over £130m p.a. (aggregated across all Commissioners nationally). Ultimately, if providers make operational adjustments to reflect the contractual changes, whole system benefits of a similar scale should be achieved”
This is a startling and somewhat bizarre interpretation of ‘average’. Average admissions rates only tells you where the middle is, not whether it is best for patients. Do some hospitals under-admit, due to lack of beds or other pressures? Are admissions warranted, or not? Who is not being admitted who should? The idea of this economic evaluation providing targets for clinical care would be laughable were it not so stupid.
Page 31
“Once a patient is referred to a hospital and a diagnosis and treatment plan is decided upon, the follow -up care can often be carried out in a community or primary care setting rather than requiring repeated visits to hospital. It is estimated that between 20% and 30% (ref 3.1) of hospital follow up visits can be better managed in a primary care setting.”
This is again referenced to the Primary Care and Emergency Departments, Report from the Primary Care Foundation, March 2010, and there is no data in it to support this assertion.
Page 33
Evidence suggests that up to 30% (ref 4.3) of non elective admissions can be avoided by managing patients in community care settings without compromising outcomes. This is achieved partly by providing community teams with access to comprehensive patient data and clinical decision support including diagnostic services. Looking after patients at home is cheaper, mostly because there is no need for 24/7 medical and nursing, and where an overnight sitting service is required, this is substantially cheaper than a hospital ward.
Slightly unbelievably, reference 4:3 is – again!- Primary Care and Emergency Departments, Report from the Primary Care Foundation, March 2010. PWC have made the same mistake again – which makes me think that this is not a one-off error, but a conceptual misunderstanding. The report did not say that we could avoid admissions by treating patients in the community instead. These patients WERE NOT GOING TO BE ADMITTED IN THE FIRST PLACE; the 30% relates to people who could be seen by GPs rather than A+E departments. This means that all the calculations which follow this non evidence based statement crumble. The same paper is referenced again when PWC decide on page 34 that
“unified information views and availability of information to enhance clinical decision making…a conservative saving of 3 minutes of nurse time would result in annualised savings to the acute trusts amounting to over 50 million
A reduced number of duplicate diagnostics may be required. If it is assumed that 0.5% of outpatient attendances would not now require a simple (eg blood) test this would result in annualised savings to the Acute Trust amounting to over £15M (aggregated across all Acute Trusts nationally) “
This is pretty amazing, given that there were no such workings out in the report being cited, and they haven’t demonstrated any such duplication of diagnostics or real-life savings on nurse time either. Wow. As for the statement that
“Through more integrated discharge management and better information availability for community staff, 30 day readmissions may be reduced (ref 4.1) “
4:1 Northern Ireland Electronic Care Record – Proof of Concept Project Evaluation Report, 21/02/2011
All I can find is some press releases http://www.northernireland.gov.uk/index/media-centre/news-departments/news-dhssps/news-dhssps-290412-poots-announces-multi.htm which announce the contractor, Orion, as the new caterer of electronic records in NI. I can’t find any data in Pubmed about this project. I have written to the http://www3.hscni.net/ecr/ system managers here to ask for this data. As it stands, there seems no publically available information which backs up PWC claim that electronic records can reduce readmissions.
The PWC report then goes on to discuss ‘supporting case studies’ which I don’t think they can be considered to be.
Page 43
Action 1; Case study 1
ePrescribing systems reducing Adverse Drug Reactions (UK)
This was produced by JAC Medicines Management, a private company who wrote software used in the ‘case study’ in Doncaster. PWC claim that this “could reduce potential adverse drug events by up to 60%.”. The link PWC provide to the research doesn’t work. On the JAC website there is this webpage http://www.jac-pharmacy.co.uk/~jacpharmacy/joomla/index.php?option=com_content&view=article&id=74:e-prescribing-and-decision-support-improve-patient-care&catid=10:articles&Itemid=12
which claims “After the introduction of e-prescribing and CDS on a four-ward medical unit at Montagu Hospital, Doncaster and Bassetlaw Hospitals NHS Foundation Trust, compliance with trust policy for recording of drug administration rose from 65% to 100% and potential adverse drug events (events with the potential to cause harm, delay recovery or result in a lack of control of symptoms) were reduced by over 61%”
This is referenced to Barker A, Kay J. Electronic Prescribing Improves Patient Safety — An Audit. Pharm J 2007; 14:225.
It doesn’t appear on PubMed. It does appear on this page of the Hospital Pharmacist webpage
A kind twitter friend sent me the full pdf. If you are a member of the Royal Pharmaceutical Society, it’s here http://www.pjonline.com/libres/pdf/hp/200707/hp_200707_audit.pdf. I am willing to bet that it wasn’t read in full by the authors. They found that adverse drug reactions (which they haven’t stated how they defined or measured) were reduced after the e-prescribing system was introduced – but it is impossible to draw conclusions about causation since it is likely that with the new system being introduced staff changed their ways of seeking information about medication – and additionally, electronic prescribing was found to increase errors in selecting the correct medication from drop down menus.
Page 43
Action 1: Case study 2
Adverse Drug Reactions prevalence as reason for hospital admissions
This is referenced to http://www.bmj.com/content/329/7
We already know that ADR cause harm. What we have less data on is how to prevent them within the NHS (which works to different terms compared with eg North America.)
Page 46
Increasing patient educational information to reduce post operative pain
This cites a study showing that online advice about pain management could reduce pain experienced by patients post operatively. The link PWC cites is broken, and since they don’t write it out as references usually are, it takes rather longer to track it down than is necessary. But it appears to be this one,
http://pubmedcentralcanada.ca/pmcc/articles/PMC2232814/pdf/procamiasymp00004-0817.pdf
which compares two groups who were both given the address of a website for information after surgery. One group got extra information about pain management, and the other one didn’t. The patients who got more information about pain management had less severe pain. But two major problems; firstly, that the pain scores when patients arrived at home (before looking at the website) was quite different between the two groups, and second, there was no comparison with non electronic paper based or verbal information. Using this study to infer anything about how information is best given would be entirely misleading.
Action 2: Case study 2
Health Unlocked – Royal National Orthopoedic Hospital (2011)
Increases patient satisfaction with 95% of patients prefer the new online process to the traditional pen and paper method; than and using traditional methods of reporting.
• The process also encourages patients to report more often, which can lead to the identification of post-operative complications faster than follow up appointments.
Certainly this is what Health Unlocked, a company who offer data management, say: http://www.healthunlocked.com/about/in-clinic-casestudies/
As do the Royal National Orthopoedic Hospital in a press release;
http://www.rnoh.nhs.uk/home/news/nhs-patients-have-the-last-word-the-success-their-treatments
Interstingly there is a puff quote from ex health minister Andrew Lansley there, saying that “This is a great example of how healthcare innovation can make a real difference to patients’ lives. We want to modernise the NHS because we believe that patients deserve the best and most innovative treatments available. We are committed to creating an environment in the NHS where new medical technologies like this can flourish”
Quite clearly, the minister was a fan of the technology – what’s far less clear is the evidence of benefit. The trial is still ongoing, meaning that we don’t know if it works – or if there are any unintended harms. I’ve contacted Health Unlocked to ask them for futher details.
Page 48
Action 4: Case study 1
“Paperless systems increasing clinical inefficiencies (I think they have made a typo – they are meaning to argue FOR paperless systems and more tech to be MORE efficient, not less efficient.)
Objectives of the trial was to eliminate time and cost associated with paper notes e.g.:
• 50% of records were not available when requested (either due to loss or use by other care provider); and
• Clerks spent 60% of their time chasing missing case notes”
Unfortunalty, and yet again, data isn’t available to examine these claims. As reference, PWC give two items
Again, a private technology company was contracted to provide this new database management service. The news stories seem to have been triggered by Alison Clare, who managed the electronic record at the trust when speaking about it at HC2011 informatics conference in Birmingham. Without the full data we can’t know whether the time efficiency was measured merely in finding the case notes or in using them – and given the multiple complaints from front line healthcare staff about the unwieldy nature of electronic medical records, we can’t be sure whether this ‘advance’ actually is one.
Page 49
“Action 4: Case study 2
Paperless systems increasing clinical inefficiencies”
This is quite funny – they’ve made the same typo AGAIN. Their links to news reports are broken, and I am presuming that as they link to news reports rather than to actual data, they haven’t seen any raw data. I think they mean to link to
http://www.businessweek.com/stories/2004-07-06/a-paperless-health-care-system
which is a piece saying things like “It’s very early, but Neaman says the results are promising. Doctors now receive patients’ mammogram test results in a single day, compared to a three-week wait before.” Yes – it’s very early. And it states that it cost 5 million on staff training and 16 hours each before they could use it. That’s a big spend on technology.
Another case study (Action 4: Case study 3) seem to have been included to make the point that some patients using A+E can be seen by primary care doctors (we already knew that, that’s why lots of out of hours services have the two on the same site.)
Action 4; Case study 4 (page 51)
This is based on a King’s Fund report
which rests on the assumption that “effective management and treatment should limit emergency admission to hospital”. This is especially interesting to me, as for part of the GP contract, I now have to analyse a selection of our practice’s admissions to hospital and essentially justify them. Essentially I have found that patients who are admitted to hospital are admitted for good reason and not lightly. The King’s Fund pronounces that an amazing range of conditions are essentially preventable, listing
Vaccine-preventable
1. Influenza and pneumonia
2. Other vaccine-preventable conditions
Chronic
3. Asthma
4. Congestive heart failure
5. Diabetes complications
6. Chronic obstructive pulmonary disease (COPD)
7. Angina
8. Iron-deficiency anaemia
9. Hypertension
10. Nutritional deficiencies
Acute
11. Dehydration and gastroenteritis
12. Pyelonephritis
13. Perforated/bleeding ulcer
14. Cellulitis
15. Pelvic inflammatory disease
16. Ear, nose and throat infections
17. Dental conditions
18. Convulsions and epilepsy
19. Gangrene
There is a real dichotomy here. On one hand right at the start of the report, we are advised that many patients have adverse drug reactions and are thus overtreated. In this part of the document, we are being advised that more treatment can prevent hospital admissions.
This is simply incorrect. Firstly, we do not have means which can tell us which patients, precisely, benefit, and which will be harmed through evidence based interventions.
Secondly, the idea that this list contains preventable conditions is laughable. For example, the King’s Fund say
“Many of these cases of influenza and pneumonia are vaccine-preventable.”
I’d suggest reading Cochrane on this http://summaries.cochrane.org/CD004876/vaccines-for-preventing-seasonal-influenza-and-its-complications-in-people-aged-65-or-older
And http://summaries.cochrane.org/CD000422/vaccination-for-preventing-pneumococcal-infection-in-adults. As for the idea that doctors admit patients with gastroenteritis, gangrene, anaemia or bleeding ulcers when they were either preventable or treatable in the community – this is neither evidenced or takes account or whether it was good for the patient.
Action 4: Care study 5
This covers a pilot study, http://www.bmj.com/content/344/bmj.e3529, whose results have not been published. To suggest that the outcomes – including a reduction in emergency admissions, outpatient referrals or ‘releasing 10-12 million a year’ can be measured at this point or even caused by the pilot is hype and misleading.
Action 4: Case study 6
Data requested, not published online,
Action 4: case study 7
Torbay Care Trust Integrated Health and Social Care (2011)
This case study is based on this King’s Fund report
http://www.kingsfund.org.uk/publications/integrating-health-and-social-care-torbay
and PWC claimed that the outcomes of the Torbay project were
- Reduced bed occupancy (daily average number of occupied beds fell from 750 in 1998/99 to 502 in 2009/10);
- Reduced emergency bed day usage (emergency bed day use for people aged over 75 fell by 24% between 2003
and 2008 and for those aged over 65 is 1920 per 1000 population compared to an average of 2698 per 1000 in 2009/10); and
- Delayed transfers of care from hospital have reduced to low level
Margaret McCartney's Blog
- Margaret McCartney's profile
- 13 followers
