House of Commons Committee of Public Accounts Access to clinical trial information and the stockpiling of Tamiflu Thirty-fifth Report of Session 2013–14 Report, together with formal minutes, oral and written evidence Ordered by the House of Commons to be printed 18 December 2013 HC 295 Published on 3 January 2014 by authority of the House of Commons London: The Stationery Office Limited £11.00 Committee of Public Accounts The Committee of Public Accounts is appointed by the House of Commons to examine ‘‘the accounts showing the appropriation of the sums granted by Parliament to meet the public expenditure, and of such other accounts laid before Parliament as the committee may think fit’’ (Standing Order No 148). Current membership Rt Hon Margaret Hodge (Labour, Barking) (Chair) Mr Richard Bacon (Conservative, South Norfolk) Stephen Barclay (Conservative, North East Cambridgeshire) Guto Bebb (Conservative, Aberconwy) Jackie Doyle-Price (Conservative, Thurrock) Chris Heaton-Harris (Conservative, Daventry) Meg Hillier (Labour, Hackney South and Shoreditch) Mr Stewart Jackson (Conservative, Peterborough) Nicky Morgan (Conservative, Loughborough) Fiona Mactaggart (Labour, Slough) Austin Mitchell (Labour, Great Grimsby) Nick Smith (Labour, Blaenau Gwent) Ian Swales (Liberal Democrats, Redcar) Justin Tomlinson (Conservative, North Swindon) Powers Powers of the Committee of Public Accounts are set out in House of Commons Standing Orders, principally in SO No 148. These are available on the Internet via www.parliament.uk. Publications The Reports and evidence of the Committee are published by The Stationery Office by Order of the House. All publications of the Committee (including press notices) are on the internet at www.parliament.uk/pac. A list of Reports of the Committee in the present Parliament is at the back of this volume. Additional written evidence may be published on the internet only. Committee staff The current staff of the Committee is Adrian Jenner (Clerk), Claire Cozens (Senior Committee Assistant), James McQuade (Senior Committee Assistant), Ian Blair and Yvonne Platt (Committee Assistants) and Alex Paterson (Media Officer). Contacts All correspondence should be addressed to the Clerk, Committee of Public Accounts, House of Commons, 7 Millbank, London SW1P 3JA. The telephone number for general enquiries is 020 7219 5708; the Committee’s email address is pubaccom@parliament.uk 1 Contents Report Summary 3 Conclusions and recommendations 5 1 The availability of clinical trial results and the sharing of information between the MHRA and NICE 2 Page Stockpiling Tamiflu and the management of the stockpile 8 11 Formal Minutes 13 Witnesses 14 List of printed written evidence 14 List of Reports from the Committee during the current Parliament 15 3 Summary The Department of Health (the Department) spent £424 million on stockpiling Tamiflu, an antiviral medicine used in the treatment of influenza, for use in a pandemic, but had to write off £74 million of its Tamiflu stockpile as a result of poor record-keeping by the NHS. There is a lack of consensus over how well Tamiflu works, in particular whether it reduces complications and mortality. Discussions over this issue among professionals have been hampered because important information about clinical trials is routinely and legally withheld from doctors and researchers by manufacturers. This longstanding regulatory and cultural failure impacts on all of medicine, and undermines the ability of clinicians, researchers and patients to make informed decisions about which treatment is best. There are also concerns about the information made available to the National Institute for Health and Care Excellence (NICE) which assesses a medicine’s clinical and cost–effectiveness for use in the NHS. 5 Conclusions and recommendations On clinical trials 1. We were surprised and concerned to discover that information is routinely withheld from doctors and researchers about the methods and results of clinical trials on treatments currently prescribed in the United Kingdom. This problem has been noted for many years in the professional academic literature, with many promises given, but without adequate action being taken by government, industry or professional bodies. This now presents a serious problem because the medicines in use today came on to the market—and were therefore researched—over the preceding decades. None of the latest proposals from regulators or industry adequately addresses the issue of access to the results of trials from previous years on the medicines in use today. Recommendation: The Department should take action to ensure that the full methods and results are available to doctors and researchers for all trials on all uses of all treatments currently being prescribed, and should also ensure that there is clear and frequent audit of how much information is available and how much has been withheld. 2. The results of clinical trials on humans are the key evidence used by regulators, researchers and clinicians to assess whether a medicine works and how safe it is. Medicine manufacturers submit evidence on products they wish to market in the UK to the Medicines and Healthcare Products Regulatory Agency (MHRA) or the European Medicines Agency (EMA). 3. The scope for independent scrutiny of a medicine’s effectiveness is undermined by the fact that the full methods and results of many clinical trials are not made available to doctors and researchers. The problem of non-publication of clinical trial results has been known since the mid-1980s. We also heard evidence that trials with positive results are about twice as likely to be published as trials with negative results. While several clinical trial registries have been established, none covers all clinical trials on all uses of all treatments currently being prescribed worldwide. There have been recent announcements by the EMA, and some manufacturers, to improve access to information about clinical trials but none adequately addresses the issue of incomplete disclosure throughout medicine. Opening up information about all clinical trials to medical researchers would support the work of regulators by permitting thorough, independent external review by doctors and researchers . Recommendation: The Department and the MHRA should ensure, both prospectively and retrospectively, that clinical trials are registered on an appropriate registry and that the full methods and results of all trials should be available for wider independent scrutiny, beyond the work undertaken by regulators during the licensing process. 4. NICE and the MHRA do not routinely share information provided by manufacturers during the process for licensing medicines. When applying for a 6 licence, manufacturers have a legal obligation to provide all the information on the safety and efficacy of a medicine that is required by European regulators. However, NICE does not have statutory powers to demand information from manufacturers, in contrast to the Institute for Quality and Efficiency in Healthcare in Germany, which performs a similar role to NICE. NICE seeks confirmation from the medicine manufacturer’s UK medical director on the completeness of information, but this may not include all clinical trials in other parts of the world, not least because UK medical directors may themselves not have full information. The MHRA confirmed there was no legal obstacle that would prevent it from sharing information with NICE. However, there is no routine sharing of the information provided by manufacturers to regulators as part of the licensing process with NICE. This leads to the risk of omissions and duplication in the collection of evidence. Recommendation: NICE should ensure that it obtains full methods and results on all trials for all treatments which it reviews, including clinical study reports where necessary; make all this information available to the medical and academic community for independent scrutiny; and routinely audit the completeness of this information. NICE and the MHRA should put in place a formal informationsharing agreement to ensure when NICE appraises medicines it has access to all of the information provided to regulators by the manufacturer during the licensing process. On Tamiflu 5. The number one risk on the Government’s national risk-assessment for civil emergencies, ahead of both coastal flooding and a major terrorist incident, is the risk of pandemic influenza. Between 2006-07 and 2012-13, the Department spent £560 million on stockpiling two antiviral medicines for use in an influenza pandemic— £424 million on Tamiflu and £136 million on Relenza. 6. There remains a lack of consensus over how well Tamiflu works and there is disagreement about whether regulators and NICE received all the information on Tamiflu during the licensing process. The MHRA is confident that European regulators received all the information on Tamiflu. However, following the hearing the Cochrane Collaboration1 wrote to the Committee to draw attention to trials where the Cochrane Collaboration concluded the EMA had incomplete information. Table 1 of Cochrane’s submission sets out the information that the Cochrane Collaboration received from the EMA in response to a request for all information held by the agency, and it is plain that for many large trials no information was available, and that for many more trials only partial information was available. The Committee shares the concern expressed by the Cochrane Collaboration when it wrote: “We find it perplexing that the regulators continue to state that they had all the available evidence”. The Cochrane Collaboration is now receiving full clinical study reports from Roche, the manufacturer of Tamiflu, which will enable the Cochrane Collaboration to complete its review of the effectiveness of Tamiflu with 1 The Cochrane Collaboration is an international network that undertakes systematic reviews of primary research in healthcare and health policy. 7 complete information for the first time. Whether or not the Cochrane Collaboration’s overall recommendation changes, it is extremely concerning that there has been a five-year delay and that there continues to be a lack of clarity over who saw what.2 Recommendation: Once the Cochrane Collaboration has completed its review of Tamiflu using all clinical study report information, the Department, MHRA and NICE should consider whether it is necessary to revisit previous judgements about the efficacy of Tamiflu. 7. The case for stockpiling antiviral medicines at the current levels is based on judgement rather than evidence of their effectiveness during an influenza pandemic. It is difficult to extrapolate the results of clinical trials involving seasonal influenza to Tamiflu’s effectiveness during a pandemic. During 2008, the Department developed a business case to establish a stockpile of antivirals and pre-influenza pandemic vaccine. The business case included increasing antiviral medicines to 80% population coverage in a two-stage process. Despite there being only limited evidence and widespread disagreement among regulators and other bodies internationally on whether Tamiflu confers any benefits on complications and mortality, the business case used an assumption that there would be a 40% to 50% reduction in complications and mortality. This assumption was based on advice from a range of experts including the Department’s Scientific Pandemic Influenza Advisory Committee. Recommendation: Before spending the £49 million to maintain a stockpile at 50% population coverage, scheduled for 2013-14, the Department should review the appropriate level of population coverage; the level of stockpiling in other countries; and take into consideration the fact that the patent for Tamiflu expires in 2016. 8. The Department wrote off £74 million of Tamiflu as a result of poor record-keeping by the NHS on how the medicine had been stored during the 2009 influenza pandemic. During the pandemic, Tamiflu was distributed to many places around the country. When unused stocks were returned, it was not clear whether they had been stored, as required, below 25C. The Department has put in place additional guidance for the storage of antivirals following distribution during a pandemic. Recommendation: The Department should seek assurances that bodies involved in the distribution of antiviral medicines during a pandemic follow the Department’s revised guidance and have robust storage and quality-control systems in place. 2 Cochrane Collaboration (June 2012), Response to oral evidence taken before the Public Accounts Committee: Clinical Trials – Tamiflu. 8 1 The availability of clinical trial results and the sharing of information between the MHRA and NICE 1. All new medicines require a licence. UK-only licences are granted by the MHRA. European Union (EU)-wide licences are granted by the European Commission with the licensing process coordinated by the EMA. In England, some medicines are also appraised by NICE to assess the clinical and cost-effectiveness for use in the NHS. Clinical trials on humans are the key source of information used to understand the safety and efficacy of a medicine. The majority of clinical trials are undertaken, or sponsored, by the medicine manufacturer. 3 2. On the basis of a Report by the Comptroller and Auditor General, we took evidence from the Department of Health (the Department), the MHRA and NICE on: the availability of clinical trial results; how the MHRA and NICE share information; and on the stockpiling of Tamiflu. We also took evidence from Dr Ben Goldacre and Dr Fiona Godlee, Editor-in-Chief of the British Medical Journal. 3. Research first highlighted the issue of unpublished clinical trials in 1986.4 Dr Goldacre noted that an NHS National Institute for Health Research review in 2010 estimated that the chance of completed trials being published is roughly half. Trials with positive results were about twice as likely to be published as trials with negative results. Dr Godlee supported this view, telling us that the pharmaceutical industry published more positive results than negative results from their trials. She noted that the British Medical Journal had published very clear summaries of systematic reviews of data on individual medicines or classes of medicines where, “when you add together the published and unpublished evidence, you get a very different picture of the quality and effectiveness of those drugs.”5 4. A number of clinical trial registries have been established including those by the EMA and the National Institutes of Health (NIH) in the USA.6 The EMA registry contains only those trials conducted in the EU since 2004. The NIH registry, launched in 2000, contains both USA trials and trials conducted outside the USA. However, neither of these registries covers all clinical trials conducted worldwide.7 5. Witnesses agreed that the results of clinical trials should be available for wider scrutiny by medical researchers and clinicians.8 This would help to scrutinise and support the work done by regulators during the licensing process and allow clinicians to make informed 3 ‘Access to clinical trial information and the stockpiling of Tamiflu’, National Audit Office, HC 125 Session 2013-14, May 2013. 4 Qq 1-2 5 Qq 1, 9, 10-11; C&AG’s Report para 1.3 6 C&AG’s Report para 2.7 and 2.13 7 Qq 33-34, 79; C&AG’s Report para 2.7 and 2.13 8 Qq 3-9, 12, 31-33, 59, 78-79 9 decisions about which treatments are best.9 In 2012, the EMA announced plans to release clinical trial data pro-actively once the licensing process has been completed. The policy was expected to come into force in January 2014 but has now been delayed—and in any case only covered trials conducted after 2014 so it would therefore do nothing to improve the evidence base for currently used treatments.10 In 2013, GlaxoSmithKline and Roche announced their commitment to share clinical study reports with personal information removed.11The MHRA told us it is “confident” it had access to “all the relevant data” and that, if a company does not provide all the data, the MHRA has enforcement powers to insist on it. The MHRA told us that both at European level and at national level, when the agencies’ scientific assessors look at the dossiers that companies bring forward, if there are additional points of information that the agencies require, a list of questions to the applicants goes back and “there is the force of European law transposed into UK law that that data shall be provided”. The MHRA told us that this includes data relating to incomplete trials, negative trials, or studies carried out for indications apart from the one for which a licence is being sought, and that “any information that bears on the safety and efficacy of that product has to be made available to the regulator in order to support a marketing authorisation”.12 6. Dr Fiona Godlee agreed that the MHRA and the EMA were entitled to all the information but added: “They are entitled to it. They haven’t asked for it”. Dr Godlee described the regulators as “busy, under-resourced and stretched” and told us: “They tend to take the manufacturers’ word for it. It is only when slightly obsessive and very scientifically determined people, like the Cochrane Collaboration and others, actually go in and look under the bonnet, and begin to see that there are not only 15 trials of Tamiflu but 123 trials of Tamiflu, of which 74 are entirely Roche-funded, Roche-controlled - Roche has the data - that you begin to see the madness of this situation: that we are getting a very partial, incomplete, misleading picture of the effectiveness of many drugs.”13 The MHRA considered that there was no legal obstacle preventing it from sharing information with NICE.14 NICE confirmed that it does not have the same legal powers as European regulators to demand information from manufacturers.15 However, in Germany, a legal obligation has been placed on manufacturers to provide the Institute for Quality and Efficiency in Healthcare, which performs a similar role to NICE, with a full list of clinical trials and supporting clinical study reports.16 NICE told us that it seeks confirmation from the medicine manufacturer’s UK medical director on the completeness of information submitted for technology appraisals, but accepted that the UK medical director may not be 9 Qq 4-6, 79, 100 10 Q 64; EMA announcement: http://www.ema.europa.eu/ema/index.jsp?curl=pages/special_topics/general/general_content_000555.jsp&mid=WC0 b01ac0580607bfa 11 Q 30 12 Q 50; Roche subsequently announced a new global policy on clinical trials dat,: www.rochetrials.com/dataSharingPolicy.action 13 Qq 15-19 14 Qq 58-59, 64 15 Qq 13, 56 16 Q 13; C&AG’s Report para 2.17 10 aware of all trials globally.17 NICE confirmed during the hearing that it would, in future, ask for confirmation that information was complete at a global level although the wording of the new declaration which medical directors are now required to sign still does not require companies to hand over full methods and results on all trials conducted on the treatment being reviewed.18 17 Qq 105-106 18 Q 108 11 Stockpiling Tamiflu and the management of the stockpile 2 7. The number one risk on the Government’s national risk-assessment for civil emergencies, ahead of both coastal flooding and a major terrorist incident, is the risk of pandemic influenza. Antiviral medicines contain an active substance which interferes with the influenza virus, stopping it from spreading. Between 2006-07 and 2012-13, the Department spent £560 million on antiviral medicines for use in an influenza pandemic £424 million on Tamiflu and £136 million on Relenza. Just under 40 million units of Tamiflu were purchased.19 8. The MHRA is confident that it, and other European regulators, received all relevant information during the licensing process for Tamiflu.20 However, this was questioned in a written submission following the hearing by the Cochrane Collaboration, an international network that undertakes systematic reviews of primary research in healthcare and health policy. The NHS National Institute for Health Research had funded the Cochrane Collaboration to conduct a review of the effectiveness of Tamiflu. Having first requested complete reports of each clinical trial on Tamiflu in 2009, the Cochrane Collaboration team is now receiving full clinical study reports from Roche, the manufacturer of Tamiflu, which will allow it to complete its review on the efficacy of Tamiflu.21 9. There is a broad consensus that Tamiflu reduces the duration of influenza symptoms and also reasonable consensus on its ability to prevent illness, in some situations. However, there is a lack of consensus over the efficacy of Tamiflu to reduce influenza complications, including pneumonia, and to reduce mortality.22 Clinical trials for Tamiflu were undertaken on people suffering from seasonal influenza. Complications and death are rare outcomes in a seasonal influenza outbreak, making it difficult for these clinical trials to establish efficacy over these outcomes. Pandemic influenza can be much more severe, as demonstrated by the 1918 pandemic, meaning judgement needs to be used about Tamiflu’s efficacy during a pandemic.23 10. During 2008, the Department developed a business case to establish a stockpile of antivirals and pre-influenza pandemic vaccine. The business case included increasing antiviral medicines to 80% population coverage in a two-stage process. Despite there being only limited evidence and widespread disagreement among regulators and other bodies internationally on whether Tamiflu confers any benefits on complications and mortality, the Department used an assumption of a 40% to 50% reduction in complications and mortality in its case to increase the antiviral stockpile to 80% population coverage.24 The assumption was based on the modelling of previous pandemics and followed advice from a 19 Q46; C&AG’s Report para 3.1 and 3.30-3.31 20 Q 50 21 Cochrane Collaboration submission to the Committee, 20 June 2013. 22 Qq 21, 46; C&AG’s Report para 11-12, Figure 4 23 Qq 100-102 24 Qq 91-92; C&AG’s Report para 3.22 12 range of experts including the Department’s Scientific Pandemic Influenza Advisory Committee.25 The business case also showed, under an alternative scenario, only minimal additional benefits from increasing the stockpile from 25% to 50% population coverage. This was due to assumptions in the modelling that the most at-risk groups would be targeted first.26 11. We asked the Department why it had written off 6.5 million units of Tamiflu at a cost of £74 million. The Department explained that these medicines had been distributed to many places around the country at the time of the 2009 pandemic. When unused stocks were returned, it was not clear whether they had been stored, as required, at below 25C. The Department had told the receiving sites about the need to store Tamiflu below 25C, but they had not had the equipment to do this because the pandemic had happened quickly. The Department told us that, as it had been a cool summer, it had kept the Tamiflu and would have used it if it had been needed. The Department confirmed that it had disposed of the stock only when it reached the end of its shelf-life.27 In 2010, the Department issued revised guidance to primary care providers on the correct procedures for storing antivirals.28 25 Qq 93-95 26 C&AG’s Report para 3.22 27 Qq 47, 49 28 C&AG’s Report para 3.31 13 Formal Minutes Wednesday 18 December 2013 Members present: Mrs Margaret Hodge, in the Chair Mr Richard Bacon Stephen Barclay Guto Bebb Jackie Doyle-Price Chris Heaton-Harris Meg Hillier Fiona Mactaggart Nick Smith Justin Tomlinson Draft Report (Access to clinical trial information and the stockpiling of Tamiflu), proposed by the Chair, brought up and read. Ordered, That the draft Report be read a second time, paragraph by paragraph. Paragraphs 1 to 11 read and agreed to. Conclusions and recommendations agreed to. Summary agreed to. Resolved, That the Report be the Thirty-fifth Report of the Committee to the House. Ordered, That the Chair make the Report to the House. Ordered, That embargoed copies of the Report be made available, in accordance with the provisions of Standing Order No. 134. Written evidence was ordered to be reported to the House for printing with the Report. [Adjourned till Monday 13 January at 3.00 pm 14 Witnesses Monday 17 June 2013 Page Dr Ben Goldacre, Wellcome Research Fellow in Epidemiology, and Dr Fiona Godlee, Editor in Chief, British Medical Journal Ev 1 Una O’Brien, Permanent Secretary, Department of Health, Sir Andrew Dillon, Chief Executive, National Institute for Health and Care Excellence, Professor Sir Kent Woods, Chief Executive, Medicines and Healthcare Products Regulatory Agency, and Professor Dame Sally Davies, Chief Medical officer, Department of Health Ev 9 List of printed written evidence 1 Roche Ev 17 2 Medicines and Healthcare Products Regulatory Agency Ev 19 3 Cochrane Neuraminidase Review Group Ev 19 4 National Institute for Health and Care Excellence Ev 23 5 Department of Health Ev 23 15 List of Reports from the Committee during the current Parliament The reference number of the Government’s response to each Report is printed in brackets after the HC printing number. Session 2013–14 First Report Ministry of Defence: Equipment Plan 2012-2022 and Major Projects Report 2012 Second Report Early Action: landscape review HC 133 Third Report Department for Communities and Local Government: Financial sustainability of local authorities HC 134 Fourth Report HM Revenue & Customs: tax credits error and fraud HC 135 Fifth Report Department for Work and Pensions: Responding to change in jobcentres HC 136 Sixth Report Cabinet Office: Improving government procurement and the impact of government’s ICT savings initiative HC 137 Seventh Report Charity Commission: the Cup Trust and tax avoidance HC 138 Eighth Report Regulating Consumer Credit HC 165 Ninth Report Tax Avoidance – Google HC 112 Tenth Report Serious Fraud Office – redundancy and severance arrangements HC 360 Eleventh Report Department of Health: managing hospital consultants HC 358 Twelfth Report Department for Education: Capital funding for new school places HC 359 Thirteenth Report Civil Service Reform HC 473 Fourteenth Report Integration across government and Whole-Place Community Budgets HC 472 Fifteenth Report The provision of the out-of-hours GP service in Cornwall HC 471 Sixteenth Report FiRe Control HC 110 Seventeenth Report Administering the Equitable Life Payment Scheme HC 111 Eighteenth Report Carrier Strike: the 2012 reversion decision HC 113 Nineteenth Report The dismantled National Programme for IT in the NHS HC 294 Twentieth Report The BBC’s move to Salford HC 293 Twenty-first Report Police Procurement HC 115 HC 53 Twenty-second Report High Speed 2: a review of early programme preparation HC 478 Twenty-third Report HC 297 HM Revenue & Customs: Progress in tackling tobacco smuggling Twenty-fourth Report The rural broadband programme HC 474 Twenty-fifth Report The Duchy of Cornwall HC 475 Twenty-sixth Report Progress in delivering the Thameslink programme HC 296 Twenty-seventh Report Charges for customer telephone lines HC 617 Twenty-eighth Report The fight against Malaria HC 618 16 Twenty-ninth Report The New Homes Bonus HC 114 Thirtieth Report Universal Credit: early progress HC 619 Thirty-first Report The Border Force: securing the border HC 663 Thirty-second Report Whole of Government Accounts 2011-12 HC 667 Thirty-third Report BBC severance packages HC 476 Thirty-fourth Report HMRC Tax Collection: Annual Report & Accounts 2012-13 HC 666 Committee of Public Accounts: Evidence Ev 1 Oral evidence Taken before the Committee of Public Accounts on Monday 17 June 2013 Members present: Mr Richard Bacon (Chair) Guto Bebb Jackie Doyle-Price Chris Heaton-Harris Fiona Mactaggart Nick Smith Ian Swales Justin Tomlinson In the absence of the Chair, Mr Bacon was called to the Chair. ________________ Amyas Morse, Comptroller and Auditor General, David Moon, Director, National Audit Office, Ashley McDougall, Director, Parliamentary Relations, NAO, Marius Gallaher, Alternate Treasury Officer of Accounts, HM Treasury, were in attendance. REPORT BY THE COMPTROLLER AND AUDITOR GENERAL Access to clinical trial information and the stockpiling of Tamiflu (HC 125) Examination of Witnesses Witnesses: Dr Ben Goldacre, Wellcome Research Fellow in Epidemiology, and Dr Fiona Godlee, Editor in Chief, British Medical Journal, gave evidence. Q1 Chair: Good afternoon, and welcome to this session of the Public Accounts Committee in which we are taking evidence on the National Audit Office Report on Access to clinical trial information and the stockpiling of Tamiflu. We will be joined later by officials from the Department of Health, the Medicines and Healthcare products Regulatory Agency and the National Institute for Health and Care Excellence, but first we have a pre-hearing with Dr Ben Goldacre, author of Bad Pharma and other books, and Fiona Godlee, the editor-in-chief of the British Medical Journal. You are both very welcome. Dr Goldacre, may I start with you? To what extent are the concerns about access to clinical trial data for Tamiflu, however important they may be, just an example of concerns about access to clinical trial data more generally? Dr Goldacre: Tamiflu is just one small microcosm. One of the things that is very striking about Tamiflu is that, at no stage, did Roche break the law, and I think that is because the law is broken. Our best available estimate right now, from an National Institute for Health Research review in 2010 summarising the results of dozens of studies that have been done on missing data, is that half all the trials that have been conducted and completed for the medicines we use today have not gone on to be published and, worse than that, that trials with positive results are about twice as likely to be published as trials with negative results. So this is a deeply ingrained cultural blind spot for medicine, but it is one we have known about since 1986. Q2 Chair: That was going to be my next question: if this is such a big problem, how long have we known about it? You say 1986. Dr Goldacre: It was first well documented quantitatively by Simes in 1986. He did a study of all the trials in one field of cancer and found that an enormous proportion of those had not been published and made available to doctors, patients and researchers. In 1986 he called for there to be a comprehensive public trials registry—a list of all the trials that have been conducted and completed—in the hope that that could be used as an audit tool. He also called for all trial results to be published. Twentyseven years later, we have still not realised those ideals. Q3 Chair: What I do not understand is this. I am not a scientist, and I was not trained as a scientist, but the scientific method entails gathering all the available data, subjecting it to as much criticism as possible and then, in the words of Karl Popper, proceeding by conjectures and refutations—by testing propositions and hypotheses and finding out whether they stand up to criticism and scrutiny. Why would there be any interest in departing from the scientific method in this way and from having the information widely available for criticism? Dr Goldacre: I very strongly agree, and I am amazed and surprised that it has never been fixed by my own profession or by industry or policy makers. There are these very odd views around. People will say, for example, “It’s okay because regulators see all this information,” but we see from Tamiflu that different regulators and different organisations around the world have come to different overall conclusions about the benefits of Tamiflu in terms of, for example, hospitalisation or the rate of complications with influenza infection. We can see that there are discrepancies in what different organisations see that Ev 2 Committee of Public Accounts: Evidence 17 June 2013 Wellcome and British Medical Journal have had access to more information, although we do not know whether they have had access to complete information, so we know that regulators and so on are not perfect, and we would not expect them to be. Q4 Chair: That, surely, is the central issue. It is not that regulators are pernicious—plainly they are there to protect the public. The issue is that they are staffed by human beings, so why would one expect them to be more or less likely to see things or miss things than anybody else? Dr Goldacre: I very strongly agree. It is a central principle of science, as you say. The Royal Society’s motto is “Nullius in verba”— Q5 Chair: Sorry, we do not do a lot of Latin in this Committee. Can you say that slowly? Dr Goldacre: On the word of no one—nullius in verba. That is because, in science, we do not trust what you say because you have a white coat or have letters after your name but because you are clear and open about the methods of your experiment and about the results, and because you explain why you think they support your hypothesis. Q6 Chair: And so that those views can be subject to criticism or even attack. Dr Goldacre: Yes, to critical appraisal. Thinking perhaps unkindly, one could argue that regulators around the world, and even health technology appraisal bodies, may not have wished for their overall summary decisions to be open to public scrutiny by other doctors and researchers. I hope that is not true, and I strongly suspect, actually, that this is just a very peculiar cultural blind spot. It is obvious that if you delete half the data points from within one study to make the line go where you think it ought to in a 15-year-old’s GCSE science experiment, people would recognise that that was research misconduct. For some reason, we do not recognise intuitively in medicine that when you withhold the results of whole studies from the public record, that produces biased overall apparent benefits of treatments, even though we know that the very next thing that happens after you publish one study is that it is added together with all the others to get the overall summary. Q7 Chair: I have one more question and then I want to bring in Mr Swales. You mentioned regulators and health appraisers having this blind spot. Earlier you said your own profession—the medical profession and doctors—has a lot to answer for. Could you just expand on that? Dr Goldacre: We set up a campaign at alltrials.net in January to try to address this problem. It was striking to me that there has never really been a concerted public campaign by senior academic and medical professional bodies. We have now got the support of 50,000 individuals, more than 100 patient groups and most of the medical and academic professional bodies in the UK, but some were slow to sign up, and a couple still have not. I cannot understand why anybody in any medical and academic professional body would not recognise that doctors, researchers, patients, payers and policy makers need to ensure that we have all the evidence openly available so people can make informed decisions about which treatment is best. We are not asking for private patient information to be put in the public domain, and we are not asking for anything that could be reasonably said to affect anybody’s commercial IP. It is very simply a matter of making sure we have all the evidence to make informed decisions and that all that information is in the public domain, to the extent that professionals can openly scrutinise it and challenge each other’s interpretations, as we would see anywhere in science. Chair: Mr Swales. Q8 Ian Swales: Could I build on this a little? We are talking about science and the needs of patients. What we have not talked about yet is commerce. We have not talked about commercial confidentiality and we have not talked about money. Will you say a bit more about what other motivations you think might be in play around this issue? Given the way you describe it, why are we where we are? I suspect there may be some other reasons. Will you say what your view is on those two issues—finance and commercial confidentiality? Dr Goldacre: First, it is important to recognise that it is not only the industry who have conflicts of interest. We know that trial results are routinely withheld in academia as well when there are no commercial interests—perhaps a surgeon has a particular interest in their particular surgical method being the best—so people can have conflicts of interest that are not financial. I suppose it is reasonable to say that companies do not want to proactively disseminate unflattering information about their products. That goes without saying. Sometimes companies try to argue that there is commercially confidential information in, for example, clinical study reports. To be absolutely clear, we are not asking for information about how a drug is manufactured. We are asking only for the information that sheds light on how well it works in patients so we can make an informed decision not just about whether it works, but about which is best and about the relative benefits. Patients are disadvantaged if they get the lesser of two available drugs in one class. I personally do not believe that it is ever acceptable, if we are thinking proportionately, to say a company should be allowed to withhold unflattering information about their drug on the grounds that it is commercially confidential in order to maintain sales. I do not believe that any patients would say that, and I think you would struggle to find medical and academic professionals who would say that. But although you would struggle to find anybody who says that clearly and publicly, you do find people muttering about commercial confidentiality in ways that then become very difficult to pin down. Q9 Ian Swales: Do you have any evidence that, for either Tamiflu or any other drug, unflattering—as you call it—information has been withheld, either from the public or from regulators, which might have affected Committee of Public Accounts: Evidence Ev 3 17 June 2013 Wellcome and British Medical Journal the decision about whether to grant a licence for a particular drug? Dr Goldacre: To be absolutely clear, I am more interested in what doctors, researchers and the public see than in what regulators see, because we know that regulators see more, although they may not necessarily see everything. I feel uncomfortable talking too much about individual products because this is such a huge, systemic problem. The best currently available evidence is the NIHR review of 2010, which summarises the results of dozens of studies on this topic. We know that right across the board, in all fields of medicine, trial results are routinely withheld and trials with positive results are more commonly published. It would be unreasonable to pull out anyone because it is a systemic problem. Q10 Ian Swales: It is. Equally, it is a human issue, isn’t it? There is a system there, but there are motivations. I am trying to explore why it is like this. Dr Godlee, do you have anything to add on why you think information would be routinely withheld? Fiona Godlee: I think that we do have increasingly clear evidence that both the medical journal system and the pharmaceutical manufacturing and drug development system are causes of the current situation, historically over a period of time. We have a situation where research has been put to use for marketing purposes, rather than for science, and we have evidence for that, as Ben says, across the board. I could pull out individual cases of drugs or manufacturers where that has happened, and you probably have in front of you evidence of that sort. But if you want us to give details, I can certainly do that. Medical journals rely on funding from reprint revenue from randomised control trials or clinical trials that have been published, which the pharmaceutical industry buy to use to market their drugs. We know that the pharmaceutical industry—and who can blame them?—are there to create value for their shareholders, and from their trials they publish more positive results than negative results. Again and again, we have evidence of unpublished results that, when combined with the published results, make the drugs suddenly look not only no longer effective but harmful. Q11 Ian Swales: Can I just finish this point? You are making quite a serious allegation. In the last couple of minutes, you have twice said that you have evidence. Fiona Godlee: This is published in the public domain. The British Medical Journal and other journals have published very clear summaries of systematic reviews of data on individual drugs or classes of drugs where, when you add together the published and unpublished evidence, you get a very different picture of the quality and effectiveness of those drugs. Q12 Ian Swales: And you strongly believe that this is for commercial reasons? That was the point of my original question: it is not to do with scientific secrecy. Fiona Godlee: I can see no scientific reason for hiding data. These are trials that have been done on members of the public. The drugs are being bought with taxpayers’ money. This is information that doctors need in order to provide safe and effective treatment for their patients. Withholding those data is a form of misconduct, and we have to see it in that light. Q13 Chair: Dr Godlee, on that subject, may I raise the letter that you wrote to the National Institute for Health and Clinical Excellence on reboxetine, Pfizer’s drug? You said: “As a vocal fan of NICE since its inception, I am sorry to see you outshone by another organisation that has shown the necessary muscle when confronted with drug manufacturers who withhold clinical trial data.” You went on to talk about IQWIG, which is the German equivalent of NICE. You said that when IQWIG “realised that it was not being given the full story on Pfizer’s drug reboxetine, it told the company that it would only approve the drug for reimbursement if all the data were provided. Pfizer delivered up the data, nearly three quarters of which had never been published. Analysis of the full dataset showed the drug to be ineffective and possibly harmful. Although this was a bad outcome for the company, I think you will agree that it was an important victory for public health.” That is an extraordinarily serious charge in relation to the German equivalent of NICE, which makes decisions on whether something is value for money and whether the German taxpayer should be funding it. You are saying that NICE does not have or does not use that muscle. Is it a case of NICE not having that power or not using that power? Fiona Godlee: It does not have that power and perhaps to some extent—I do not know—it may not use what it does have with enough force. Certainly IQWIG, which is the German equivalent of NICE, has statutory rights to obtain information. In this case, Pfizer, the manufacturer of reboxetine, was declining to provide the full clinical trial data, and IQWIG was aware that there were other trials and asked for them. As is recorded in my letter, about three quarters of the information had never been published. When the data were combined, this drug, which was being used by people with depression, was found to be not only ineffective but harmful. That is an example of another regulatory system. One has to emphasise the enormous effort that IQWIG had to go through. IQWIG’s staff were put under enormous pressure by the pharmaceutical industry in Germany. Even though that is a triumph for the public interest and patient safety, it was still done at enormous cost. I want to segue into the huge, ridiculous Alice in Wonderland situation we have at the moment, where academics and regulators are doing a piece of investigative work for information that should be in the public domain. Q14 Chair: Indeed, but what I fail to understand is that we were told by the NAO that, although NICE does not appear to have access in the way that IQWIG in Germany does, the MHRA does have access and pharmaceutical companies have a legal obligation to give the MHRA all the data. Fiona Godlee: If I may just turn to Tamiflu, because I have the specific figures on that, as I understand it Ev 4 Committee of Public Accounts: Evidence 17 June 2013 Wellcome and British Medical Journal from the Cochrane Collaboration—I should give them full credit for this, because Tom Jefferson and his team at the Cochrane Collaboration have done an amazing job to uncover this, along with Deborah Cohen at the BMJ. They have discovered that there are about 123 trials of Tamiflu, not all of which are therapeutic trials; Roche is aware of 74 completed trials. Of those, I have been told that the European Medicines Agency received 15 incomplete accounts of trials and NICE received four incomplete accounts of trials. That is the sequence of events. The EMA is able to demand additional information, but NICE is not. NICE is reliant on the EMA. It can go back to the drug company, but it cannot demand it. Q15 Chair: What I don’t understand is, the EMA and the MHRA, as licensing authorities, are pretty much in the same position, are they not? The EMA is the European one and the MHRA is the UK one, but they are performing the same function of licensing, and they are both, are they not, entitled to all the information? Fiona Godlee: They are entitled to it. They haven’t asked for it. Q16 Chair: They don’t ask for it? Fiona Godlee: They don’t ask. Q17 Chair: So they, as licence providers, once they have been through their processes, are not making decisions on the basis of an entire data set, but on part of the data set. Why? Fiona Godlee: I would ask the same question: why? That is the question the Committee has to address. Why do the public, why do those making clinical decisions for patients on their behalf, not have full access to all the information? That is a piece of detective work. It is madness; it has to change. Q18 Chris Heaton-Harris: Does the FDA have better information? Fiona Godlee: The FDA probably have the same rights to the information. The FDA is staffed up with more internal investigative, statistical expertise than the EMA, and therefore they attempt to do a lot more of the analysis themselves. There are differences, but in terms of the amount of information they have, I think they have access to the same. They can get all the information if they ask for it, but I think they are overburdened and, as Ben has said, we can’t expect the regulator always to get it right. Q19 Chair: One of the things the Report says, in paragraph 7, comparing the FDA and the European agencies, is “The main difference is that the FDA asks for patient-level data upon which it may conduct its own analysis. The European agencies would require more analytical capacity to be able to do this.” That is to say, not more analytical capacity to ask for the data, but more analytical capacity to do something with it. Are you saying that, because they don’t have the analytical capacity that the FDA has to do something with it, they think, “Well, there’s no point in asking for it,” so they don’t get it in the first place? Fiona Godlee: You will have to ask the EMA about why they don’t ask. I think they don’t ask because they think they have it all. They are relying on the drug companies to provide them with information, and the drug industry provides them with what they do, and they are busy, under-resourced and stretched, so they say, “That’s great; thanks for this,” and they tend to take the manufacturers’ word for it. It is only when slightly obsessive and very scientifically determined people, like the Cochrane Collaboration and others, actually go in and look under the bonnet, and begin to see that there are not only 15 trials of Tamiflu but 123 trials of Tamiflu, of which 74 are entirely Rochefunded, Roche-controlled—Roche has the data—that you begin to see the madness of this situation: that we are getting a very partial, incomplete, misleading picture of the effectiveness of many drugs. Q20 Chair: I should have said at the beginning that Roche had originally made clear their willingness— indeed, keenness—to attend this Committee, but they pulled out on Friday. I don’t know why, so people will have to draw their own conclusions. Looking at your letter to NICE, you say, “Now that serious doubts have been raised about the evidence behind claims” for Tamiflu’s “effectiveness and safety”—which is a serious claim—“I am asking you to withdraw approval” for Tamiflu “until NICE has received and reviewed the full clinical trial data”. What is the current situation? What response have you had from NICE to that request? Fiona Godlee: We got a very good, prompt and thorough response from Michael Rawlins, in response to that letter, which we also published. He said that he would do a further review within NICE, to see what was needed, and at the moment we have not yet received that; so he has spoken, I think it is fair to say, and also we have had a response from the researchers who did the review for NICE. NICE relies on external academic groups to do the systematic reviews for them, in some cases, and the team that did that have responded. They are convinced that they have seen all that they need to see, and that therefore NICE’s review was adequate. I think I and others are less convinced of that. Q21 Chair: If I were to summarise the opinion, or opinions, on Tamiflu and its effectiveness, would I be roughly accurate if I were to say there is a very broad consensus that Tamiflu is effective in reducing flu-like symptoms; there is a quite broad consensus—but not as broad—that it has some effectiveness in prevention of illness, or prophylaxis, although that consensus became smaller once the Cochrane group withdrew; and there is still less consensus, or probably, it is fair to say, no consensus, on the extent to which Tamiflu helps with preventing complications such as pneumonia and things that lead to death? Fiona Godlee: That is a very good summary. I think most people would agree with that. To add a bit of a gloss, as far as I am aware, Tamiflu has only ever been compared with placebo. It has not ever been compared directly with, for example, paracetamol, or indeed a stiff whisky or something similar, so we are left with some doubt about its effectiveness in prevention. Committee of Public Accounts: Evidence Ev 5 17 June 2013 Wellcome and British Medical Journal Q22 Chair: You make me wonder how much whisky we could have bought for £560 million. Fiona Godlee: Well, absolutely. It is a good question. It depends on the quality of the whisky. Q23 Chair: Yes, indeed. Can I be clear? I imagine either of you can comment on this. Why did we, the taxpayers, spend £560 million? Actually, I think some of it was spent on Relenza; £424 million was spent on Tamiflu. Why did we spend £424 million on Tamiflu? Was it so that we could reduce flu-like symptoms by a couple of days? Fiona Godlee: I will give you my brief answer; I am sure Ben has views on this. I think it was politically expedient. There was an outbreak of potentially serious influenza. There was a World Health Organisation recommendation that countries should do this. I should say that there is a whole host of information about what led to that WHO recommendation, with industry-funded advisers helping WHO reach that decision. The UK was confronted with a situation in which it wanted something. There isn’t anything else for pandemic flu. To cut a long answer short, I would say it was bread and circuses to keep the populace happy, and I think it was misleading and wrong, especially as the alternative, paracetamol, is well understood, and Tamiflu has adverse effects, apart from its cost. Dr Goldacre: It is fair to say that it may have been a combination of wishful thinking and perhaps an ambition to reduce public panic in the event of a dreadful pandemic, but when we fail to recognise that there is uncertainty about the effects of treatments, we also miss opportunities to resolve that uncertainty. In 2008 and 2009, for example, when we were giving out Tamiflu on telephone lines to people who wanted it, I think we had the opportunity to conduct a cluster randomised trial that would have helped to resolve the uncertainty about whether Tamiflu really does reduce the rate of complications and hospitalisation. Not only have we failed to make an informed decision and to allow independent doctors and researchers to scrutinise the decision that we made—full chemical study reports minus individual patient data have not been made publicly available—but by allowing ourselves to believe that we could be fairly certain, we have lost the opportunity to conduct a simple cluster randomised trial at low cost, which would have helped reduce that uncertainty and answered this question. Q24 Ian Swales: Can I build on a specific? The Report says on page 16 that the European Medicines Agency issued infringement proceedings against Roche last October following inspection by the Medicines and Healthcare products Regulatory Agency, because the MHRA found that a significant amount of safety data gathered by Roche on 46 medicines authorised in the UK, including Tamiflu, had not been fully reported. First, is Roche a rogue company, or can we assume that everybody operates like this? Secondly, should the public be worried that a company is withholding safety information to that extent? Dr Goldacre: The Roche investigation, as I understand it, is looking at the withholding of about 80,000 or 85,000 individual reports. These kinds of infringement happen. I think there is an investigation going on at the moment that was supposed to report by now or by the end of June. To me, this is not an issue of safety; it is an issue of effectiveness and relative effectiveness. Q25 Ian Swales: So it is not just safety data? You think we are talking about efficacy as well? Dr Goldacre: My primary concern is that when we withhold the results of clinical trials, we are withholding information about effectiveness. I worry that that is one reason why this issue has not received enough attention. When there is a drug that does more harm than good, or information about harm is withheld, that is easier for TV producers and newspaper journalists to grasp hold of and appreciate, but I think that just as much harm is done when we spend money on drugs when we are not sure what the benefits are, or when we allow a situation to arise in which we are misled about the relative benefits of two treatments. Q26 Ian Swales: To answer my question, if the MHRA go into another company besides Roche, do you think that they will find the same story about commonly solved medicines? What I am really trying to get at is: are Roche a particular case, or is this how the industry as a whole works? Dr Goldacre: Roche, more broadly, to give one example, are no longer members of the ABPI and do not follow— Q27 Chair: Sorry, could you say that name again? Dr Goldacre: The ABPI—the Association of the British Pharmaceutical Industry. It is the UK pharmaceutical representative body. Q28 Chair: And did Roche pull out of it, or were they kicked out of it? Dr Goldacre: They did not do well in one particular case, and then they left, so they are not subject to the ABPI’s self-regulation framework. But that is still only self-regulation and, to be absolutely clear, it is preposterous, in my view, that this issue of access to clinical study reports is left in the hands of selfregulation with no prominent legislation. Ian Swales: Are you saying that they are even a rogue in the world of self-regulation—they have now left the UK self-regulation framework? Q29 Chair: For the record, could you say the name of the body again? Fiona Godlee: The Association of the British Pharmaceutical Industry. Dr Goldacre: To be clear, they are quite a problematic body in themselves. After I drew attention to these problems in my book, they issued a statement in which they said that all these problems are historic and they have all been long addressed, which is not— Chair: We are very used to that answer in this Committee on a whole range of things. Dr Goldacre: I can imagine that. Ev 6 Committee of Public Accounts: Evidence 17 June 2013 Wellcome and British Medical Journal Q30 Chair: Apart from Roche, are all the other major manufacturers in the pharmaceutical industry members of this association? Dr Goldacre: I believe so, yes. But, to be absolutely clear, there are no good grounds at the moment to believe that any company or group of individuals is any better than any other for withholding access to clinical trial results, with the possible exception of GlaxoSmithKline. As you may know, they have recently made a clear commitment to share clinical study reports—with personally identifiable information removed—going back to their foundation at the beginning of 2000, 2001. And they have signed up to our alltrials.net campaign, so they are proactive on this. That is still at the status of a promise thus far, and I am always very sceptical and cautious, but they are making all the right sounds. Q31 Ian Swales: May I finish on this paragraph? The NAO Report says that a timetable has been agreed for releasing this safety data. Then, it says, “There were no immediate safety concerns”. Those two ideas seem out of step—if we have not got the safety data, how do we know that there are not any safety concerns? My original question was: should the public be worried by this sort of framework? Dr Goldacre: The public should be more worried about the fact that there is no competent legislation to ensure that doctors and researchers have access to all the methods and results of all the trials that have been done on all the uses of all of the treatments that we are currently prescribing in medicine. The fact that we do not have simple, clear, obvious pathways for access to this information, so that we could have a serious, professional discussion about which treatments work best, means that we cannot practice evidence-based medicine in the way that we purport to. Q32 Ian Swales: Just thinking about solutions to this, how would we do this in a way that was not tainted by commercial interests? What would your recommendation be going forward? It is one thing to make all the data available, but what would we do with it and who would do that? Dr Goldacre: We already have people in place who go through and summarise this information: independent academics; researchers; doctors; and the Cochrane Collaboration. There are 14,000 academics worldwide working on this stuff, making summaries that are gold standard and used almost universally by doctors and payers and researchers in different parts of the world. The key issue is access to this information, and I think that we have had a spectacular lack of ambition on the part of policy makers, who I think have not been adequately informed about these problems by regulators, but also by regulators and professional bodies. It would be very straightforward to fix. Q33 Ian Swales: It seems to me that, given the quantity of work and the balance of power, this is one for international collaboration. Thank heavens we are in the EU—sorry, that was an in-joke—but should we be looking to form collaborations in a wider way, so that this work is done once around the world? How would that work? Dr Goldacre: The issue of who should summarise this stuff is secondary; there are huge numbers of people out there who already summarise the more limited information that is already made available, so that is not a problem. The problem is getting access to the full methods and results of all these trials, and there is a very simple solution. The European Medicines Agency has a clinical trials database, which, like all clinical trials registries, is an administrative legacy project. People think that it is a list of all the trials that have been done on all the uses of all the medicines that we currently prescribe, but in fact it is a list of all trials conducted in a European country since March 2004; that is a much smaller set of trials and it is hopeless. In America, there is another registry, which, again, is incomplete, but incomplete by design. There has been a huge lack of ambition. The European Medicines Agency could very straightforwardly say—the legislation is up for revision in two years—to everybody who markets a drug anywhere in Europe, “Look, we have the European trials register, and, as you know, it is a bit different from everybody else’s. It contains all the results of all the trials that have been conducted on all the uses of all the medicines that are currently being prescribed. Here are the forms. Please write down all the information and we will put it online and make it publicly available.” Q34 Ian Swales: Even if the trial took place outside Europe? Dr Goldacre: Increasingly, trials are conducted in Brazil, Russia, India and China, such as trials conducted on antidepressants for anxiety and PTSD. We need all that evidence to make informed decisions. It is absolutely mind-boggling that nobody has ever sat down and said, “Why don’t we ask?” People are preoccupied with how the world would cave in and people would refuse to hand such information over, but we have not tried asking. The discussion around access to such information often becomes heated because, inevitably, it only happens when people think that something funny is going on, like what happened with reboxetine or Tamiflu, but actually this is a banal, routine issue that covers the whole of medical practice, and we should take the heat out of it by having universal legislation. Q35 Chair: Specifically in relation to the influenza pandemic, what would you have done differently to prepare for the risk of an influenza pandemic? Dr Goldacre: First, I would have ensured that all the methods and the results of all the trials that had ever been done on all of these treatments were made publicly available, so that all the most intelligent and well trained pharmacoepidemiologists in the world would have been able to scrutinise it, because regulators and health technology appraisal bodies miss signals. That is not because they are stupid—they are very clever, highly motivated and well trained—but these are difficult problems. If we look at some of the biggest problems spotted in medicine, such as those with Vioxx, rosiglitazone—Avandia—and the evidence base for Tamiflu, they were spotted not by regulators but by independent doctors and researchers, Committee of Public Accounts: Evidence Ev 7 17 June 2013 Wellcome and British Medical Journal who often had to work very hard, sometimes even using leaked information, to work out where the problematic signals were. Regulators miss things. I would put everything in the public domain, so that we could have as many good people as possible working on these issues. Secondly, I would use the fact that we, as a Government, are paying for this stuff to roll it out in the context of a pragmatic, clustered, randomised control trial, so that where there is uncertainty, we would generate new knowledge. That is actually something that we as the NHS are very bad at doing. We have huge opportunities in the UK, because it is common for health care to be provided free at the point of access by the state, but it is fairly unusual for it to be provided through one administrative body with universal coverage and electronic health care records that cover everybody. We could routinely be saying, wherever there is uncertainty about new medicines, “We are only approving this for use if it is in the context of a randomised control trial,” and we will then resolve the existing uncertainties about how good or bad it is. I am amazed that we have never been sufficiently ambitious about doing that. We invented the basic notions of evidence-based medicine in the ‘50s, ‘70s and maybe ‘80s and then we sat back and awarded ourselves a point as if we had rolled it out properly, but we have not: we leave uncertainties unresolved and vitally important information to be analysed behind closed doors only. We have to fix that. Q36 Chair: Dr Godlee, do you have anything to add? Fiona Godlee: I would agree that, faced with a pandemic, which is a scary thing—the public are worried and the Government want to look like they are doing what they should be doing—secrecy does not serve us well. The claim that this information is “commercial in confidence” is something that we should absolutely challenge. Obviously, the molecule or drug is patented, which is fine—that is shareholder value—but the information about whether the drug works is something that should be in the public domain if the public are paying for it. We need legislation to ensure that that happens. Obviously, if it were happening voluntarily, we could do without legislation, but that is not what the situation is. Can I add to the reply to Ian Swales’s question about what the solution might be? Unless we can find a solution to the “commercial in confidence” problem, we have to recognise that the pharmaceutical industry has an irreducible conflict of interest in relation to the way it represents its drugs in science and in marketing. Unless we can resolve this in a way that is more in the public interest and in patients’ interests, I would argue that drug companies should not be allowed to evaluate their own products. Q37 Chair: They independently? Fiona Godlee: Yes. should be evaluated Q38 Chair: We are a value-for-money Committee: we don’t look at policy; we want to know if things are effective, efficient and economic. With something like this, knowing whether it works or not— Fiona Godlee: Is essential. Q39 Chair: Is essential. Obviously, we as a Committee are not equipped to look at the scientific case, but we want to make sure that the architecture that makes that happen—that ensures that things happen effectively, efficiently and economically— does so. Did the Comptroller and Auditor General want to come in? Amyas Morse: I have a question, but before I ask it, I should say that I think I need to declare a slight interest; I had a very brief moment as a member of what I think I am right in remembering was called the Flu Pandemic Board—they wanted to have some nonexecs on it—and I think I attended one meeting of it. I need to say that, leading into another point. Q40 Chair: This was in a previous incarnation? Amyas Morse: Yes. Actually, interestingly enough, it was, but it was cut short by my changing into this incarnation. Now, I would like to ask something. Not setting aside anything that you have been saying about publication of research or anything of that sort, as a practical matter, if you find yourself in the situation where you have got World Health Organisation guidance, are we saying that we think it was a faulty decision to have supplies of Tamiflu available or not? Fiona Godlee: For the UK? Amyas Morse: Yes, for the UK. That is what we are addressing. Fiona Godlee: I can see the difficulties of the politicians and the medical officers involved in that decision. I absolutely understand that they had a problem. They wanted to be able to provide something. In the end, we do not know whether what they provided was value for money, and that is the problem your Committee faces: we just do not have the answer. That is a situation that I guess nobody wants to be in in the future. We want to be able to say either we are going to spend £500 million on stockpiling a potentially useful drug based on this information, or we are going to spend the £500 million on something else. You do not have the information in front of you to decide whether that was the right decision or not; nor do I. What I do know is that paracetamol is cheap and easy and has known side effects, whereas Tamiflu is relatively expensive and has an unknown safety and effectiveness profile. That is the information I can provide. Q41 Fiona Mactaggart: I think I am hearing from you, Dr Godlee, that Tamiflu was used by politicians as a placebo, in effect. That is what it sounds like. What I am concerned about is the point that Dr Goldacre made right at the beginning when he said that he didn’t think that at any stage Roche had broken the law, but the law is broken. Both of you are arguing that this requires a change in the law. What I don’t think I understand is why the regulatory bodies can’t say, “We are not going to authorise your drug to have NHS money spent on it unless we have 100% of all the trials.” Why can’t they do that? I have not quite Ev 8 Committee of Public Accounts: Evidence 17 June 2013 Wellcome and British Medical Journal understood, from all the papers I have read, why they can’t do that, if they had some balls—excuse my language. Dr Goldacre: That is not what we have asked them to do. As a society, through our legislative frameworks, we have asked regulators to make a decision about whether a drug is better than nothing in order for it to come on the market. We have not asked them to ensure that the information that they see is made publicly available, so that doctors, researchers and other investigators can make their own judgment about the quality of that information. My impression is that everybody here is very happy to follow the current regulations and nobody has had any ambition to change them. Regulators, for whatever reason, have not come to you and said, “We don’t think it is enough for us to see this stuff behind closed doors,” and obviously the companies have not asked for legislation requiring greater transparency, but actually, having spent a lot of time speaking to them—some of them behind closed doors—in trying to get them to sign up to our AllTrials campaign, my impression is that these organisations are rule followers. They often do not have a senior medic on their main board of directors; they are auditors, accountants, managers—people who will do whatever is asked of them in order to conduct their business. The pharmaceutical industry will do whatever is required of them. To an extent, it is easier for them to operate in a framework in which perhaps they can elide over good quality evidence about the relative risks and benefits of different companies’ treatments—it softens the sting of producing the second or third best treatment in the class if there is some obfuscation about which is the best, so you can still get some return on your investment for making the second or third best drug in class—but I get the impression that they would be very happy to comply with any legislation that was asked of them. There is certainly resistance. They have lobbied hard in Europe to try and stop greater legislation from coming in—they have raised some preposterous canards, like pretending that we have asked for people’s confidential medical records to be posted in public when we simply have not—but I think they would comply with that legislation; it is just that we have never asked that of them. I am disappointed that regulators have not asked for greater transparency and that they have been happy with seeing stuff themselves behind closed doors, but there it is. They haven’t. Q42 Fiona Mactaggart: Are you confident that regulators see enough of the stuff? Fiona Godlee: That is what I was going to say. I think that regulators do not know at the moment. That is because the whole universe of studies done on an individual drug is obscure and difficult to unpick. What we are heading for, in an ideal world, is a situation where we would know that this number of patients have been randomised into this number of trials and these are the summary results. So the AllTrials campaign and others before us have been asking for registration of trials and summary results to be made available. Trial registration now happens, but it is far from complete, and summary results are being made available, but again it is far from 100%. We know what we need to have, and the regulators— over-burdened, under-resourced and doing their best—are not aware of the universe of trials. There was one case, the IQWiG case, of reboxetine in Germany: that was one drug, and for whatever reason they applied themselves to this one drug. It involved a vast amount of work and a huge amount of political effort, and jobs were put on the line because of the pharmaceutical pressure on them. That was just one drug. That is just not sustainable across the whole piece. Q43 Chair: Jobs put on the line in IQWiG? Fiona Godlee: Yes. Q44 Chair: They had to withstand that, as it were. Fiona Godlee: Absolutely. Huge lobbying and pressure from industry to remove them. Q45 Chair: We are running out of time. We have other officials from whom we need to take evidence. In conclusion, can I clarify one point? The NAO Report states: “This memorandum does not attempt to draw conclusions on whether Tamiflu is value for money.” Are you saying that on the present available evidence—what is publicly available—it is not at the moment possible to make a conclusion about whether it is value for money? Fiona Godlee: I think I am saying that it depends on what value you put on public reassurance. There is no doubt that having the Tamiflu there reassured the public, because people were told that it was effective. I do not know if that is true. If people meant it to be effective in reducing infection and symptoms, I am afraid I cannot tell you whether that is value for money. It may have been politically value for money, but whether it was medically value for money is highly dubious. Chair: Thank you very much indeed. Thank you for your time. Committee of Public Accounts: Evidence Ev 9 Examination of Witnesses Witnesses: Una O’Brien, Permanent Secretary, Department of Health, Sir Andrew Dillon, Chief Executive, National Institute for Health and Care Excellence, Professor Sir Kent Woods, Chief Executive, Medicines and Healthcare Products Regulatory Agency, and Professor Dame Sally Davies, Chief Medical Officer, Department of Health, gave evidence. Chair: We now move to part two. We are joined by Sir Andrew Dillon from the National Institute for Health and Care Excellence. You have changed your title— Sir Andrew Dillon: Again, yes. Chair: So we can’t call you NICE any more, although I’m sure you are. Sir Andrew Dillon: You can. It’s still NICE. Q46 Chair: We are joined also by Professor Dame Sally Davies, who is the chief medical officer at the Department of Health; by Una O’Brien, who is the permanent secretary and whom we have seen many times before; and by Professor Sir Kent Woods, who is the chief executive of the Medicines and Healthcare Products Regulatory Agency. You are all very welcome. May I start with you, Ms O’Brien? Are you satisfied with the regime and the completeness and transparency of your access to data on drugs such as Tamiflu? Una O’Brien: I would start by saying that clearly there is a range of evidence about the effectiveness of Tamiflu. I think you rightly alluded to it in your questions of the previous witnesses, and I thought your summary was exactly as we would see it. Obviously, I am here this afternoon with three of the best technical experts in the country, so we can get into the detail of your questions as far as you wish to go, but what I want to say at the outset, if I may, is that the risk of pandemic influenza is the No. 1 risk on the Government’s national risk assessment for civil emergencies. It is ahead of coastal flooding, ahead of a major terrorist incident, so it is a matter that is taken profoundly seriously within the Department of Health in the way we plan, model, take account of scientific evidence and take account of historical data. It is because of this, and it is in that context, that we have to make judgments about the relative merits of all the medicines that could be available, whether that is antivirals or the ability to plan for the development of vaccines, which unfortunately can take up to six months after a flu outbreak occurs. So the main point I want to make at the outset is that this is a matter we take extremely seriously and we have to balance our judgments about the scientific evidence and the value for money in the context of the scale of the risk and uncertainty that we are planning for. Q47 Chair: You’ve said that you take it “profoundly seriously” and “extremely seriously”. Can you then account for the way in which so much of the Tamiflu stockpile was destroyed because of very basic errors about how it should be stored? That does not sound to me like profound seriousness at all. Just to remind the Committee and the members of the public, 6.5 million units were written off, at a cost of £74 million, due to poor record keeping by the NHS about their storage environment. It was not that they had been poorly stored—in other words, in a fridge at 8° or below—but that the people who had stored them did not know whether they had been stored in a fridge at 8° or below. They might have been or they might not have been, so because the information simply was not available, £74 million of public money went on tablets that were thrown away that might, perfectly possibly, have been still okay to use, but we didn’t know. That doesn’t sound to me profoundly serious. Una O’Brien: The first thing to say is that they were actually not thrown away. What you are referring to is the technical write-off in the accounts. Sally can give you more details in a moment, but my understanding, having looked into it, was that when those medicines were distributed at the time of the pandemic in 2009, they were distributed to many, many different places around the country, and when they were returned—those that were not used—we did not have the records to prove, totally reliably, that they had been kept at room temperature, and because of that, they were technically written off, although they were in fact kept until their expiry date— Q48 Chair: And then thrown away. Una O’Brien: That is, in effect, what happens with a stockpile. The nature of a stockpile is that you maintain the things for the period that they are viable and then— Q49 Chair: And then they stop being viable and then you destroy them. That is inevitable in the process; I think the Report uses the word “inevitable” somewhere. My point was that if your attitude to this whole area is one of profound seriousness and extreme seriousness—those were your words—why would you not ensure, in that spirit of profound and extreme seriousness, that something as basic and as important as how they were stored was equally taken profoundly and extremely seriously? Why would you not ensure that? Una O'Brien: We obviously learned that lesson. The key thing here is the evidence of how they were kept. There was not sufficient to enable us to say that we could absolutely nail them when they were returned to us, bearing in mind that they were distributed thinking that they would in fact be used. Let me ask Sally to fill in on the detail to complete the answer to your question. Dame Sally Davies: Tamiflu needs keeping at under 25°, and Relenza at under 30°, so that is room temperature, and it was not a hot summer. Ideally, and in planning for the future, we need to make sure that we have those spiral temperature measures that can reassure us that it was kept at that temperature. Despite telling the receiving sites that that was what was needed, they did not have the equipment because the pandemic happened quickly. We were not able to say definitely that they had been kept below 25°, but it was a cool summer. We kept the drugs and we would have used them if people had needed them. Ev 10 Committee of Public Accounts: Evidence 17 June 2013 Department of Health, NICE and MHRA Q50 Chair: May I just give Sir Kent Woods and Sir Andrew Dillon an opportunity to answer my first question: to what extent are you satisfied with the completeness and transparency of your access to data from Roche on Tamiflu? Sir Kent Woods: I am confident that we had access to all the relevant data, because there is legal force to require that. If a company does not provide us with all the data, we have enforcement powers to insist on it. The licensing of Tamiflu was carried out by the centralised European route, through the European Medicines Agency. Both at European level and at national level, when our scientific assessors look at the dossiers that companies bring forward, if there are additional points of information that we require, a list of questions to the applicants goes back and, as I have said, there is the force of European law transposed into UK law that that data shall be provided. Whether the data relates to incomplete trials, negative trials, or studies carried out in other indications apart from the one for which a licence is being sought, it matters not. Any information that bears on the safety and efficacy of that product has to be made available to the regulator in order to support a marketing authorisation. Q51 Chair: At request? To go back to our earlier hearing, can you clarify the difference between the different kinds of information that may be supplied? You, we were told earlier by the NAO, have the legal right to obtain this information. That is to say, a pharmaceutical manufacturer is legally obliged to give you information. They are legally obliged to give you all the information about the drug. Are they legally obliged to give you all the information about the drug that they hold, or are they legally obliged to give you all the information about the drug that you ask for? Sir Kent Woods: All the information that they have about that product when they bring it forward for marketing authorisation. Q52 Chair: Including any trials that have been conducted on it anywhere in the world? Sir Kent Woods: Yes. Q53 Chair: And the results of them? Sir Kent Woods: Yes. Q54 Chair: So if they had a series of trials and they told you only about some of them rather than all of them, they would be in breach of their legal obligation to you? Sir Kent Woods: Indeed. Q55 Chair: And it is not a case of what you did or did not ask them for; you asked them for everything? Sir Kent Woods: Yes. Q56 Chair: Can we talk about the relationship between you and NICE. NICE, Sir Andrew, does not have the legal right to obtain everything, does it? Sir Andrew Dillon: No, we do not have the same legal powers as the MHRA, but then of course we do a different job. NICE is concerned about the comparative effectiveness of treatment, so we rely on the work done by the regulatory body, which is concerned about the safety, efficacy and quality of treatments. We are generally only interested in a subset of the data that is published or available in clinical studies that have been run on a particular drug. We begin from the position that the drug has an effect which is sufficient for it to justify its licence and it is safe— Q57 Chair: You start from that position as it were because the MHRA has given its imprimatur, so you know you can start from that position, or at least you think you can. Sir Andrew Dillon: Yes, we start from that very confident position. Q58 Chair: Sir Kent, when you receive from a pharmaceutical manufacturer all the information that it has about a drug, are you, in some sense, prohibited from handing that information on to anyone else? Do you have any restrictions through some form of confidentiality agreement that prevent you from handing it on to others, including to other public bodies such as NICE, or not? Sir Kent Woods: It is a very important, and to some extent, changing area. Under the 1968 Medicines Act, the regulatory staff were explicitly precluded from passing information that they had gathered, as part of their regulatory work, to other parties. That has been repealed, and under the regulations that were introduced in 2012, that constraint is removed. We do not have a problem with sharing data with, for instance, NICE, as a public body. Q59 Chair: When you say that you don’t have a problem, you mean that you don’t have a legal problem. You are able to do it. Sir Kent Woods: That’s right. There are two issues. One is that there may be, within the data that we receive, some information which is personal confidential information. There may be identities of individuals in there—in the clinical study report or elsewhere. So, to that extent, information cannot be put in the public domain directly without being redacted. The second and very vexed issue is about what is commercial in confidence information. It has been argued—and is still argued in some quarters—that with some clinical trial data, before the regulatory decision to license or not to license has been made, that clinical trial data will contain commercial in confidence information. Our own view is that, when there is commercial in confidence information in a dossier, it usually relates to manufacturing and quality, and not necessarily to the clinical trials evidence, so our constraints on releasing clinical trials data are much more around data protection issues and redacting personal information than they are about commercial in confidence issues. We would certainly support the view that the clinical trials data supporting a licensing decision should be put in the public domain once the licensing decision has been taken. Q60 Ian Swales: Surely that is a trivial issue, isn’t it? Taking somebody’s name out and calling them Committee of Public Accounts: Evidence Ev 11 17 June 2013 Department of Health, NICE and MHRA patient A or patient B—you’re not seriously saying that that causes prevention of publication of important information. I know it is a clerical job, but surely you cannot accept the argument that that information cannot be published because it has somebody’s name in it. Sir Kent Woods: Can I give you some figures on that? I am, at the moment, the chairman of the European Medicines Agency, as well as my executive role in the MHRA, and since 2010, the EMA has adopted a policy of data transparency. We have released approximately 1.9 million pages of documents. It has cost the agency around €3 million to do that, and most of the work is the labour of going through thousands and thousands and thousands of pages to redact the information. Q61 Ian Swales: Why not make the manufacturers do it? Why not only accept the information if they have already anonymised it? They have plenty of money. Surely they don’t get through the door if they give you that kind of—that is an excuse, isn’t it, which we surely shouldn’t accept? Sir Kent Woods: We don’t think that the regulator should be acting as the archivist here. If the company is going to go to an appraisal body such as NICE, seeking a market for its product, it should provide the information in toto needed to support a decision. I think that is the responsibility of the company, and I quite agree with you. I think what is needed for the future is clarity and the redaction at source of any information— Q62 Chair: When you say at source, do you mean by the company? Sir Kent Woods: Yes. Q63 Chris Heaton-Harris: I looked up your biography before I came, Sir Kent, so I know you have a European hat as well. Therefore, you see 26 or 27—soon to be 28, and maybe a few more—cases of best practice, where similar regulators across Europe have a relationship. The main recommendation of the Report is that you should work better together. What are the barriers to stop you from doing that at this time? Sir Kent Woods: Do you mean better with NICE? Chris Heaton-Harris: With NICE, sorry. Sir Kent Woods: We have had, for a long time, a memorandum of understanding with NICE. In fact, it predates the MHRA and goes back to the Medicines Control Agency. We are perfectly willing to revisit and revise that, but in practice, the working relationship between the MHRA and NICE has been entirely constructive, and we have not had situations where we have had to refuse to provide NICE with information that it wishes to have. We both report to the Secretary of State. To that extent, we are not talking about the public dissemination of information; we are talking about the availability of information between one part of the health system and another. Q64 Chair: Just for the avoidance of doubt, there are no legal impediments that prevent you sharing information with NICE. Sir Kent Woods: I do not think there are. Having said that, the policy of clinical trial data release at the EMA has currently been brought to a halt by a restraining order put on it by the European Court, at the request of— Q65 Chair: Can you be clear about which European Court you are talking about? Is it the Court of Justice? Sir Kent Woods: The Court of Justice, yes. The reason for that is that two companies have claimed that their rights are being infringed. Q66 Chair: Which ones? Sir Kent Woods: InterMune is one. I forget the name of the second but I can let you know. Q67 Chair: If you could write to us, that would be great. Sir Kent Woods: Those companies have asked that the EMA should not release data until there has been a court hearing. Q68 Chair: Mr Moon, do you have the other name? David Moon: I think Ben might. Dr Goldacre: It is InterMune and AbbVie. Sir Kent Woods: So, when I say that I do not know of any legal impediment, I have to say that that particular matter is sub judice. Q69 Chair: Sir Andrew, when you are doing your work—and you have said your work for NICE is a different job from the MHRA—do you at times have occasion to ask the MHRA for information, that you know they will have that you do not have? Sir Andrew Dillon: Our first port of call is the company, because after all they have supplied the information to the MHRA in the first place, so they have it. We have already established a dialogue with the company, because we are engaged with them in an appraisal. They will already have supplied us with the initial data set that we asked for. If we need to go beyond that, if we think there is something that they have not supplied, or we want to explore the data they have given us in more detail, we go to the company first of all. We have not been in the position where our work has been frustrated to the point where we have had to terminate an appraisal because we have not been able to get that additional data from a company. Q70 Chair: One thing the Report makes clear in paragraph 7 on page 5 is that the MHRA and the EMA—the European Medicines Agency—do not ask for as much information as the United States FDA, the Food and Drug Administration, at the initial application stage. The main difference according to the Report is that the FDA asks for patient-level data upon which it may conduct its own analysis. It goes on to say, “The European agencies would require more analytical capacity to be able to do this.” I think that means that the European agencies would require more analytical capacity to conduct their own analysis, not that they would require more analytical capacity to be able to ask for patient-level data. That is an important difference. Is it correct that you ask Ev 12 Committee of Public Accounts: Evidence 17 June 2013 Department of Health, NICE and MHRA for less information than the FDA? Is the reason you do so because it would not be of any use to you because you do not have, in the words of the Report, “the sufficient analytical capacity” that you would require to do something with it? Sir Kent Woods: The analytical capacity that is used to assess a dossier that comes in for a centralised licence at the EMA is actually provided by the member state agencies. For any application, there will be two member state agencies—the rapporteur and corapporteur—that will take on responsibility for doing that assessment. Q73 Chair: In my nice big pile in the corner there is everything, and you are not asking for everything, are you? Sir Kent Woods: If you take a clinical study that a company brings forward to support the efficacy of its product, there is a clinical study report, which is an enormously detailed summary of the study they have done and the results they have obtained. Those results are in summary form rather than being the raw data. Q71 Chair: Have you been in the position of having to talk to yourself when you have brought forward something for potential approval? You are saying that in this case the MHRA would bring it forward to the European Medicines Agency for consideration. Sir Kent Woods: No. If a company makes an application to the EMA the scientific committee—the Committee for Medicinal Products for Human Use— will decide two member states to lead on that assessment. So the analytical capacity actually comes from the member states. We certainly have statistical expertise within the MHRA. Not every agency does, but within the European network there is the analytical capacity, but perhaps not on the scale that the FDA has provided it. We will use that analytical capacity if we wish to probe further. Q75 Chair: You are legally able to get it if you want it. Sir Kent Woods: Indeed. Q72 Chair: You must first have the information. This appears to suggest that you do not ask for as much information in the first place because you do not have the analytical capacity. Those are quite separate things. I will give you an analogy with the US Justice Department and the information that cigarette companies were forced to disclose about tobacco. Ms O’Brien is nodding; she will know what I am about to say. I think there were 11 million pages of information. We looked at the evidence in this Committee, and I think we are about to look at tobacco smuggling again at some point in the not too distant future. Once it was dug into—because there were 11 million pages it took some while to do that— it became quite clear that tobacco companies were taking account of illicit duty, not paid routes to market, in determining their overall advertising budgets. They were all caught bang to rights, and the only reason why that was possible was because all the data was disclosed. I am a lay person and a nonscientist, and most lay people’s starting point would be, “Give me every known thing in the human universe about this drug and I will put it in the corner and I will say, “That is everything that the human species knows about Tamiflu”, and then we will decide how we deal with it. But you don’t do that, do you? You ask for less because you have less analytical capacity. That is what it says in paragraph 7. Sir Kent Woods: No, it is not so much about how much—we are talking about the level of detail that the company is asked to provide. Q74 Chair: But you could get the raw data if you wanted. Sir Kent Woods: We could. Q76 Chair: So it is a question of whether you ask for the raw data or not. Sir Kent Woods: Absolutely. If we were not satisfied with the analytical approach that the company had taken, we could go back to the level of the raw data and do it again, but we do not routinely do that. Q77 Fiona Mactaggart: I should say that I have been to a number of events that have been sponsored by the ABPI. People might think that that might influence my views here, but one of the things I am interested in is that it sounds as though it would be possible to ask for that data. We heard from our earlier witnesses that if that data was publicly known, other people would do certain bits of analysis, such as a number of GPs who feel that something does not match their experience. That is the kind of thing that could trigger a different analysis and a challenge. Using the wisdom of masses might help us get more effective analysis of clinical trials. The present system prevents that from occurring, does it not? Sir Kent Woods: I disagree with that. There are three levels of transparency, and it is important to understand what is implied by each of them. At the first level, we need to know that a study has been done and we need to know its results. That is absolutely a given. Secondly, the regulator needs to see the clinical study reports of every one of those trials, which provide the summarised data. Q78 Fiona Mactaggart: Does that include the ones that companies abandon because they think they will not like the results? Sir Kent Woods: Absolutely. If it provides relevant information on risk and benefit, we need to know it. That will include safety data, for instance. The third level of transparency is down to individual measurements of blood pressure, temperature or whatever of every single patient randomised in that trial. I do not think that it would help the general public to have that information available. It might well be necessary for academics to have access to that patient-level data, if they wished to carry out a systematic review—a meta-analysis—using individual patient data. That is the most refined form of metaanalysis, which is done from time to time when there is an absolutely crucial question and you need to draw Committee of Public Accounts: Evidence Ev 13 17 June 2013 Department of Health, NICE and MHRA information from every single trial that has been done. The individual patient-level data will not be of great help to the general public. Q80 Chair: They took no data from published studies? Dame Sally Davies: No, and I could go on. Q79 Fiona Mactaggart: I was not suggesting it be made available to the general public, because I am one of those who simply would not understand it. I was suggesting that one issue is relatively expert clinicians feeling that the reality they are seeing challenges the conclusions to which you have come, and I think there have been such cases. Therefore, those clinicians may want to have the data to be able to underpin their sense that, “If I am practising in south Wales, there may be something in the south Wales gene pool that produces a different result.” If they could see the full data, they could, from their experience, challenge the conclusions to which you have come. I do not see a good reason for not doing it. Dame Sally Davies: Perhaps I could come in on this. We have a system of the regulator looking at efficacy and safety and NICE looking at cost-effectiveness. Most of our clinicians—we are talking generally about doctors, as the drug is only prescribed—do not have either the time or the expertise to do the analysis that you are talking about. Fiona Mactaggart: Indeed. Dame Sally Davies: Therefore, we set out, as a system, to give our patients and public the best service we can in terms of efficacy, safety and costeffectiveness. I, with all my colleagues, have worked hard on transparency; the Government believe in that. Personally, in the mid-2000s I led a lot of work at the WHO on their clinical trial registry work to agree the domains and to ensure that the WHO work lined up with clinicaltrials.gov in the States, with EMEA— now EMA—and with everyone else. Registration is the most important thing, so that we know what trials have been done, and that is improving. Summaries of results are useful. Individual patient data is important when a researcher who understands the methodology has a good hypothesis to test. What we are now working to do increasingly is making that data redacted and often available only in a safe haven to protect the patient’s privacy, which is paramount so that these things can be contested and looked at. Let me turn to the Cochrane Collaboration. We fund that from the National Institute of Health Research, which is the funding route for the Department of Health for clinical, evaluative and applied policy and research, and I hold that budget. The Cochrane Collaboration was set up initially by us in the mid’90s. We fund over half of the collaborative editorial groups; I have been an editor. I believe strongly in this, and I believe in the hierarchy that you have descriptive studies, going up to case control—better than that; it is a randomised control trial—and then you have meta-analysis. But there are different ways of looking at it. Having read it, I do not believe— Kent and Andrew may have other views—that the Cochrane review on Tamiflu is the last answer, not just because they do not have all the data, but, first of all, because they made up their hypotheses once they had got data, and that is not standard research practice. They extracted data from 25 studies but excluded 42 and took no data from published studies. Q81 Chair: I thought the whole point was that they withdrew the non-published information. Dame Sally Davies: Yes, but then they left out what was published. Q82 Chair: Can you write to us about this? It is very interesting. I would like to see a longer, more detailed and more forensic explanation of what you were just saying than is possible in the time available in this Committee. Dame Sally Davies: Absolutely. We will. I want to finish by saying that we continue to fund this and we believe in an open debate. We were going to close the funding down because Roche had not agreed to supply the data. They have now agreed, so we are continuing to fund them in order to allow the open debate. Chair: May I bring in the Comptroller and Auditor General, and then Mr Swales? Q83 Amyas Morse: I just have a basic, auditor-type question. Are you limited at all by resource as to the amount of work you can undertake, or at least the style of work that you can undertake? We have seen that in other areas of our work. Is your budget going up or down, taking one year compared with another? Please may I ask that question to each witness, just to establish whether you are resource-constrained or not? Sir Kent Woods: The MHRA operates as a trading fund. Under the will of Parliament, we levy fees for the work that we do. These are statutory fees that are reviewed every year. The resource that we use is not from the taxpayer; it is from those who are regulated. We are fortunate in that because there is then a relationship between the work that we have to do and the resource that we have to do it. The issue is not that we are so resource-constrained that we cannot do the job properly. We are seeking to do the process of regulation efficiently and we do not think it would be efficient to re-do analyses that a company has done when we can probe, if we wish, to make sure that it has done that properly. It really is a matter of regulatory efficiency rather than doing a rush job because we do not have the resources to do it properly. Sir Andrew Dillon: Our budget is going down but we are becoming more efficient, so we have been able to protect our investment. Q84 Mr Bacon: You cannot levy fees in the way the MHRA can. Sir Andrew Dillon: No, we do not, but we have been able to protect the capacity to engage with companies on this kind of issue, so I feel quite confident that we have the resources to do our particular job. Q85 Mr Bacon: Professor Davies, as chief medical officer, what is happening to your budget? Dame Sally Davies: Actually, as chief medical officer, I have no budget. Ev 14 Committee of Public Accounts: Evidence 17 June 2013 Department of Health, NICE and MHRA Q86 Mr Bacon: You set a fine example to us all, and I want to put that on the record. This Committee has no budget either. Dame Sally Davies: With respect to pandemic planning, if we make the appropriate business case— I highlight that we did so right through to the Treasury—we can buy what we need. Q87 Ian Swales: Perhaps Sir Kent could answer this. Returning to paragraph 2.10, which I referred to earlier, I do not know whether to be worried or encouraged by the fact that the MHRA prosecuted someone for falsifying clinical trials and has found two companies—GlaxoSmithKline and, most relevantly today, Roche—not giving up important safety information. Clearly, it is encouraging that you found that, but what does it tell us about the climate out there, why did you go into those companies, and how often do you do this kind of work? In other words, what does that section tell us about what you do? Sir Kent Woods: We have a division that is committed to inspection, enforcement and standards. It has a finite capacity, so we are using its resources in a riskbased way. Over the last couple of years, we have developed some statistical algorithms, which we think are world class, to allow us to spot the factors that would point us in the right direction. For example, if we see information from a company that does not tally with what we know, if a company has a poor inspection history, or if we have soft intelligence from one place or another, that is where we would prioritise our inspection resource, and that is precisely how this particular matter came to light. It was because the company in question had gone through a major reorganisation and had completely redeveloped its pharmacovigilance database that we spotted those as potential risk factors. We carried out the inspection and the result is as you see. Q88 Ian Swales: To repeat the question I asked the previous witnesses, do you think, particularly regarding the second bullet point about Roche, the 46 medicines and the safety issues, that the public should be worried, are they technical administrative matters you found that we do not need to worry about, or do we not know? Sir Kent Woods: When the information came to light, we asked two immediate questions. First, was there a public health issue that we did not know about? The medicines on which there have been some failures of response on safety data were medicines about which we already knew a great deal, and the question was whether the additional data that we had not seen would alter our estimate of the risk-benefit. We were able to look fairly quickly at that, and when we did not see an immediate safety concern it was for exactly that reason. There is a deeper question about whether it was an error or there was culpability, and that is a legal matter. As the infringement proceedings are going ahead, I cannot comment, but it illustrates first that we inspect, secondly, that if anything is amiss we find it, and thirdly, that we have legal powers. Q89 Ian Swales: If I am an independent researcher or a pharmaceutical company, how often can I expect a knock on the door from people in your organisation? Sir Kent Woods: I cannot give you a quantitative answer to that. We are of course responsible for pharmaceutical production, not just in the UK and not just in Europe, but around the world. I think there are 42 countries, at the last count, manufacturing the raw materials for medicines that come into the European Union in 1,600 different sites. It is a large task and we put our resource where we think it will most likely deliver the goods. Q90 Ian Swales: I think it is particularly research that we are talking about here. I do not know where the Chair has got to in the proceedings, but are we going on to Tamiflu itself? Chair: Yes, by all means. Please ask about that while I confer with Ms Doyle-Price. Q91 Ian Swales: Okay, I am now referring to sections 322 and 323 of the Report, which talk about the evidence for the effectiveness of antivirals. I do not know whether Ms O’Brien or Mr Dillon would be the best person to answer this, or maybe the chief medical officer. Section 323 contains some quite worrying statements in terms of the decision that was taken. This is about the Scientific Pandemic Influenza Advisory Committee and the point that the paper you were looking at stated, “there was no published evidence on a reduction in mortality due to antiviral use and very limited evidence on reduced complications and hospitalisations.” Then it says: “The Scientific Pandemic Influenza Advisory Committee…advised that an assumption of 40 to 50 per cent reduction in both hospitalisations and deaths should be used in the modelling” for the paper. This sounds like a straightforward doctoring of the numbers to make a case. How would you respond to that? Dame Sally Davies: Let me try and explain. There was an H1N1 epidemic in 1918, when 25% of the population caught it and 2% of them died, which meant that there were 316,000 deaths. It was massive. I believe that at the time we started our pandemic planning—I was in charge of research, not the CMO—we were worried about an H5N1 epidemic coming our way. Q92 Chair: H5N1 was avian flu—bird flu. Dame Sally Davies: Yes. Which looked as if it— though it did not have much human-to-human transmission—had high mortality rates. So when the Mexican swine flu appeared, which was an H5N1, our experience was from the 1918 epidemic. Put that vision of what a pandemic can do—we do have the figures for the impact on the NHS, on attendance at one site of employment and the economy—and the unknown about how severe it will be, besides the evidence we had, which was not for a pandemic at all; it was for using Tamiflu in seasonal flu, which, outside the risk groups of people who are either elderly or have pre-existing disease, is not usually very severe. We worked with a number of scientific groups to look at what was likely to be the impact, but we were clear Committee of Public Accounts: Evidence Ev 15 17 June 2013 Department of Health, NICE and MHRA that—our modellers and our scientific advisory groups advised that we could have an assumption of 40% to 50% reduction in hospitalisations and deaths, for the modelling to build the business case, but that the evidence was not there because we had not had a serious pandemic. We still have— Q93 Chair: So the 40% to 50% was sort of plucked out of the air, in effect. Is that right? Dame Sally Davies: No, it was no plucked out of the air. Q94 Chair: Well, where was it plucked from? The Report says, in paragraph 15: “The paper stated that there was no published evidence” and you have just said there was no evidence, so how did you come up with a number? If it was not plucked from somewhere, how did it— Dame Sally Davies: It was worked out by looking at historical pandemics, as to what would be reasonable and we had three sets of modellers looking at this. We had scientific groups expert in flu and three sets of modellers—absolutely independent academics; the Health Protection Agency; independent and Government modellers—looking at it, who advised us to work on those assumptions. What I would like to come back to is that, when we reviewed the data after the pandemic, it looked as though there was about that level of reduction of severe cases. So we think that the modelling for a pandemic was actually pretty good. Q95 Ian Swales: We will come back to the post justification in a moment. I certainly take Ms O’Brien’s comment about this being the No. 1 risk. We are not trying to minimise that at all. We are talking about the data on which a £400 million spending decision was taken. It is about the data that we are taking it on. I know about the politics. I wonder if we would have bought as much if it hadn’t had “flu” in the title. Obviously, the public were more—I cannot even remember the name of the other one, which tells you something about brand marketing. We are talking about the process by which this huge spending decision was taken. It sounds very much as though it was a finger in the air. Alongside the earlier evidence about the doubt about the efficacy of this product, I guess we are groping for whether the public should be concerned, not so much about the flu, but about the way we spent this amount of money. That is what this Committee is about. That is really what I am trying to drive at. Dame Sally Davies: This, as I said, came from a standing scientific advisory group, and it was reviewed. We had some scientific work jointly with the Royal Society and the Academy of Medical Sciences. The then Government chief scientific adviser was concerned about whether we were getting it right, and we had a colloquium, bringing all our planning papers, with about 20 external scientists and the Government chief scientific adviser. They all felt—all agreed—that those papers were the best effort that we could make to base our planning on. We used vast amounts of external scientific advice. It was not a finger in the air; it was modelled, and it was carefully done. Q96 Ian Swales: We all remember this was an international issue; in fact, it started in other countries. Are you aware of what other Administrations facing the same issue as your organisation did? Did anybody else buy this amount of this particular drug, or did other people make different judgments? Dame Sally Davies: Other people started to stockpile. We have a history of being very good at planning; the WHO always congratulates us. We were one of the few countries—I would have to write to you to tell you how many did—that put in place an advance purchase agreement for vaccines. We knew that if you get a pandemic, you not only get a new bug; there is no resistance to it in the community, and there are no vaccines. For flu, it takes six months to produce early vaccines, and before you get to a number that will be enough, it is another six to 12 months. We have to protect our public in that first six to 12 months. The only known protection is the antivirals, and we knew that if we waited for a pandemic, everyone would be panicking and demanding them. There would be a three-month running for small supplies. Our history as a nation is to plan and think it through, and that is what we did, but other countries have stockpiles, such as Japan. We have a stockpile of 50% at the moment; Japan has a stockpile of 40%, and France and Germany have stockpiles. Q97 Ian Swales: And that is as a proportion of the population? Dame Sally Davies: Yes. I should point out that Tamiflu makes up only 35% of that, because of course you can get resistance, so we have 15% Relenza. We do not want full Relenza, because you can get resistance to that, it is not licensed for under-five-yearolds and you have to sniff it, so it is quite difficult to give. Q98 Ian Swales: There is also a comment on the same page that there was a scenario indicating that covering 50% of the population would only yield small additional benefits to a stockpile of 25% in a worst-case scenario, and no additional benefits under other scenarios. How did you come to the decision about how much to stockpile, given that some people were saying that if you have enough for 25% of the population, that will solve the problem? You bought double that. Dame Sally Davies: My understanding, though if I am wrong we will write and correct, as I was not party to this, was that we were not only trying to provide treatment for the at-risk groups, but, because it was going to be a pandemic, for a broader group, and actually as prophylaxis, to protect people: to try— which we did try, as you know, with school closures and treating contacts; my daughter was treated—to contain it for as long as possible, in order to give time for the preparations to be put in place on the ground, and as long as possible, or as short as possible, before vaccines became available. So we upped it because we were not just having a treatment for at-risk groups, but a broader treatment and a prophylaxis stockpile. Ev 16 Committee of Public Accounts: Evidence 17 June 2013 Department of Health, NICE and MHRA Q99 Ian Swales: But the suggestion is that that offered no additional benefits. That is the phrase; and some views were that there would be no additional benefits for buying that amount—that doubling the amount was probably not good value. Dame Sally Davies: But without that, we could not have tried prophylaxis. We would have been stuck with just hygiene measures, with school closures. I go back to the fact that, in the event, it was not a particularly serious virus, except if you were one of the pregnant women who ended up on ECMO for eight weeks, or one of the thousands who died—the excess winter deaths were over 3,000—then you would say it was severe; but we needed to try and protect the population. fact, Mr Goldacre talks about the difference between seasonal flu and a pandemic extensively in the book. Sir Kent Woods: If I can follow that point, when you are doing a randomised trial, it is difficult to demonstrate effects against very rare outcomes. For seasonal flu, death is a very rare outcome. For pandemic flu, it is a far more common outcome. To say that you did not demonstrate a reduction in serious illness, hospitalisation or mortality, for a molecule that we know specifically interferes with the replication of the influenza virus, by no means precludes that, and, indeed the fact that death is such a rare event makes it a little difficult to extrapolate a negative outcome from trials in seasonal influenza, to say “Therefore it is not going to work.” Q100 Chair: We have to bring this to a conclusion. Unfortunately various Members need to take part in the Second Reading of the Pensions Bill. Sir Kent, could we just return to this question of who has access to the information; because you said earlier that much of the patient-level data really wouldn’t be of much interest to the general public, which may or may not be true. There might be individuals and groups who might find it useful, although most members of the public probably wouldn’t, and wouldn’t understand it. What I find odd and difficult to understand is, given that you yourselves don’t always ask for a look at the patient-level data, why would one come to the conclusion that this is somehow something which is a sphere where it is okay for only the regulators to be looking at this? The evidence we heard earlier suggests that you are staffed up by well motivated and highly intelligent people, but that, plainly, because you are human beings, you too can make mistakes. Wouldn’t it be better, in terms of critical appraisal, to have a wider pool of eyes looking at this problem than you currently have? Sir Kent Woods: I agree with that sentiment, and we certainly wouldn’t wish to, as somebody said earlier, do our work behind closed doors. The question is whether it is our responsibility to make those data available, or the companies’ responsibility; but we would certainly welcome multiple pairs of eyes. If we take the European system, any guidance, any opinion that comes out of the CHMP has had 27 members states’ experts looking at it; so to that extent there are multiple eyes. I would just like to come back, if I may, Chairman, to this question about the trials evidence; because we must never forget that all those trials we are talking about were done in seasonal flu—and we are planning for a pandemic. The difference between seasonal flu and pandemic flu—and we go back to 1918 as a worst-case scenario: the lethality of that influenza virus was 100 to 200 times greater. So I am uneasy about the suggestion that Tamiflu is a kind of expensive paracetamol for something which is actually like a bad cold. Pandemics are really serious public health problems. Q102 Chair: I do appreciate that. It does, though, make it less scientific, doesn’t it? Sir Kent Woods: One has to make judgments in advance, on the basis of best evidence. I totally agree. Q101 Chair: I think we all appreciate that. The issue is about the efficacy in the event of a pandemic. In Q103 Ian Swales: One question on this: none of us on this side of the table has anything like the medical knowledge that is on the other side of the table, so we can’t really question that kind of thing. I guess we are in the value-for-money game here. If I can just test another angle of that, because you would be able to answer it. Chair: You’ll have to be quick. Q104 Ian Swales: Yes, very quick. Of the 40 million units we have purchased, only 2.4 million have been used in the last three or four years. Was there any different commercial arrangement that would have been possible, that would have both met the need but also managed the finance better for the taxpayer? Una O'Brien: All I can say is that we have sought to get the very best possible procurement and to beat down every last pound. We will continue to do that, particularly when Tamiflu comes off licence in a couple of years’ time. If I may, I think it is important to add that, if you look at the cost and break it down, it is 1p per household per day. We are really in a form of insurance here. On the best evidence available, we are trying to make a judgment, and I would rather be here explaining why we did it than explaining why we did not have a stockpile and why we did not have Tamiflu available to people during the mild outbreak in 2009. I think it is about looking at the whole picture. Q105 Chair: I must say that it is a neat headline—it always is when you come down to 1p when we have a population of 62 million people—but the fact is that a ratio of 40 million units purchased to 2.4 million units used suggests that there might be a little bit of room for improvement. Sir Andrew, we are running out of time, so may I just finish with you? Why doesn’t NICE make it obligatory for manufacturers to confirm at a global level that the information provided for appraisals is complete? Sir Andrew Dillon: When we approach companies and ask them to make a submission to our routine technology appraisal process, we ask the medical Committee of Public Accounts: Evidence Ev 17 17 June 2013 Department of Health, NICE and MHRA director to confirm that, to his or her knowledge, all the relevant information is available for the appraisal. Sir Andrew Dillon: We are going to ask a better question. We will insert the word “global.” Q106 Chair: Yes, I know, but that was not my question. The Roche study in Shanghai was a huge study that Roche in Switzerland apparently did not know about, which is relevant here. The question I asked is why NICE does not ask companies to confirm that all the relevant information is made available at a global level. These are, after all, global companies. You could do that, couldn’t you? Sir Andrew Dillon: Yes. That is the purpose of the question. Q108 Chair: Will you write to us with the question that you are going to ask? Sir Andrew Dillon: Yes, I will. Chair: Thank you very much. I thank you all very much for your time. It is possible that we will return to this at some point in the future, but in the meantime I thank you all very much. We appreciate it. Q107 Chair: Why don’t you ask a better question? Written evidence from Roche 1. Introduction 1.1 Roche is a leading manufacturer of innovative medicines in a range of therapeutic areas, including cancer, rheumatoid arthritis and infectious diseases, such as hepatitis C and influenza. Many of our treatments have changed the standard of care in difficult to treat conditions, extending and enhancing the lives of millions of patients. 1.2 We operate two autonomous research units, as well as 150 research partnerships all over the world, to foster diversity of research and translate science into medicines. In 2012 we invested nearly 8.5 billion Swiss Francs in research and we now have 72 new molecular entities in clinical development. Last year there were 2,280 clinical trials in operation involving Roche medicines, involving 35,720 healthcare centres across the world. In total, 326,642 patients were involved in these trials. 1.3 Clinical trials are critical for determining the safety and efficacy of new medicines and the clinical value of diagnostic tests. They also provide important information on the cost-effectiveness of a treatment or diagnostic test and how a treatment improves quality of life. This information is shared with regulatory authorities and payers in order to gain marketing approval and, ultimately, reimbursement. Roche also publishes the results of our clinical trials through numerous channels, such as peer reviewed journals and online, as we recognise that healthcare professionals, researchers, patients and the public are also interested in knowing about potential new therapies. We work with numerous health authorities, academic institutions and independent researchers across the world in a transparent, open and collaborative manner and we should not be judged solely on the example of one interaction with one review group, during which relationships could have been managed better by both parties. 1.4 Roche provided evidence to the National Audit Office (NAO) to support its inquiry into clinical trial data on Tamiflu. Given the recent publicity relating to our decision around disclosure of patient-level clinical trials data on oseltamivir (Tamiflu), we also extended an offer to the NAO and the Public Accounts Committee (PAC) to share in detail Roche’s data on this medicine, as well as to answer any specific questions the Committee may have regarding the data and to discuss the reasons for the approach we have taken. We also gave both written and oral evidence to the Science and Technology Committee inquiry into Clinical Trials. 1.5 We welcome the report of the NAO Access to clinical trial information and the stockpiling of Tamiflu which concluded that: — All stockpiles of Tamiflu sufficient to cover between 25 and 50% of the populations are cost effective. — Regulators are confident that they receive all the required and requested information from manufacturers when licensing new medicines. — Regulators’ assessments of Tamiflu are broadly in agreement as to its safety and efficacy, despite different approaches towards review. — The UK’s stockpile of anti-virals is in line with international guidance and is likely to be justified even with more cautious assessments of their efficacy. 1.6 In February 2013, we announced a new policy expanding access to our clinical trial data. Within the scope of this policy, we have also begun to provide all Roche-held Clinical Study Reports on Tamiflu to researchers from the Cochrane Collaboration. Ev 18 Committee of Public Accounts: Evidence 2. Tamiflu 2.1 Roche has recently been the subject of concerns raised about transparency of clinical trial data following our inability in 2009 to agree with the Cochrane Acute Respiratory Infections Group the release of patientlevel clinical trials data on Tamiflu. We stand behind the robustness and integrity of our data supporting the efficacy and safety of Tamiflu, which has been shared with all relevant regulators according to their requirements and guidelines. When considering the case of Tamiflu, it is important to note that: — Tamiflu has been reviewed and approved by regulatory authorities in over 80 countries and over 95 million patients have received this medicine since it was first licensed and made available. — Clinical trials and real-life experience from flu pandemics have shown that Tamiflu is effective in reducing the severity and duration of symptoms in those infected with flu and decreases the risk of developing the illness if there is contact with an infected individual. — Various analyses of Tamiflu show a benefit in reducing the duration of symptoms, fever and time to return to normal sleep, health and activities, as well as reducing occurrence of lower respiratory tract complications (including bronchitis) requiring antibiotics in infected patients. — Tamiflu is recommended as a flu antiviral by public health bodies worldwide including the US Center for Disease Control & Prevention (CDC), the European Centre for Disease Prevention & Control (ECDC) and the World Health Organization (WHO). — The US FDA, which requests re-analysable patient-level data, has recently extended the license for Tamiflu, approving its use in children two weeks of age and over. This recent approval further substantiates the safety and efficacy of Tamiflu. 2.2 During the discussions over the stockpiling of Tamiflu as part of the government’s preparations for an influenza pandemic Roche was asked to provide significant amounts of information to UK health authorities such as the Department of Health, the Health Protection Agency and the Scientific Pandemic Influenza Advisory Group. We were also asked to provide information on an often daily basis by UK and international authorities at the height of the 2009 influenza pandemic. Roche complied with each and every request to the very best of our ability, and our willingness to work with authorities in a transparent, open and collaborative way has not been questioned. 2.3 The vast majority of health authorities request specific and extensive information on a medicine when considering whether to grant marketing authorisation. The U.S. Food & Drug Administration (FDA) specifically requests anonymised patient datasets whereas the European Medicines Agency (EMA) does not. The FDA reprogrammes and re-analyses the data in order to verify the analysis performed by the company. The EMA rather interrogates the sponsor and requests additional analysis or reanalysis from the company directly. 2.4 Differing approaches to the assessment of Tamiflu by different health authorities, such as the FDA, EMA and the Japanese Ministry of Health, Labour and Welfare, have resulted in some differences to the license in their respective countries. However, the US, EU nations and Japan have taken a broadly similar approach to the stockpiling of anti-virals as a central part of their pandemic preparedness plans. A high degree of consensus exists as to the safety and efficacy of Tamiflu in the treatment of seasonal influenza, and what its role might be in the event of a pandemic. 3. Tamiflu Clinical Trial Data 3.1 Over the past 15 years Roche has been the sponsor for 81 trials into Tamiflu. Of these, one was terminated before any patients were enrolled, and 74 are now completed. Of the 74 completed Roche sponsored trials, all are now in the public domain either as a primary publication or secondary publication or on Rochetrials.com. 3.2 Roche receives requests regarding the release of clinical trial data from academic and independent institutions worldwide. As part of this, we request an analysis plan and signed confidentiality agreement, given some of the complexities inherent in making available patient-level data which was generated many years ago on the basis of consent forms which were never intended to enable such access. In addition the merit for any request should be assessed to ensure that the pre-planned analyses are based on clearly defined scientific and clinically relevant questions. 3.3 In relation to an initial request from the Cochrane Acute Respiratory Infections Group for access to data on Tamiflu, we provided large volumes of information in 2009 which we believed was sufficient to answer their questions. The reviewers questioned and did not sign a confidentiality agreement. In circumstances where concerns are raised about the detail of a confidentiality agreement, it is usual to investigate alternative arrangements that protect patient confidentiality, commercial sensitivities and provides them with the reassurance they require. In this instance, no such discussion was had, a mutually acceptable position was not reached and therefore patient-level data was not released to the review group. 3.4 We understand and support calls for our industry to be more transparent about clinical trial data. In February 2013, we announced a new policy expanding access to our clinical trial data, which supports the provision of clinical study reports on request and, furthermore, analysable patient-level data in a legitimate environment that ensures patient confidentiality and protects legitimate commercial interests. Within the scope Committee of Public Accounts: Evidence Ev 19 of this policy, we have also begun to provide all Roche-held Clinical Study Reports on Tamiflu to researchers from the Cochrane Collaboration. 3.5 We maintain the highest ethical standards in the conduct of our clinical trials and transparency of our interactions with all external parties for all of our medicines. We recognise, however, that following the debate about Tamiflu there is legitimate policy interest in our data. Roche is confident in the data supporting Tamiflu and this is why we have offered to share Roche’s data on Tamiflu with both the NAO and the Committee, answer any specific questions it may have and discuss the reasons for the approach we have taken. 4. Real World Data 4.1 Roche believes, when considering the overall benefit-risk of a medicine, all available data should be taken into account. This includes both formal clinical trial data as well as “real world” data generated during a medicine’s routine clinical usage. This approach offers important insights into how a medicine can be used to maximum effect, supports evaluations of cost effectiveness, informs pricing and enables authorities to ensure that treatment is delivering value for money. 4.2 There are already good examples of real world surveillance of drug efficacy, although more can and should be done. For example, the WHO conducts detailed global surveillance of influenza resistance to antivirals such as Tamiflu. Trials and real-life experience from the 2009–10 flu pandemic have both shown that Tamiflu is effective in reducing the severity and duration of symptoms in those infected with flu and decreases the risk of developing the illness if there is contact with an infected individual. 17 June 2013 Written evidence from the Medicines and Healthcare Products Regulatory Agency During my attendance at the Public Accounts Committee hearing on clinical trials and Tamiflu, I referred to two cases currently before the European Court involving two companies that were seeking to prevent disclosure of their clinical trials data. I can now provide some further details as clarification of the points I made at the hearing. The European Medicines Agency (EMA) has been ordered by the European Union General Court of Justice not to provide documents as part of two access-to-documents requests until a final ruling is given by the Court. On 26 April, the Agency was ordered to grant interim relief to AbbVie and InterMune and to suspend its decision to release the concerned documents until a final judgement in the main case is made. These include clinical study reports from both companies. The EMA is considering whether to appeal the interim decisions. Pending the outcome of the final judgement on the main cases, the EMA will continue with its policy to grant access to documents. Requests for access to documents similar to those contested by AbbVie and InterMune will be considered on a case-by-case basis in the light of the court orders. Since November 2010, the EMA has released over 1.9 million pages in response to such requests not 1.6 million as I incorrectly stated at Q60. This is the first time that the policy has been legally challenged. Sir Kent Woods Chief Executive 24 June 2013 Written evidence from the Cochrane Neuraminidase Review Group We are the team responsible for the NIHR-funded Cochrane review on Tamiflu, referred to extensively in the committee hearing. We would like to take the opportunity to set the record straight, with regard to inaccurate statements that arose in the oral evidence taken before the Public Accounts Committee on 17 June 2013. Point 1 Dame Sally Davies reported: “Having read it, I do not believe—Kent and Andrew may have other views— that the Cochrane review on Tamiflu is the last answer, not just because they do not have all the data, but, first of all, because they made up their hypotheses once they had got data, and that is not standard research practice.” Point 1 Response: With regard to the statement, “making up their hypotheses,” our review followed the standard Cochrane methodology of first writing a protocol of how we plan to conduct our analysis. This protocol was then peerreviewed, revised and published in the Cochrane Library in January 2011. However in the process of conducting the review we found multiple discrepancies and other unexpected observations in the clinical study reports and regulatory documents as compared to the published data alone. Based on these unexpected findings we developed “post-protocol hypotheses” which we labelled as such to be transparent about the timing. The results Ev 20 Committee of Public Accounts: Evidence from testing these hypotheses are clearly labelled as post-hoc in the review and conclusions are appropriately tentative. In terms of standard research practice, it is wholly appropriate to perform such analyses, and the methods of these analyses were subjected to peer review before publication. It is important to recognize that such analyses do not affect the primary and secondary outcomes, but they attempt to understand whether the estimates of effect sizes are modified by biases in the trial design. For example, the placebo capsules in the Tamiflu trials contained dehydrocholic acid, a compound which is not present in the active treatment, and is known to cause diarrhea if taken at high doses. We therefore hypothesized that it was important to determine if the presence of dehydrocholic acid increases the incidence of gastrointestinal harms in the placebo arm, thus reducing the apparent differences between the two arms in the trial. We therefore formulated a post-protocol hypothesis concerning this to test out the possible effect. A similar example relates to the content of GSK’s placebo for the trials of their antinfluenza drug zanamivir (Relenza). While reviewing the US FDA critique of zanamivir, we noted their concern over the apparent drop in lung function following zanamivir inhalation, which causes bronchospasm in susceptible individuals and was contained in both the active and the placebo blisters. This principle of using a matching placebo is of course correct, but may have had the effect of increasing the incidence of bronchospasm (or asthma-related episodes) in both arms. This is clearly reported as a warning in the 1999 FDA-approved labelling: “Because the placebo consisted of inhaled lactose powder, which is also the vehicle for the active drug, some adverse events occurring at similar frequencies in different treatment groups could be related to lactose vehicle inhalation”. It was logical therefore to test whether certain harms might be related to placebo content. A further protocol analysis was developed because in the course of reading the Roche Tamiflu clinical study reports, we learned that some of the trial populations deemed influenza infected was determined after randomization into the study, based on the results of laboratory testing by culture and/or antibody rise, rather than at baseline as we had believed. We noted that in all trials the proportion of patients deemed infected with influenza was lower in the active arm compared to the control arm. Our analysis indicates that this is a systematic problem probably due to oseltamivir suppressing the antibody response to influenza. It is vital that placebo and active groups of patients have the same chance of being classified as influenza infected, otherwise any comparison between groups deemed to be infected with influenza will be potentially affected by bias and will essentially be a non-randomised comparison. This is an important issue, particularly if the trial medication affects the production of antibodies; the selection of the influenza infected population is confounded by taking the trial medication. All efficacy analyses in the original trials were conducted in the influenza infected population, and almost all of Roche’s published journal articles of the treatment of influenza report on this subpopulation of individuals deemed influenza infected without any explanation of the differences in numbers between the two arms of the studies. Point 2 Dame Sally Davies: “They [our Cochrane group] extracted data from 25 studies but excluded 42 and took no data from published studies.” Chair: “They took no data from published studies?” Dame Sally Davies: “No, and I could go on.” Point 2 Response: This is an important point, as it could be misunderstood to mean that we either purposefully or inadvertently excluded important trials in the public domain from our review. This is not the case. It is important to understand that not all Roche Tamiflu trials have been published in the scientific literature, and not all trials that have been published have been published accurately. However for all Roche Tamiflu trials, very detailed reports called “Clinical Study Reports” do exist and are typically thousands of pages long each. In the course of updating our review over the past years we identified and reported important discrepancies between the way a particular trial was reported in published literature [reference 1,2,3,4] versus how it was reported in the far more detailed clinical study reports, our Cochrane review therefore decided to use the clinical study reports as the primary document to extract the data from, to ensure the most complete and accurate analysis possible. For example: the two most cited published trials of oseltamivir either did not mention serious adverse events (Nicholson 2000, published in The Lancet), or stated that “... there were no drug related serious adverse events” (Treanor 2000, published in the Journal of the American Medical Association, JAMA). Indeed, these findings were repeated by bodies such as the UK NHS: “No serious adverse events were noted in the major trials and no significant changes were noted in laboratory parameters” (UKMIPG 2001). However, the equivalent clinical study reports for these two trials (known by their study identifiers WV15670 and WV15671, respectively) describe 10 serious adverse events (in nine participants), some of which were classified as “possibly” related to Tamiflu. Committee of Public Accounts: Evidence Ev 21 Enquiries with the first authors were unrewarding as no one had apparently seen the raw data and at least one report had been ghost written (Cohen D. Complications: tracking down the data on oseltamivir. BMJ 2009;339:b5387). Based on these experiences, and Roche’s promise to release full clinical study reports for at least 10 of its trials as well as new freedom of information policies at the European Medicines Agency, we made a decision to base our review on clinical study reports and not journal publications or conference abstracts. For example, we decided not to use the only published material for trial M76001, which is a conference abstract of around 300 words in length, despite this being the largest treatment trial ever undertaken on Tamiflu (with just over 1,400 people of all ages). We note that this is the only document published in the public domain for this trial. In terms of this 300-word abstract, we do not know who actually wrote it. This preclude its use over that of the clinical study report. As part of its investigation into Tamiflu decision making, Channel 4 News found that Professor Treanor— the only author named on the conference abstract—said that he didn’t actually participate in study M76001 and doesn’t remember presenting it at a meeting in 2000. Channel 4 put these facts to Roche and Dr David Reddy, Roche’s Global Pandemic Taskforce leader, responded: “It’s not infrequent that you may have somebody who authors but they don’t actually present it at a conference, it depends upon their availability.” (D Cohen, BMJ 2009) Therefore, given these issues with the published data, and given our finding that 60% (3145/5267) of patient data from randomised, placebo controlled phase III treatment trials of oseltamivir have never been published, it is wholly appropriate for us to refer to and use the clinical study reports as the principle source document. For study M76001 this decision meant that we used the 1,514 page clinical study report that we obtained from the European Medicines Agency via freedom of information requests as opposed to data from a 300 word published abstract. Just this month, we received the remaining portions of the M76001 clinical study report, and altogether it is 9,825 pages long. Therefore, the statement that we included 25 studies but excluded 42 is incorrect. In the published Cochrane review we state: “For 42 studies we were unable to obtain sufficient information to determine their suitability for further assessment and analysis in our review (see Characteristics of studies awaiting classification). Rather than exclude these studies outright we have decided to retain them pending confirmation of data from the additional clinical study report modules. For the oseltamivir trials (WV15799; WV16193; WV15759/WV15871 (WV15799; WV16193; WV15759/WV15871; WV15819/ WV15876/WV15978; MV21737; JV15824; NV16871; MV22841; WV15825; MV21118; JV15823; WV16277; ML20589) we wrote to the manufacturers seeking validation of aspects of methods and results of the trials but received no answer. According to our rules these trials had not been validated and we excluded them from entering Stage 2 of the review.” Of note, 20 oseltamivir trials were not available for data extraction. However in April 2013, Roche informed the Cochrane Group that it would release redacted clinical study reports for 74 trials of Tamiflu that it sponsored. We are now in the process of receiving these reports. We find it perplexing that the regulators continue to state they had all the available evidence. To the best of our knowledge, we had all of the oseltamivir trial data the European Medicines Agency had received, which we obtained via a freedom of information request (22,239 pages of data) (see Table 1). Table 1 THE PROPORTION OF CLINICAL STUDY REPORTS WE OBTAINED FROM THE EUROPEAN MEDICINES AGENCY Trial JV15823 JV15824 M76001 ML16369 ML20542 MV21879 NP15757 NV16871 NV25118 WP16263 1 No. participants 316 308 1,459 478 534 862 59 329 9 400 Parts (“modules”) of oseltamivir clinical study reports obtained (dark circles) and still not obtained (light circles) Synopsis only Synopsis only ????? No data obtained No data obtained No data obtained ????? “Core Report” and “Study Documentation”1 No data obtained ???? Study pages obtained 32 19 1,514 0 0 0 445 614 0 8,545 While the nomenclature is different, “Core Report” and “Study Documentation” are largely of the same form as what is referred to as “Module 1” and “Module 2” for other studies. Ev 22 Committee of Public Accounts: Evidence Trial WV15670 WV15671 WV15673/15697 WV15707 WV15708 WV15730 WV15758 WV15759/15871 WV15799 WV15812/15872 WV15825 WV15876/15819/15978 WV16193 WV16277 Total No. participants 726 629 1,562 27 385 60 698 335 962 404 572 741 808 451 13,114 Parts (“modules”) of oseltamivir clinical study reports obtained (dark circles) and still not obtained (light circles) ????? ????? ????? ???? ???? ???? ????? ????? ????? ????? ????? ????? ???? No data obtained Study pages obtained 1,032 1,018 804 458 661 525 1,126 1,121 900 683 875 973 894 0 22,2392 Note that these 22,239 pages largely comprise incomplete clinical study reports (only one study report was complete), but this is because the European Medicines Agency itself only had incomplete reports so could not provide compete reports. We are therefore unsure who besides Roche had the complete study reports that we are currently receiving thanks to Roche’s new promise, and where these data were held at the time of regulatory approval. Other groups, independent of ours, have noted the weakness in the available evidence, particularly with regard to the claim made by Roche that its clinical trials show that Tamiflu lowers the risk of complications. Burch et al (2009), in their UK funded HTA project “Overall, little information was available on the effects of either drug on the incidence of complications, and there were very few events, in both the healthy adult and at-risk populations. Furthermore, weaknesses in the available evidence limit the reliability and the ability to generalise any results relating to the effect of these drugs on the rates of complications.” (Burch J, Corbett M, Stock C, Nicholson K, Elliot AJ, Duffy S, et al. Prescription of anti-influenza drugs for healthy adults: a systematic review and meta-analysis. Lancet Infect Dis 2009;9:537–45.) We would like to conclude that in our review, even with such a huge amount of data available, we acknowledge that we still do not have and cannot convey the full picture. However, the Cochrane team undertaking this work has completed in excess of 100 reviews across multiple areas of health care, and is involving extensive expertise into this review process. The review, to date, has the most extensive data set ever used in this area and as such provides a transparent assessment and outlines the important threats to validity of the trial results. We therefore stand by our current conclusions: “We found a high risk of publication and reporting biases in the trial programme of oseltamivir. Sub-population analyses of the influenza infected population in the oseltamivir trial programme are not possible because the two arms are non-comparable due to oseltamivir’s apparent interference with antibody production. The evidence supports a direct oseltamivir mechanism of action on symptoms but we are unable to draw conclusions about its effect on complications or transmission.” We are currently receiving full clinical study reports containing study protocol, statistical analysis plan and individual patient data to clarify outstanding issues. We would therefore welcome, if called upon by the Committee, the opportunity to set the record straight. Signed by Cochrane Neuraminidase Review Group Carl Heneghan University of Oxford, Department of Primary Care Health Sciences, Oxford, Oxfordshire, UK Correspondence to carl.heneghan@phc.ox.ac.uk Tom Jefferson The Cochrane Collaboration, Roma, Italy Peter Doshi Johns Hopkins University, Division of General Internal Medicine, Baltimore, Maryland, USA Mark Jones Queensland Health, Centre for Healthcare Related Infection Surveillance and Prevention, Brisbane, QLD, Australia 2 We received 25,453 total pages containing 22,239 unique pages. (For some trials, EMA sent duplicate pages.) Committee of Public Accounts: Evidence Ev 23 Chris Del Mar Bond University, Centre for Research in Evidence-Based Practice (CREBP), Gold Coast, Queensland, Australia Rokuro Hama Japan Institute of Pharmacovigilance, Osaka, Japan Matthew Thompson University of Oxford, Department of Primary Care Health Sciences, Oxford, Oxfordshire, UK References 1. Jefferson T, Jones M, Doshi P, Thompson M, Henegan C. Ensuring safe and effective drugs: who can do what it takes? BMJ 2011;342:c7258 2. Doshi P, Jones M A, Jefferson T. Rethinking credible evidence synthesis. BMJ 2012;344:d7898 doi: 10.1136/bmj.d7898. 3. Jefferson T, Jones M A, Doshi P, Del Mar C B, Heneghan C J, Hama R, Thompson M J. Neuraminidase inhibitors for preventing and treating influenza in healthy adults and children. Cochrane Database of Systematic Reviews 2012, Issue 1. Art. No.: CD008965. DOI: 10.1002/14651858.CD008965.pub3. 4. Doshi P, Jefferson T, Del Mar C (2012). The Imperative to Share Clinical Study Reports: Recommendations from the Tamiflu Experience. PLoS Med 9(4): e1001201. doi:10.1371/journal.pmed.1001201 20 June 2013 Written evidence from National Institute for Health and Care Excellence At the end of the meeting of the Committee for Public Accounts on Monday 17 June, during which the Committee considered access, by NICE, to data from pharmaceutical companies, you asked me to write to you with the revised wording of the declaration we ask company medical directors to sign. The current wording is: “I confirm that all relevant data pertinent to the [STA] [MTA] have been disclosed to the Institute. (Signed) Medical Director (or equivalent)”. Note: STA and MTA refer to the two different forms of appraisal we use. From August, we intend to use the following wording: “I confirm that all the data necessary to address the remit and scope of the technology appraisal as issued by the Department of Health and NICE, within the Company’s or any of its associated companies’ possession, custody, or control in the UK or elsewhere in the world, have been disclosed to NICE or its authorised agents. (Signed) Medical Director (or equivalent senior registered medical practitioner”. Sir Andrew Dillon Chief Executive 26 June 2013 Written evidence from Department of Health RE: ACCESS TO CLINICAL TRIAL INFORMATION AND THE STOCKPILING OF TAMIFLU (HC 125) I offered to write with comments about the latest Cochrane Review on Tamiflu. As I explained to your Committee, we spend approximately £5 million a year supporting the Cochrane Collaboration and I myself was an “Editor” for many years until I became Chief Medical Officer. So below is a critical commentary of the 2012 Cochrane review of neuraminidase inhibitors for the prevention and treatment of influenza,1 which discusses its findings in relation to previous systematic reviews on this topic. Background The original Cochrane review of neuraminidase inhibitors for influenza in healthy adults was published in 1999. Standard systematic review methods (updated literature searches and pharmacovigilance data) were subsequently used to update the review in 2006, 2008, and 2009. These pre-2012 Cochrane reviews and other systematic reviews were based on similar sets of data from published trials and, where available, manufacturer evidence submissions to regulatory and other authorities.2,3,4 Ev 24 Committee of Public Accounts: Evidence The National Institute of Health Research (NIHR) contributed towards the costs of the 2009 review, as part of our pandemic “flu call”. The relevant Collaborative Review Group (CRG) is based in Australia and the main authors in Italy and Australia. Up to 2009, these reviews were not supported by NIHR. NIHR also funded a technology assessment review for NICE3 in 2008–09, well before the pandemic flu call, and based on available published literature at the time. This concluded that both zanamivir and oseltamivir were effective in reducing duration of symptoms (between 0.5 days and 1.5 days for oseltamivir), though questioned the clinical significance of this effect. There was some evidence of efficacy in terms of seasonal prophylaxis (particularly in at-risk elderly people) but little information was available on the incidence of complications. Interventions appeared to be well tolerated, with a relatively low occurrence of subjects experiencing drug-related adverse events and drug-related withdrawals. The evidence largely related to the treatment and prevention of seasonal flu. The 2009 update of the Cochrane review reported similar findings to this and other systematic reviews, but after noting inconsistencies between unpublished clinical study reports (CSRs) and published reports of clinical trial results, the Cochrane authors withdrew their review and entered into a dispute with Roche over gaining access to the manufacturer’s unpublished data. At various times, it seemed likely that they would receive data from companies and therefore were given funding by the NIHR HTA programme to finish this work: however, the data received have never been complete enough to satisfy the reviewers. Due to concerns about publication and reporting biases, and discrepancies between published reports and the (usually confidential) clinical study reports of the same trials, the Cochrane authors decided to disregard journal publications in the 2012 update entirely, instead focusing solely on data identified presented in CSRs and information submitted to regulatory authorities. This is not standard practice and concerns me as to study completeness. This new review consists of two interwoven components: an update of the original systematic review, with restrictions on the data used as outlined below and an unusual and evolving methodological/investigative exploration of new research methods, not typically found in a Cochrane review. Reporting of both components in the same report resulted in a lack of clarity about some aspects of the systematic review. The review was refereed before inclusion in the Cochrane library: the reviewers were a “consumer” referee, a methodologist/GP, an influenza specialist, a statistician, and an editor/methodologist. It was signed off jointly by the Editor-in-Chief (Dr D Tovey) and by the CRG lead (Chris del Mar). It has resulted in heated debate with many experts in the field as to its relevance. The 2012 Cochrane Review Objectives, identification and selection of data The original objective was to assess the effectiveness and harms of neuraminidase inhibitors for influenza in all age groups, by using data from all relevant RCTs, whether published or unpublished. In addition to searches of electronic databases, trial registries, regulator websites and NICE submissions, the authors entered into correspondence with manufacturers (Roche, GSK) and regulators (FDA„EMA; JSBA). Data were rigorously cross-referenced to identify a list of ail potentially relevant trials. These methods are more intensive than those typically undertaken in systematic reviews, and are likely to have identified the most comprehensive set of studies to date, which is good. Similar to previous reviews, the 2012 Cochrane review included RCTs that evaluated neuraminidase inhibitors against placebo or standard care for prophylaxis, post-exposure prophylaxis or treatment of influenza in previously healthy people with seasonal flu, ie not pandemic. Since the authors have been promised individual participant data by GSK, they decided to assess zanamivir in a separate review. Therefore much of the 2012 review focuses on oseltamivir, rather than neuraminidase inhibitors in general as stated in the title. The key difference from previous reviews was that a two-stage process for selecting and analysing trials was used: in stage 1, data from relevant trials were extracted from the original CSRs where available, or alternatively “CSRs” were reconstructed from trial data spread across several regulatory documents. In stage 2, only trials reporting methods and results to CONSORT standards and for which all data were entirely consistent both within and across different sources, would be analysed. These minimum standards for inclusion of data are much more stringent than typically mandated by systematic reviews. The plan was to apply standard Cochrane methods to trials included in stage 2. Results Sixty-seven studies met the criteria for inclusion in stage 1 (30 oseltamivir, 31 zanamivir, 6 peramivir), of which 42 currently remain listed as “awaiting classification” because the authors “require” further information from CSRs before being able to quality assess and extract the required data. Thus the review only has data for 37% of potentially eligible trials. Of the 25 trials included in stage 1 (15 oseltamivir, 10 zanamivir), none were considered adequate to be included in stage 2 for full analysis. The authors stated that trials were excluded from stage 2 if the available data was either incomplete or contained inconsistencies. Committee of Public Accounts: Evidence Ev 25 The authors did not state whether minor inconsistencies were tolerated; excluding whole trials on the basis of very minor data discrepancies would be an extreme position that strives for complete data accuracy at the cost of rejecting other potentially valid information. Efficacy outcomes Despite their statement that no studies met the stage 2 inclusion criteria, the authors nevertheless proceeded to report a meta-analysis of time to first symptom alleviation for a subset of eight oseltamivir studies, for which they had obtained at least partial unabridged CSRs. This suggested a statistically significant reduction in symptom duration for oseltamivir of around 21 hours (95% CI -29.6 to -13.0; five RCTs) in patients with influenza-like illness. Based on an analysis of all eight trials, they found no benefit in terms of hospitalization for oseltamivir (odds ratio 0.95, 95% CI 0.57 to 1.61; risk difference 0.00, 95% CI -0.01 to 0.01). These meta-analyses only include a small proportion of the potentially relevant trials identified by the review (8/30 oseltamivir RCTs; 27%), so the validity of the estimates is questionable. Complications and harms The authors stated standard definitions of complications were not prepared or incorporated into the trials; the authors refer to reporting of cases “otitis media” “pneumonia” or “bronchitis” that was based on local definitions, making it impossible to attribute a cause and draw conclusions. The analysis of serious harms and dropouts was delayed until the review authors have access to CSR modules containing detailed case reports. Post-protocol analyses The review also presented a series of post hoc analyses, the most prominent of which investigated the relationship between oseltamivir and antibody production. Post hoc analysis is not standard Cochrane analysis. The authors concluded that a possible interaction between oseltamivir use and antibody production resulted in laboratory-confirmed influenza-infected sub-populations that were not comparable across treatment arms. The hypotheses for these post-hoc analyses arose out of the analysis of the data and were not pre-specified: and must therefore be considered exploratory and hypothesis generation, rather than true hypothesis testing. Commentary Whereas previous reviews mainly analysed published trials and data sources accessible through standard research methods, the 2012 Cochrane review sought to avoid publication and selective reporting biases by recreating the entire trial programme for each drug, and then obtaining comprehensive validated data summaries for every trial. This highly meticulous and restrictive approach is radically different from standard Cochrane systematic review methods. The Cochrane authors’ assertion that clinical study reports and regulatory comments are likely to give the less biased, more complete, and more insightful set of data than publications is probably justified in this case. Their approach identified several potentially relevant trials that have never been published (eg eight RCTs of oseltamivir), and their detailed scrutiny of the included data permitted exploratory analyses that would not have otherwise been possible. However, the review was unable to access all the data they had hoped, either because CSRs for all potentially relevant trials were not available within the timeframe of the review, or because their rigorous approach and extremely strict inclusion criteria meant that trials with any incomplete or inconsistent data were excluded from further analysis. This review is therefore largely unable to draw firm conclusions about the efficacy and safety of oseltamivir, as the authors acknowledge. The authors have made concerted and ongoing efforts to obtain all the relevant evidence, and the nonavailability of CSRs for some trials is beyond their control. The authors stated that all data (eg denominators) should be both “broadly consistent” within documents and have “consistency” across different documents. This is a sensible part of their validation strategy. However, it is unclear precisely how restrictive they have been in terms of completeness and consistency of data. If entire trial documents were excluded on the basis of inconsistencies for a single, less crucial, outcome, then valuable data on other outcomes from the same trial may have been rejected. The 2012 report is based on only a small proportion of the potentially eligible trials; 42 of the 67 potentially relevant trials either have yet to provide CSRs or are awaiting clarification. It is not clear how many of these will ultimately provide sufficient data to be included in stages 1 or 2 of the review, In addition, the review of zanamivir data has been postponed entirely while the authors await individual participant data from the manufacturer. This raises the question of whether the review was at a stage suitable for publication given it is only partially completed. In this respect, the 2012 review is more of a progress report than a complete update of the systematic review evidence. Ev 26 Committee of Public Accounts: Evidence This was behind my statement that “Having read it, I do not believe...that the Cochrane review on Tamiflu is the last answer, not just because they do not have all the data, but, first of ail, because they made up their hypotheses once they had got data, and that is not standard research practice. They extracted data from 25 studies but excluded 42 and took no data from published studies … and then they left out what was published...” Like previous reviews, the Cochrane 2012 efficacy meta-analyses were based on an incomplete subset of known trials, except that trial inclusion was determined by the availability of CSR data rather than by publication status. It is not dear why these meta-analyses were conducted at all, given the authors’ assertion that no trials were eligible for inclusion in stage 2 of their review. It may be that they made a pragmatic decision to analyse, the data available at the time, but this is not explicitly stated. However, much like previous reviews, the value of this meta-analysis is questionable since it is based on incomplete evidence for seasonal flu. Once the access to data is complete, it is not clear whether the authors will adhere to their current highly restrictive Stage 2 selection criteria (which might result in few or no included trials) or make a pragmatic decision to analyse the best available evidence, with caveats around the subsequent findings. The review provides no information on complications or harms because it had zero tolerance for omissions or discrepancies among data sources, and subsequently included no studies in the full analysis. Conclusions This review is complex and difficult to follow because of the combination of methodological development; more rigorous methods than usual in selecting inclusion criteria than a traditional review using data which the reviewers seem to reject at first; and the inclusion of much hypothesis generation, not always well separated from hypothesis testing. Nevertheless, its key findings are: — that until all relevant data are exposed to scrutiny, there is uncertainty about both the critical benefits (mortality, preventing complications) in seasonal flu and serious harms of both neuraminidase inhibitors; — that previous systematic reviews of oseltamivir for influenza only had access to trials likely to be biased in favour of treatment; and — that the review team are breaking new methodological ground, particularly within Cochrane but more broadly within systematic reviewing. It may be that their approach could be improved on, and indeed, by putting the next protocol out to consultation to elicit feedback, this is something the team acknowledges. This review is not the last word but an interim report and with our backing via NIHR, moreover, if promises to provide data are met by industry, we should be nearing the definitive review. As I said: “I want to finish by saying that we continue to fund this and we believe in an open debate. We were going to dose the funding down because Roche had not agreed to supply the data. They have now agreed, so we are continuing to fund them in order to allow the open debate”. As far as pandemic use goes, results not just from either Cochrane report but wide scientific consultation, for example with the Scientific Advisory Group and wider academic colloquium, were taken into account5 The findings from these groups were consistent with the WHO view on stockpiling antivirals. There was published and unpublished evidence of around 50% reduction in complications (and thus reasonably deaths) though with large uncertainties accepted by the consensus of those consulted. As I said at the hearing, this was shown to be correct in 2009. “In Emergency planning we must often take best estimates from the available evidence rather than wait until we have the absolutely the ‘best’ possible evidence as the latest Cochrane report required. We based our decision to stockpile Tamiflu on the scientific consensus given the available evidence and this consensus has been shown to be consistent with what was observed in 2009”. Finally, I was also asked which other EU Member States had a similar antiviral policy. Individual Member States have not put this information in the public domain. I am able to say that figures that have been reported by EU member states to the Commission relating to antivirals stockpiled show that around 25% of EU Member States consider that they have sufficient antivirals to cover their needs. Around 40% of Member States indicate an antiviral stock ranging from “limited” to 30% population coverage. References 1 Jefferson T, Jones M A, Doshi P, Del Mar C B, Heneghan C J, Hama R, Thompson M J. Neuraminidase Inhibitors for preventing and treating influenza in healthy adults and children. Cochrane Database of Systematic Reviews 2012, Issue 1. Art. No.: CD008965. DOI: 10.1002/14651858.CD008965.pub3. 2 Burch J, Corbett M, Stock C, Nicholson K, Elliot A J, Duffy S, et al. Prescription of anti-influenza drugs for healthy adults: a systematic review and meta-analysis. Lancet 2009; 9(9): 537–45. Committee of Public Accounts: Evidence Ev 27 3 Tappenden P, Jackson R, Cooper K, Rees A, Simpson E, Read R, et al. Amantadine, oseltamivir and zanamivir for the prophylaxis of influenza (including a review of existing guidance no. 67): a systematic review and economic evaluation. Health Technology Assessment 2009; 1 l(iii, ix–xii): 1–246. 4 Turner D, Wailoo A, Nicholson K, Cooper N, Sutton A, Abrams K. Systematic review and economic decision modelling for the prevention and treatment of influenza A and B. Health Technology Assessment 2003;7(35):iiiiv, xi-xiii, 1–170. 5 Muthuri S, Venkatesan S, Myles P, Van-Tarn N. Updated evidence base of neuraminidase inhibitors in the mitigation of pandemic influenza, March 2013. 26 July 2013 Printed in the United Kingdom by The Stationery Office Limited 1/2014 32605 19585