Bayesian Methodsin Regulatory Science

Gary L. Rosner, Sc.D.

Regulatory-Industry Statistics Workshop Washington, D.C. 13 September 2018

What is Regulatory Science?• US FDA

‣ Regulatory Science is the science of developing new tools, standards, and approaches to assess the safety, efficacy, quality, and performance of all FDA-regulated products.

�2

www.fda.gov/ScienceResearch/SpecialTopics/RegulatoryScience/default.htm Accessed 5 June 2018

What Do I Mean by “Bayesian Approaches”?• Truly Bayes:

‣ Choose design points to maximize expected utility

- E.g., sample size for certain precision at fixed cost

• “Stylized” Bayes (or “calibrated” Bayes, etc.)

‣ Bayesian formulation of study data

‣ Choose design points to achieve “desirable” frequentist operating characteristics

- E.g., choose prior parameters to attain monitoring rules good sensitivity, specificity, & expected N

�3

Acceptance When Want to Borrow Information• Change in device from previous version

‣ Historical information reasonable if minor change

• Population pharmacokinetics & pharmacodynamics

‣ Population modeling & hierarchical models

• Pediatric drug development

‣ Borrow information

• Rare diseases or adverse events

• Early phase studies (e.g., phases 1 & 2)

�4

When Are Bayesian Statistical Methods Not Accepted• When want to be “objective”

‣ Concern about influence of prior distribution

- But frequentist methods are subjective, too!

‣ Model,

‣ Null & alternative hypotheses,

‣ Incorporating current trial in context of others

• Confirmatory drug studies

�5

Sheiner: Two Major Learn-Confirm Cycles in Clinical Drug Development• 1st cycle:

‣ Phase 1: Learn what dose is tolerated

‣ Phase 2: Confirm dose has promise of efficacy

- Make decision based on this learn-confirm cycle

• 2nd cycle:

‣ Phase 2B: Learn how to use the drug in patients

‣ Phase 3: Confirm in large representative pt pop’n that therapy achieves acceptable benefit:risk ratio

- If acceptable, approval is granted�6

Clin Pharmacol Ther 61:275-91, 1997

Sheiner (cont’d)• Learning & confirming are distinct

‣ Different goals, designs, methods of analysis

- Analysis choice: Hypothesis testing or estimation?

- Learning involves estimation

‣ “The [B]ayesian view is well suited to this task because it provides a theoretical basis for learning from experience; that is, for updating prior beliefs in the light of new evidence.”

- Confirming involves hypothesis testing

�7

Clin Pharmacol Ther 61:275-91, 1997

Neyman & Pearson• Need more than a test based on probability to establish

truth of a particular hypothesis

�8

Neyman, J., and Pearson, E. S. (1933). On the Problem of the Most Efficient Tests of Statistical Hypotheses. Philosophical Transactions of the Royal Society of London. Series A, 231, 289-337.

R. A. Fisher• Discusses forming summary statistics & evaluating

deviation relative to a dist’n (for “tests of significance”).

�9

Fisher, R. A. (1934). Statistical Methods for Research Workers, 5th ed. Edinburgh: Oliver and Boyd.

SN Goodman: Toward Evidence-Based Medical Statistics 1 & 2• A p value seems to provide

‣ a measure of evidence against H0 from this study

‣ a means to control Type 1 error regarding rejecting H0 in the long run.

• But, no single number can do both jobs!

‣ Describe long-run behavior & meaning in this study.

‣ Treat this study as one of infinitely many & provide a measure of evidence for results in this study.

�10

and

Goodman, S. N. (1999). Toward Evidence-Based Medical Statistics 1 & 2 Ann Intern Med 130, 995–1004 & 1005-1013.

Hypothesis Testing• Answer a question

‣ Do observations agree with predictions based on one hypothesis more than the other?

• N-P testing

‣ Decision rule to limit long-run risks of errors

• Bayesian testing

‣ Compare to

- Could be with Bayes factor and Jeffreys’s criteria

�11

Pr(HA | y)Pr(H0 | y)

,Pr(HA)

Pr(H0)AAAEyXichVNdb9MwFPW2AKN8rINHJGTRIW0vI+EFxNMoEprESxF0m9R0lePctNbsONgOaxvliR/Bb+Gn8ASP8C9wPjrWtAhLsY7vOSf2vfYNEs60cd3vG5tbzo2bt7Zvt+7cvXd/p7374ETLVFHoU8mlOguIBs5i6BtmOJwlCogIOJwGF28K/vQzKM1k/NHMEhgKMo5ZxCgxNjRqf/AjRWjm99T+8eg19gULsR9E2Sw/yOuouxzFfpeNx+IZXnL+VR/ko3bHPXTLgVeBV4PO0duvj38ihHqj3a1zP5Q0FRAbyonWA89NzDAjyjDKIW/5qYaE0AsyhoGFMRGgh1mZfY6f2kiII6nsFxtcRq87MiK0nonAKgUxE93kiuA6bpCa6OUwY3GSGohptVGUcmwkLkqJQ6aAGj6zgFDF7FkxnRBbFGML3vJjuKRSCBKHmQ+JzsuZcRnbfBRcZ+3+xQSG5Ms2zcZ5OYsmE1jxhBjLXrIQLPJF2pCI9D+C4h/aEJVX8NwvF8uaEPiVqMRrVdNKMf03GRSbiCtFQzKv/PO1frFIwjAeQiaa5u4y323y9naKKopATrMSN8sg8oE3rAV+IHlYPAW8hzve3oq2ziNoEtNFgk1iVhOzJiHSmikuZl3VF3S5aCp4wKtTl889UxDaxqwyiKQ0sTSg2RxsDq+s1Xak1+y/VXDy/NBzD733tjW7qBrb6BF6gvaRh16gI3SMeqiPKPqGfqBf6LfzzvnkTJ15Jd3cqD0P0dJwvvwBLWGzpA==AAAEyXichVNdb9MwFPW2AqN8rINHJGSxIW0vJeEFxNPoJDSJlyJoN6npKse5aa3ZcbAd1jbKEz9iv4WfwhMST/AvcD461rQIS7GO7zkn9r329WPOtHGc7xubW41bt+9s323eu//g4U5r91Ffy0RR6FHJpTrziQbOIugZZjicxQqI8Dmc+hfHOX/6BZRmMvpkZjEMBRlHLGSUGBsatT56oSI09brq4GT0FnuCBdjzw3SWHWZV1FmOYq/DxmPxAi85/6oPs1Frz2k7xcCrwK3A3tG7q6c/j/tX3dHu1rkXSJoIiAzlROuB68RmmBJlGOWQNb1EQ0zoBRnDwMKICNDDtMg+w89tJMChVPaLDC6iNx0pEVrPhG+VgpiJrnN5cB03SEz4epiyKE4MRLTcKEw4NhLnpcQBU0ANn1lAqGL2rJhOiC2KsQVvehFcUikEiYLUg1hnxcy4jGw+Cm6ydv98AkOyZZtm46yYRZ3xrXhCjGUvWQAWeSKpSUTyH0H+D22Iykp47hWLZU0A/FpU4LWqaamY/pv0803EtaImmZf++Vq/WCRhGA8gFXVzZ5nv1Hl7O3kVhS+naYHrZRDZwB1WAs+XPMifAt7He+7+irbKw68T00WCdWJWEbM6IZKKyS9mXdUXdLGoK7jPy1MXzz1VENjGLDMIpTSRNKDZHGwOb6zVdqRb779V0H/Zdp22+8G2ZgeVYxs9Qc/QAXLRK3SETlAX9RBF39AP9Av9brxvfG5MG/NSurlReR6jpdH4+gfSo7TWAAAEyXichVNdb9MwFPW2AqN8rINHJGSxIW0vJeEFxNPoJDSJlyJoN6npKse5aa3ZcbAd1jbKEz9iv4WfwhMST/AvcD461rQIS7GO7zkn9r329WPOtHGc7xubW41bt+9s323eu//g4U5r91Ffy0RR6FHJpTrziQbOIugZZjicxQqI8Dmc+hfHOX/6BZRmMvpkZjEMBRlHLGSUGBsatT56oSI09brq4GT0FnuCBdjzw3SWHWZV1FmOYq/DxmPxAi85/6oPs1Frz2k7xcCrwK3A3tG7q6c/j/tX3dHu1rkXSJoIiAzlROuB68RmmBJlGOWQNb1EQ0zoBRnDwMKICNDDtMg+w89tJMChVPaLDC6iNx0pEVrPhG+VgpiJrnN5cB03SEz4epiyKE4MRLTcKEw4NhLnpcQBU0ANn1lAqGL2rJhOiC2KsQVvehFcUikEiYLUg1hnxcy4jGw+Cm6ydv98AkOyZZtm46yYRZ3xrXhCjGUvWQAWeSKpSUTyH0H+D22Iykp47hWLZU0A/FpU4LWqaamY/pv0803EtaImmZf++Vq/WCRhGA8gFXVzZ5nv1Hl7O3kVhS+naYHrZRDZwB1WAs+XPMifAt7He+7+irbKw68T00WCdWJWEbM6IZKKyS9mXdUXdLGoK7jPy1MXzz1VENjGLDMIpTSRNKDZHGwOb6zVdqRb779V0H/Zdp22+8G2ZgeVYxs9Qc/QAXLRK3SETlAX9RBF39AP9Av9brxvfG5MG/NSurlReR6jpdH4+gfSo7TWAAAEyXichVNNb9MwGPa2AKN8dXDkYtEhbZeS7ALiNMplEpci6Dap6SrHedNZs+NgO6xtlBO/ip/CiSv8C5yPjiUtwlKsx+/zPLHf136DhDNtXPfH1vaOc+fuvd37nQcPHz1+0t17eqplqiiMqORSnQdEA2cxjAwzHM4TBUQEHM6Cq/cFf/YVlGYy/mwWCUwEmcUsYpQYG5p2P/mRIjTzh+rgZPoO+4KF2A+ibJEf5nXUbUaxP2CzmXiFG86/6sN82u25fbcceB14NeihegynezsXfihpKiA2lBOtx56bmElGlGGUQ97xUw0JoVdkBmMLYyJAT7Iy+xy/tJEQR1LZLza4jN52ZERovRCBVQpiLnWbK4KbuHFqojeTjMVJaiCm1UZRyrGRuCglDpkCavjCAkIVs2fF9JLYohhb8I4fwzWVQpA4zHxIdF7OjMvY5qPgNmv3LyYwJG/aNJvl5SzaTGDFl8RY9pqFYJEv0pZEpP8RFP/Qhqi8ghd+uWhqQuA3ohJvVM0rxfzfZFBsIm4ULcmy8i83+sUqCcN4CJlomwdNftDm7e0UVRSBnGclbpdB5GNvUgv8QPKweAp4H/e8/TVtnUfQJuarBNvEoiYWbUKkNVNczKaqr+hy0VbwgFenLp97piC0jVllEElpYmlAsyXYHN5aq+1Ir91/6+D0qO+5fe+j2zse1L25i56jF+gAeeg1OkYnaIhGiKLv6Cf6hX47H5wvztxZVtLtrdrzDDWG8+0P9pWxiQ==

Pr(HA | y)AAAEinichZNdb9MwFIa9LcAoAzrgjhtrHdIQUpXshq+bUrjYZZHoNqnpKsc5ba3ZcWQ7rGmU/7Jb+Ef8G5yPTmtahKVYr/28J/Y5toOYM21c98/O7p7z4OGj/cetJwdPnz1vH7441zJRFIZUcqkuA6KBswiGhhkOl7ECIgIOF8H114Jf/ASlmYx+mDSGsSCziE0ZJcZOTdqv/IE6OZt8wb5gIfaDaZbmbyftjtt1y4Y3hVeLTu/If3eLEBpMDveu/FDSREBkKCdajzw3NuOMKMMoh7zlJxpiQq/JDEZWRkSAHmfl9nP8xs6EeCqV/SKDy9n7ERkRWqcisE5BzFw3WTG5jY0SM/0wzlgUJwYiWi00TTg2Ehe1wCFTQA1PrSBUMbtXTOdEEWpsxVp+BDdUCkGiMPMh1nnZMy4jm4+C+9SuX3RgSL4eptksL3vRJIE1z4mx9IaFYJUvkoZFJP8xFP/Qhqi8kld+OVj3hMDvTKXe6lpUjsW/YVAsIu4cDcuyil9ujRerJAzjIWSiGdxf5/0mt6dTVFEEcpGVulkGkY+8cW3wA8nD4irgY9zxjje8dR5BEyxWCTZBWoO0CURSk+JgtlV9hctB08EDXu26vO6ZgjDHdQZTKU0kDWi2BJvDJxtqX6TXfH+b4vy067ld77vX6fVR1fbRa3SETpCH3qMeOkMDNEQULdEt+oV+OwfOqfPR+VxZd3fqmJdorTnf/gL/vZrOAAAEinichZPdbtMwFMe9LcAog3XAHTfWOqQhpCnZDV83pXCxyyLRbVLTVY5z2lqz48h2WNMob8EDcAtvtLfB+ei0pkVYivW3f/8T+xzbQcyZNq57u7W94zx4+Gj3cevJ3tNn++2D5+daJorCgEou1WVANHAWwcAww+EyVkBEwOEiuP5S8IsfoDST0XeTxjASZBqxCaPE2Klx+6XfV8dn48/YFyzEfjDJ0vzNuN1xT9yy4XXh1aLTPfTf/rztpv3xwc6VH0qaCIgM5UTroefGZpQRZRjlkLf8RENM6DWZwtDKiAjQo6zcfo5f25kQT6SyX2RwOXs/IiNC61QE1imImekmKyY3sWFiJu9HGYvixEBEq4UmCcdG4qIWOGQKqOGpFYQqZveK6YwoQo2tWMuP4IZKIUgUZj7EOi97xmVk81Fwn9r1iw4MyVfDNJvmZS+aJLDmGTGW3rAQrPJF0rCI5D+G4h/aEJVX8sovB6ueEPidqdQbXfPKMf83DIpFxJ2jYVlU8YuN8WKZhGE8hEw0g3urvNfk9nSKKopAzrNSN8sg8qE3qg1+IHlYXAV8hDve0Zq3ziNogvkywSZIa5A2gUhqUhzMpqovcTloOnjAq12X1z1TEOa4zmAipYmkAc0WYHP4aEPti/Sa729dnJ+eeO6J983rdHuoarvoFTpEx8hD71AXnaE+GiCKFugX+o3+OHvOqfPB+VRZt7fqmBdopTlf/wLsMZxUAAAEinichZPdbtMwFMe9LcAog3XAHTfWOqQhpCnZDV83pXCxyyLRbVLTVY5z2lqz48h2WNMob8EDcAtvtLfB+ei0pkVYivW3f/8T+xzbQcyZNq57u7W94zx4+Gj3cevJ3tNn++2D5+daJorCgEou1WVANHAWwcAww+EyVkBEwOEiuP5S8IsfoDST0XeTxjASZBqxCaPE2Klx+6XfV8dn48/YFyzEfjDJ0vzNuN1xT9yy4XXh1aLTPfTf/rztpv3xwc6VH0qaCIgM5UTroefGZpQRZRjlkLf8RENM6DWZwtDKiAjQo6zcfo5f25kQT6SyX2RwOXs/IiNC61QE1imImekmKyY3sWFiJu9HGYvixEBEq4UmCcdG4qIWOGQKqOGpFYQqZveK6YwoQo2tWMuP4IZKIUgUZj7EOi97xmVk81Fwn9r1iw4MyVfDNJvmZS+aJLDmGTGW3rAQrPJF0rCI5D+G4h/aEJVX8sovB6ueEPidqdQbXfPKMf83DIpFxJ2jYVlU8YuN8WKZhGE8hEw0g3urvNfk9nSKKopAzrNSN8sg8qE3qg1+IHlYXAV8hDve0Zq3ziNogvkywSZIa5A2gUhqUhzMpqovcTloOnjAq12X1z1TEOa4zmAipYmkAc0WYHP4aEPti/Sa729dnJ+eeO6J983rdHuoarvoFTpEx8hD71AXnaE+GiCKFugX+o3+OHvOqfPB+VRZt7fqmBdopTlf/wLsMZxUAAAEinichZNNb9MwGMe9LcAog3XAjYtFhzQuVbILb5dROOxYJLpNarrKcZ601uw4sh3WNMp34QrfiG+D89JpTYuwFOtv//5P7OexHSScaeO6f3Z295wHDx/tP+48OXj67LB79PxCy1RRGFHJpboKiAbOYhgZZjhcJQqICDhcBjdfSn75A5RmMv5usgQmgsxiFjFKjJ2adl/6Q3VyPv2MfcFC7AdRnhVvp92e23erhjeF14geatpwerR37YeSpgJiQznReuy5iZnkRBlGORQdP9WQEHpDZjC2MiYC9CSvtl/gN3YmxJFU9osNrmbvR+REaJ2JwDoFMXPdZuXkNjZOTfR+krM4SQ3EtF4oSjk2Epe1wCFTQA3PrCBUMbtXTOdEEWpsxTp+DLdUCkHiMPch0UXVMy5jm4+C+9SuX3ZgSLEeptmsqHrRJoE1z4mx9JaFYJUv0pZFpP8xlP/Qhqiiltd+NVj3hMDvTJXe6lrUjsW/YVAuIu4cLcuyjl9ujRerJAzjIeSiHTxY54M2t6dTVlEEcpFXul0GUYy9SWPwA8nD8irgY9zzjje8TR5BGyxWCbZB1oCsDUTakPJgtlV9hatB28EDXu+6uu65grDATQaRlCaWBjRbgs3how21L9Jrv79NcXHa99y+983rnQ2at7mPXqHX6AR56B06Q+doiEaIoiX6iX6h386Bc+p8cD7V1t2dJuYFWmvO178CnJlF

Pr(H0 | y)AAAEinichZNdb9MwFIa9LcAog3XAHTfWOqQhpCrZDV83VeFil0Wi26SmqxzntLVmx5HtsKZR/stu4R/xb3A+Oq1pEZZivfbzntjn2A5izrRx3T87u3vOo8dP9p+2nh08f3HYPnp5oWWiKAyp5FJdBUQDZxEMDTMcrmIFRAQcLoObrwW//AlKMxn9MGkMY0FmEZsySoydmrRf+wN1ej5xsS9YiP1gmqX5u0m743bdsuFN4dWi0zv2398hhAaTo71rP5Q0ERAZyonWI8+NzTgjyjDKIW/5iYaY0Bsyg5GVERGgx1m5/Ry/tTMhnkplv8jgcvZhREaE1qkIrFMQM9dNVkxuY6PETD+OMxbFiYGIVgtNE46NxEUtcMgUUMNTKwhVzO4V0zlRhBpbsZYfwS2VQpAozHyIdV72jMvI5qPgIbXrFx0Ykq+HaTbLy140SWDNc2IsvWUhWOWLpGERyX8MxT+0ISqv5LVfDtY9IfB7U6m3uhaVY/FvGBSLiHtHw7Ks4pdb48UqCcN4CJloBvfXeb/J7ekUVRSBXGSlbpZB5CNvXBv8QPKwuAr4BHe8kw1vnUfQBItVgk2Q1iBtApHUpDiYbVVf4XLQdPCAV7sur3umIMxxncFUShNJA5otwebw2YbaF+k139+muDjrem7X++51en1UtX30Bh2jU+ShD6iHztEADRFFS3SHfqHfzoFz5nxyvlTW3Z065hVaa863v7m/mr0=AAAEinichZPdbtMwFMe9LcAoAzrgjhtrHdIQUpXshq+bqnCxyyLRbVLTVY5z2lqz48h2WNMob8EDcAtvtLfB+ei0pkVYivW3f/8T+xzbQcyZNq57u7O75zx4+Gj/cevJwdNnz9uHL861TBSFIZVcqsuAaOAsgqFhhsNlrICIgMNFcP2l4Bc/QGkmo+8mjWEsyCxiU0aJsVOT9it/oE7OJi72BQuxH0yzNH87aXfcrls2vCm8WnR6R/67n7e9dDA53LvyQ0kTAZGhnGg98tzYjDOiDKMc8pafaIgJvSYzGFkZEQF6nJXbz/EbOxPiqVT2iwwuZ+9HZERonYrAOgUxc91kxeQ2NkrM9MM4Y1GcGIhotdA04dhIXNQCh0wBNTy1glDF7F4xnRNFqLEVa/kR3FApBInCzIdY52XPuIxsPgruU7t+0YEh+XqYZrO87EWTBNY8J8bSGxaCVb5IGhaR/MdQ/EMbovJKXvnlYN0TAr8zlXqra1E5Fv+GQbGIuHM0LMsqfrk1XqySMIyHkIlmcH+d95vcnk5RRRHIRVbqZhlEPvLGtcEPJA+Lq4CPccc73vDWeQRNsFgl2ARpDdImEElNioPZVvUVLgdNBw94tevyumcKwhzXGUylNJE0oNkSbA6fbKh9kV7z/W2K89Ou53a9b16n10dV20ev0RE6QR56j3roDA3QEFG0RL/Qb/THOXBOnY/O58q6u1PHvERrzfn6F6YznEM=AAAEinichZPdbtMwFMe9LcAoAzrgjhtrHdIQUpXshq+bqnCxyyLRbVLTVY5z2lqz48h2WNMob8EDcAtvtLfB+ei0pkVYivW3f/8T+xzbQcyZNq57u7O75zx4+Gj/cevJwdNnz9uHL861TBSFIZVcqsuAaOAsgqFhhsNlrICIgMNFcP2l4Bc/QGkmo+8mjWEsyCxiU0aJsVOT9it/oE7OJi72BQuxH0yzNH87aXfcrls2vCm8WnR6R/67n7e9dDA53LvyQ0kTAZGhnGg98tzYjDOiDKMc8pafaIgJvSYzGFkZEQF6nJXbz/EbOxPiqVT2iwwuZ+9HZERonYrAOgUxc91kxeQ2NkrM9MM4Y1GcGIhotdA04dhIXNQCh0wBNTy1glDF7F4xnRNFqLEVa/kR3FApBInCzIdY52XPuIxsPgruU7t+0YEh+XqYZrO87EWTBNY8J8bSGxaCVb5IGhaR/MdQ/EMbovJKXvnlYN0TAr8zlXqra1E5Fv+GQbGIuHM0LMsqfrk1XqySMIyHkIlmcH+d95vcnk5RRRHIRVbqZhlEPvLGtcEPJA+Lq4CPccc73vDWeQRNsFgl2ARpDdImEElNioPZVvUVLgdNBw94tevyumcKwhzXGUylNJE0oNkSbA6fbKh9kV7z/W2K89Ou53a9b16n10dV20ev0RE6QR56j3roDA3QEFG0RL/Qb/THOXBOnY/O58q6u1PHvERrzfn6F6YznEM=AAAEinichZNNb9MwGMe9LcAoAzrgxsWiQxqXKtmFt8tUOOxYJLpNarrKcZ521vwS2Q5rGuW7cIVvxLfBeem0pkVYivW3f/8n9vPYjhLOjPX9Pzu7e96Dh4/2H3eeHDx99rx7+OLcqFRTGFHFlb6MiAHOJIwssxwuEw1ERBwuopsvJb/4AdowJb/bLIGJIHPJZowS66am3VfhUB+fTX0cChbjMJrlWfFu2u35fb9qeFMEjeihpg2nh3tXYaxoKkBayokx48BP7CQn2jLKoeiEqYGE0Bsyh7GTkggwk7zafoHfupkYz5R2n7S4mr0fkRNhTCYi5xTEXps2Kye3sXFqZx8mOZNJakHSeqFZyrFVuKwFjpkGannmBKGaub1iek00odZVrBNKuKVKCCLjPITEFFXPuJIuHw33qVu/7MCSYj3MsHlR9aJNIme+JtbRWxaDU6FIWxaR/sdQ/sNYootaXoXVYN0TA78zVXqra1E7Fv+GUbmIuHO0LMs6frk1XqySsIzHkIt28GCdD9rcnU5ZRRGpRV7pdhlEMQ4mjSGMFI/Lq4CPcC842vA2eURtsFgl2AZZA7I2EGlDyoPZVvUVrgZtB494vevquuca4gI3GcyUslJZMGwJLodPLtS9yKD9/jbF+Uk/8PvBt6B3Omje5j56jd6gYxSg9+gUnaEhGiGKlugn+oV+ewfeiffR+1xbd3eamJdorXlf/wK8j5k0

Pr(y | H0) vs. Pr(y | HA)AAAEr3ichZNdb9MwFIa9rcAoXx1ccmOtQ9qEVCW74eOqlJtdFomuk5q2OM5Ja82OI9vZ2kb5LfBnuIVr/g3OR6c1LcJSrDfneU/scxz7MWfaOM6fvf2DxoOHjw4fN588ffb8Revo5aWWiaIwoJJLdeUTDZxFMDDMcLiKFRDhcxj6159zPrwBpZmMvpplDGNBZhELGSXGhqatD15fnaaeH+Jlhj3BAnwxdc6wZ2BhlEjxje7Y+Lbn09m01XY6TjHwtnAr0e4ee29/IIT606ODiRdImgiIDOVE65HrxGacEmUY5ZA1vURDTOg1mcHIyogI0OO0qDHDb2wkwKFU9okMLqL3M1IitF4K3zoFMXNdZ3lwFxslJnw/TlkUJwYiWi4UJhwbifOG4YApoIYvrSBUMbtXTOdEEWpsW5teBLdUCkGiIPUg1lkxMy4jW4+C+9Sun09gSLaZptksK2ZRJ741z4mx9JYFYJUnkppFJP8x5N/QhqislBOveNn0BMDvTIXe6VqUjsW/oZ8vIu4cNcuqzF/tzBfrIgzjAaSintzb5L06t6eTd1H4cpEWut4GkY3ccWXwfMmD/FfAJ7jtnmx5qzr8OlisC6yDZQWWdSCSiuQHs6vra1y81B3c5+Wui989VRDkF7CoIJTSRNKAZiuwNXy0qfZGuvX7ty0uzzuu03G/uO1uD5XjEL1Gx+gUuegd6qIL1EcDRNF39BP9Qr8bbmPYmDS+ldb9vSrnFdoYDfYX+smooA==AAAEr3ichZNNb9MwGMe9LcAobx0cuVjrkDYhVQkXXk6lXHYsEl0nNW1xnKetNTuObGdrGuVbcOfTcIXzvg1O0k5rWoSlWP88v/8T+3kcBzFn2rju7d7+gfPg4aPDx40nT589f9E8enmhZaIo9KnkUl0GRANnEfQNMxwuYwVEBBwGwdWXgg+uQWkmo28mjWEkyCxiU0aJsaFJ86PfU6eZH0xxmmNfsBCfT9wz7BtYGCUyfK3bNr7t+Xw2abbctlsOvC28lWh1jv23P247aW9ydDD2Q0kTAZGhnGg99NzYjDKiDKMc8oafaIgJvSIzGFoZEQF6lJU15viNjYR4KpV9IoPL6P2MjAitUxFYpyBmruusCO5iw8RMP4wyFsWJgYhWC00Tjo3ERcNwyBRQw1MrCFXM7hXTOVGEGtvWhh/BDZVCkCjMfIh1Xs6My8jWo+A+tesXExiSb6ZpNsvLWdRJYM1zYiy9YSFY5YukZhHJfwzFN7QhKq/k2C9fNj0h8DtTqXe6FpVj8W8YFIuIO0fNsqzylzvzxboIw3gImagndzd5t87t6RRdFIFcZKWut0HkQ2+0MviB5GHxK+AT3PJOtryrOoI6WKwLrIN0BdI6EMmKFAezq+trXL7UHTzg1a7L3z1TEBYXsKxgKqWJpAHNlmBr+GRT7Y306vdvW1y8a3tu2/vqtTpdVI1D9Bodo1Pkofeog85RD/URRT/RL/Qb/XE8Z+CMne+VdX9vlfMKbQyH/QXnPaomAAAEr3ichZNNb9MwGMe9LcAobx0cuVjrkDYhVQkXXk6lXHYsEl0nNW1xnKetNTuObGdrGuVbcOfTcIXzvg1O0k5rWoSlWP88v/8T+3kcBzFn2rju7d7+gfPg4aPDx40nT589f9E8enmhZaIo9KnkUl0GRANnEfQNMxwuYwVEBBwGwdWXgg+uQWkmo28mjWEkyCxiU0aJsaFJ86PfU6eZH0xxmmNfsBCfT9wz7BtYGCUyfK3bNr7t+Xw2abbctlsOvC28lWh1jv23P247aW9ydDD2Q0kTAZGhnGg99NzYjDKiDKMc8oafaIgJvSIzGFoZEQF6lJU15viNjYR4KpV9IoPL6P2MjAitUxFYpyBmruusCO5iw8RMP4wyFsWJgYhWC00Tjo3ERcNwyBRQw1MrCFXM7hXTOVGEGtvWhh/BDZVCkCjMfIh1Xs6My8jWo+A+tesXExiSb6ZpNsvLWdRJYM1zYiy9YSFY5YukZhHJfwzFN7QhKq/k2C9fNj0h8DtTqXe6FpVj8W8YFIuIO0fNsqzylzvzxboIw3gImagndzd5t87t6RRdFIFcZKWut0HkQ2+0MviB5GHxK+AT3PJOtryrOoI6WKwLrIN0BdI6EMmKFAezq+trXL7UHTzg1a7L3z1TEBYXsKxgKqWJpAHNlmBr+GRT7Y306vdvW1y8a3tu2/vqtTpdVI1D9Bodo1Pkofeog85RD/URRT/RL/Qb/XE8Z+CMne+VdX9vlfMKbQyH/QXnPaomAAAEr3ichVNdb9MwFPW2AKN8dfDIi0WHtL1UCS98PI3yssci0XVS0xbHuems2XFkO1vTKL+FX8MrPPNvcD46rWkRlmId33Nu7HPtGyScaeO6f/b2D5wHDx8dPu48efrs+Yvu0csLLVNFYUQll+oyIBo4i2FkmOFwmSggIuAwDq6/lPz4BpRmMv5msgSmgixiFjFKjA3Nux/9oTrJ/SDCWYF9wUJ8PndPsW9gaZTI8Y3u2/i25vPpvNtz+2418DbwGtBDzRjOjw5mfihpKiA2lBOtJ56bmGlOlGGUQ9HxUw0JoddkARMLYyJAT/PKY4Hf2kiII6nsFxtcRe9n5ERonYnAKgUxV7rNlcFd3CQ10YdpzuIkNRDTeqMo5dhIXBYMh0wBNTyzgFDF7FkxvSKKUGPL2vFjuKVSCBKHuQ+JLqqZcRlbPwrus3b/cgJDis00zRZFNYs2E1jxFTGWvWUhWOSLtCUR6X8E5T+0Iaqo4cyvFpuaEPidqMI7Vctasfw3GZSbiDtFS7Kq81c788XahGE8hFy0kweb/KDN29spqygCucwr3C6DKCbetBH4geRh+RTwMe55x1vaxkfQJpZrg20ia4isTYi0YcqL2VX1NV0t2goe8PrU1XPPFYRlA1YOIilNLA1otgLr4ZNNtR3ptftvG1y863tu3/vq9c4GTW8eotfoDTpBHnqPztA5GqIRougH+ol+od+O54ydmfO9lu7vNTmv0MZw2F/9macX

Condition on data

Which Do We Want?• Evidence one trt is superior to the other?

‣ Estimate with precision?

• Decision rule regarding hypothesis?

‣ “Yes” reject? “No” do not reject?

• Decision rule regarding next step?

‣ Continue to next phase of study?

‣ Approve the treatment for indication?

�12

Decision Theory & Clinical Trials• Why decision theory?

‣ Clinical trials: Purpose is to lead to decisions

- What dose(s) to use?

- How best to apply the therapy?

- What is the next step for evaluating therapy?

- Should patients receive this therapy from now on?

- Which patients receive the most benefit?

‣ Put results in context via formal decision analysis �13

Why not make decisions explicit and coherent?

Decision Theory & Clinical Trials• Clinical trial design involves decisions, too

‣ Sample size,

‣ Duration of follow-up,

‣ Stopping rules,

‣ Whether to run the study in the 1st place

�14

Why not make these decisions explicit and coherent?

• “Value model”

‣ Weight each “value” (utility for each outcome)

‣ Develop scenarios (simulate the trial)

‣ “Multiply each grade by the weight of each value and add up the numbers for each scenario. The scenario with the highest score wins.”

Decision Theory• Consider

‣ Set of possible actions:

- E.g., stop the study; move to phase 3

‣ Set of possible outcomes:

‣ Parameters characterizing stochastic nature:

- E.g., treatment effect, model parameters

‣ Utility function:

�16

Y

⇥

A

u(a)

Bayesian Optimal Design• “Bayes” action maximizes expected utility

‣ Expectation to account for sources of uncertainty

- Uncertainty in parameters

- Variation in data resulting from action

- Choose:

�17

U(a) =RY

R⇥ u(y)pa(y | ✓)p(✓)d✓dy

p(✓)

pa(y | ✓)

a⇤ = argmaxU(a)

Application• Cancer & Leukemia Group B wanted to study Taxol

‣ Large population of women

‣ 3-hour infusion

‣ Many participating hospitals

‣ Outpatient

�18

Problem• Cannot carry out extensive sampling

‣ Large study

‣ Many institutions

• Devise limited-sampling scheme

‣ Optimal sampling times

�19

Stroud JR, Müller P, Rosner GL. Optimal sampling times in population pharmacokinetic studies. Applied Statistics. 2001;50(3):345-59.

!20

Objective: Maximize Precision

Toxicity risk = f(AUC) or g(Tcv)

!21

Paclitaxel PK Sampling Times

/

3 1 2

oVm

oKm

k 21

k 31

k 13

KmmV /

infusion

Time (Hours)

Log

Con

cent

ratio

nlo

g(µM

/L)

0 3 5 10 15 20 25

-4-3

-2-1

01

DrugInfusion

100 mg/m275 mg/m250 mg/m2

!22

Optimal Sampling Times for AUCu(t, y) =

Z{�(✓)� E [�(✓) | y, Y, t]}2 p(✓ | y, Y, t)d✓

��1

� kpX

i=1

t2i I{ti>8}

n=1 n=2Cost Coeff

k t* U* t* U*0.00000 (3) 0.53 (3,25) 0.900.00004 (3) 0.53 (3,10) 0.740.00008 (3) 0.53 (3,7) 0.70

Dose Optimization

TARGETAUC too

low!AUC too

high!

�23

Asymmetric Loss Function• Want AUC in “optimal” range

• Loss function

�24

AUCll AUC AUCul

100 200 300 400 500

010

2030

4050

AUC

Loss

L(auc) =

8><

>:

L�(auc,AUCll) if auc < AUCll0 if AUCll auc AUCulL+(auc,AUCul) if auc > AUCul

Posterior for Pt’s PK Parameters

�25

1st pt in 3rd study, accounting for studies 1 & 2

100 200 300 400 500

010

2030

4050

AUC

Loss

Optimal Dose w.r.t. Posterior

Loss Function Expected Loss

�26

E[u(y, d, ✓)] =

ZL [AUC(y)] p(y | d, ✓)p(✓ | Data)dyd✓

Learn About Benefit:Risk• Weigh chance of benefit

against risk of adverse event

‣ Dose finding based on trade-offs between probs of efficacy and toxicity

�27

Thall PF, Cook JD (2004). Dose-Finding Based on Efficacy-Toxicity Trade-Offs. Biometrics 60, 684-693.

Trippa: Response-Adaptive Designs for Basket Trials• Strategies for choosing among baskets

• Objective function is Bayesian decision rule (Exp. utility)

‣ Decisions during study work toward maximum information at the end of the study

‣ Fully sequential hard!

‣ Bayesian Uncertainty Directed (BUD) designs

- Approximate optimal decision rules (myopic)

- Goal: Design rule that approximates Bayesian decision and that one can compute

�28

Utility Function: Information Measure• Information measure consistent with goal of study

‣ E.g., entropy of estimate, posterior variance, entropy of posterior distribution of an indicator

- Entropy of posterior distribution

�29

X ⇠ p(X), H[p(X)] = �E[log p(X) | Data]AAAEsXichZNdb9MwFIa9rcAoXx1ccmOtQxoCqgQuQCCkqoC0yyHRrajOiuOctlbtOMQOaxvlv8Cv4RYu+Tc4H53WrAhLsV77eU/sc2z7keDaOM6fre2dxrXrN3ZvNm/dvnP3Xmvv/olWScygz5RQ8cCnGgQPoW+4ETCIYqDSF3Dqz97l/PQbxJqr8JNZROBJOgn5mDNq7NSo9WaAieYSR4eDx08x+ZrQAB8N85GH3+JnH4ZEqElBMZE8wMTA3KTvqaGZN2q1nY5TNHxVuJVod/fJk+8IoePR3s4ZCRRLJISGCar10HUi46U0NpwJyJok0RBRNqMTGFoZUgnaS4ssM/zIzgR4rGL7hQYXs5cjUiq1XkjfOiU1U11n+eQmNkzM+JWX8jBKDISsXGicCGwUzkuGAx4DM2JhBWUxt3vFbEpjyowtbJOEcM6UlDQMUgKRzoqeCxXafGK4TO36eQe2eOthmk+yopd14lvzlBpLz3kAVhGZ1Cwy+Y8h/4c2NM5KeUaKwbonAHFhKvRG17x0zP8N/XwReeGoWZZl/HJjvFwlYbgIIJX14N4679W5PZ28itJX87TQ9TLIbOh6lYH4SgT5VcAHuO0eXPFWefh1MF8lWAeLCizqQCYVyQ9mU9VXuBjUHcIX5a6L657GEGS4ymCslAmVAc2XYHN4bUPti3Tr7++qOHnecZ2O+9Ftd3uobLvoIdpHh8hFL1EXHaFj1EcM/UA/0S/0u/Gi8bnxpeGX1u2tKuYBWmuN2V/DrqhbAAAEsXichZPdbtMwFMe9rcAoH+vgkhtrHdIQUDVwAQIhVQWkXQ6JbkV1VhzntLVqxyF2WNsob8EL8DTcwuXeBuej05oWYSnW3/79T+xzbHuh4Nq025db2zu1Gzdv7d6u37l77/5eY//BqVZxxKDHlFBR36MaBA+gZ7gR0A8joNITcOZN32f87DtEmqvgs5mH4Eo6DviIM2rs1LDxto+J5hKHR/0nzzD5FlMfHw+ykYvf4ecfB0SocU4xkdzHxMDMJB+ooak7bDTbrXbe8LpwStHsHJCnPy4785Ph/s458RWLJQSGCar1wGmHxk1oZDgTkNZJrCGkbErHMLAyoBK0m+RZpvixnfHxSEX2CwzOZ69HJFRqPZeedUpqJrrKsslNbBCb0Ws34UEYGwhYsdAoFtgonJUM+zwCZsTcCsoibveK2YRGlBlb2DoJ4IIpKWngJwRCneY9Fyqw+URwndr1sw5s8VbDNB+neS+rxLPmCTWWXnAfrCIyrlhk/B9D9g9taJQW8pzkg1WPD+LKlOuNrlnhmP0betki8spRsSyK+MXGeLlMwnDhQyKrwd1V3q1yezpZFaWnZkmuq2WQ6cBxSwPxlPCzq4APcdM5XPOWeXhVMFsmWAXzEsyrQMYlyQ5mU9WXOB9UHcITxa7z655E4Ke4zGCklAmUAc0XYHN4Y0Pti3Sq729dnL5oOe2W88lpdrqoaLvoETpAR8hBr1AHHaMT1EMM/US/0G/0p/ay9qX2teYV1u2tMuYhWmm16V+wIqnhAAAEsXichZPdbtMwFMe9rcAoH+vgkhtrHdIQUDVwAQIhVQWkXQ6JbkV1VhzntLVqxyF2WNsob8EL8DTcwuXeBuej05oWYSnW3/79T+xzbHuh4Nq025db2zu1Gzdv7d6u37l77/5eY//BqVZxxKDHlFBR36MaBA+gZ7gR0A8joNITcOZN32f87DtEmqvgs5mH4Eo6DviIM2rs1LDxto+J5hKHR/0nzzD5FlMfHw+ykYvf4ecfB0SocU4xkdzHxMDMJB+ooak7bDTbrXbe8LpwStHsHJCnPy4785Ph/s458RWLJQSGCar1wGmHxk1oZDgTkNZJrCGkbErHMLAyoBK0m+RZpvixnfHxSEX2CwzOZ69HJFRqPZeedUpqJrrKsslNbBCb0Ws34UEYGwhYsdAoFtgonJUM+zwCZsTcCsoibveK2YRGlBlb2DoJ4IIpKWngJwRCneY9Fyqw+URwndr1sw5s8VbDNB+neS+rxLPmCTWWXnAfrCIyrlhk/B9D9g9taJQW8pzkg1WPD+LKlOuNrlnhmP0betki8spRsSyK+MXGeLlMwnDhQyKrwd1V3q1yezpZFaWnZkmuq2WQ6cBxSwPxlPCzq4APcdM5XPOWeXhVMFsmWAXzEsyrQMYlyQ5mU9WXOB9UHcITxa7z655E4Ke4zGCklAmUAc0XYHN4Y0Pti3Sq729dnL5oOe2W88lpdrqoaLvoETpAR8hBr1AHHaMT1EMM/US/0G/0p/ay9qX2teYV1u2tMuYhWmm16V+wIqnhAAAEsXichZNdb9MwFIa9LcAoH+vgkhuLDmlIMCVwAQIhTQWkXQ6JbkVNVhzntLVqx8F2WNsoP4Zfwy1c8m9wPjqtaRGWYr32857Y59gOE860cd0/W9s7zo2bt3Zvt+7cvXd/r73/4EzLVFHoUcml6odEA2cx9AwzHPqJAiJCDufh9H3Bz7+D0kzGn808gUCQccxGjBJjp4btt33sayZwcth/+gz731IS4ZNBMQrwO/z848DnclxS7AsWYd/AzGQfiCF5MGx33CO3bHhdeLXooLqdDvd3LvxI0lRAbCgnWg88NzFBRpRhlEPe8lMNCaFTMoaBlTERoIOszDLHT+xMhEdS2S82uJy9HpERofVchNYpiJnoJismN7FBakavg4zFSWogptVCo5RjI3FRMhwxBdTwuRWEKmb3iumEKEKNLWzLj+GSSiFIHGU+JDove8ZlbPNRcJ3a9YsObPFWwzQb52UvmiS05gkxll6yCKzyRdqwiPQ/huIf2hCVV/LCLwerngj4lanUG12zyjH7NwyLRcSVo2FZVPGLjfFimYRhPIJMNIO7q7zb5PZ0iiqKUM6yUjfLIPKBF9QGP5Q8Kq4CPsAd72DNW+cRNsFsmWATzGswbwKR1qQ4mE1VX+Jy0HTwkFe7Lq97piDKcZ3BSEoTSwOaLcDm8MaG2hfpNd/fujh7ceS5R94nr3Pcrd/mLnqEHqND5KFX6BidoFPUQxT9QD/RL/Tbeel8cb46YWXd3qpjHqKV5kz/AsZ+ptI=

see also Piantadosi (2005) Clinical Trials 2:182-192

Can Incorporate Frequentist Criteria in Utility Function• Bayesian design optimizes expected utility

‣ Utility function can include different considerations

- Sample size or cost

- Precision

- Number of patients who benefit

- Prediction of future study outcomes

‣ E.g., Anscombe (’63), Berry & Ho (’88), Lewis & Berry (’94), Carlin, Kadane, & Gelfand (’98), Stallard, Thall, & Whitehead (’99), Lewis, Lipsky, & Berry (’07), Trippa, Rosner, & Müller (’12), Ventz & Trippa (’15)

�30

Screening designs for drug development 597

Finally, we conclude in Section 7 with a final discussion of features and limitations of the proposedapproach.

2. DRUG SCREENING

Our approach is based on casting the screening process as a formal decision problem. The basic ingredi-ents of a decision-theoretic setup are an action space A of possible decisions d ∈ A, a probability modelp(θ, y) for all relevant random variables, including parameters θ and future data y, and a utility func-tion u(d, θ, y). The probability model is conveniently factored into a prior probability model p(θ) and asampling model p(y|θ). It can be argued (DeGroot, 2004) that a rational decision maker should choosean action in A to maximize the expectation of u. The expectation is with respect to p, conditioning onall data observed at the time of decision making, and marginalizing over all parameters and all futuredata. Sometimes, the action space is restricted to decisions that satisfy certain constraints, for exampleprespecified bounds on type I and type II errors (false-positive and false-negative rates). In such cases, themaximization is carried out over the restricted set.

2.1 Action space and probability model

Let yti be the outcome at time t = 1, . . . , T for treatment i ∈ At , where At is the set of treatments beingconsidered at time t . We assume a finite time horizon T for the entire screening process, and we allow fora random number of treatments at any given time t .

After observing the outcomes yti , i ∈ At , we make a sequential stopping decision dti for each treat-ment. We denote with dti = 0 the action of removing treatment i from At and with dti = 1 the action ofcontinuing recruitment for treatment i . If we decide dti = 0, then a terminal second-step decision ai indi-cates whether to abandon treatment i (ai = 0) or whether to recommend to proceed with a confirmatoryphase III study (ai = 1).

Finally, before the next decision at time t + 1, new treatments might be proposed and added tothe set At+1. Let "nt denote the number of new treatments arising in period t and denote with π j =Pr("nt = j), for j = 0, 1, . . . , its probability distribution. In the last period, T , continuation is notpossible. That is, dT i = 0 for all i ∈ AT . Figure 1 illustrates the sequence of decisions and observations.

Fig. 1. Multiple binomial experiments are available at time t . Some of them are dropped, and some are introduced atthe end of period t .

Platform or Master Protocols

• At any one time, multiple phase II studies

�31

Rossell, Müller, Rosner (2007) Biostatistics Ding, Rosner, Müller (2008) Biometrics

Decisions• At each analysis-decision time t

‣ Make decision dti for current study or trt

- Abandon current study trt

- Stop study and move to phase 3

- Continue current study

• Could have multiple trt-studies at t

‣ Then have

‣ Only considering one at a time now�32

dti = 1:

dti = 2:

dti = 3:

dt = (dt1, dt2, . . . , dtk)

dt = dt1

�33

Utility Function• Utility at decision-time t (for current trt)

€

ut (dt ,θ,Yt ,YIII ) =

€

−{c1 × n1 × t + c2 × n2}+b ×{θnew −θold}I[z>z1−α ]

€

−c1 × n1 × t if stop & discard

Gain if “significant” phase 3;

gain proportional to effect

Phase 3 sample size & cost

Predict phase 3 outcome

�34

Stopping Boundaries

2000 sims

5 per cohort c1 = 0.14 c2 = 0.7 b = 290 n2 = 96✓0 = 0.2

✓A = 0.5

✓old ⇠ Beta(20, 80)

Our Colleagues in Other Disciplines Want Our Help• Work with colleagues in other fields

• They want our help

�35

Operskalski JT, Barbey AK (2016). Risk literacy in medical decision-making. Science 352:413-414

Clin Chem 62:1285-6, 2016

Opportunities• Work with colleagues in other fields

�36

Summary: Why Bayes?• Easier to combine or incorporate information

‣ Bayesian paradigm corresponds to learning

- External information feeds priors

• Frequentist methods seem impractical in some cases

‣ Evidence of treatment benefit for rare diseases

• Interest in complex designs & decision making

‣ Outcome adaptive randomization

‣ Matching treatments to subgroups

�37

Conclusions• Clinical drug development involves learning &

confirming

• Bayesian inference has a place in regulatory science

• Hypothesis testing in and of itself is not evil

‣ Need better measures of weight of evidence than p

• Clinical research involves decisions

‣ Incorporate statistical decision theory

‣ Include predictive dist’n for frequentist tests

�38

Thank You!