FACTS6.3Capabilities 1

FACTS6.3CapabilitiesBerryConsultants’FixedandAdaptiveClinicalTrialSimulation(FACTS)softwareisthemostpowerful,versatile,andfastestsimulationtoolonthemarkettodayforadvancedclinicaltrialdesign.ThisdocumentprovidesasummaryofthekeycapabilitiesandfeatureswithinFACTS.Oneofthedifficultiesindevelopingasoftwareforageneral-purposeclinicaltrialsimulatoristheextra-ordinarydiversityoffeaturesrequiredfordifferenttrials.Anyattempttobuildasinglesimulatorseemstobedoomedtofailureeitherbecauseitlackstoomanycriticalfeaturesorithassomanyfeaturesthatitbecomeshardtouse,unreliable,anddifficulttoextend.Alternativelybuildingmanydifferentsimulators,eachforadifferenttrialtype,tendstoleadtosimulatorsthataretoospecifictoaparticulardesign.Whilsttheresultingsimulatorissimpletouse,itistoolikelytolackcriticalfeaturesthatyourdesignneeds.Also,unlessthemanysimulatorsarescrupulouslymaintained,theyarelikelytodivergewhichwillmakeitdifficulttocomparetheirresultstodeterminewhichisbest.FACTSsolvestheseproblemsbybeingbuiltasspecificsimulationengines,eachofwhichspansawholeclassoftrialdesigns.Eachclassofdesignisaimedatansweringaverydifferentclinicalquestionandthusanydivergenceintheenginesisunlikelytobeanissue.

The FACTS Simulation Engines FACTSCore Fortrialswhereoneormoretreatmentarmsarebeingtested.

Theengineallowsforthesearmstobedifferentdosesandincludedoseresponsemodelling.Theprimaryclinicalquestionsbeingaddressedare:“isthereatreatmenteffectandisitbigenoughtobeclinicallyrelevant?”andpossibly“whicharmshouldbeselectedforfurtherdevelopment?”.

FACTSEnrichmentDesigns Fortrialswhereasingletreatmentisbeingtestedinanumberofdifferentsettings–thesecouldbedifferentpatientsubgroupsordifferentindications.Theprimaryclinicalquestionsbeingaddressedare:“isthereatreatmenteffectandisitbigenoughtobeclinicallyrelevant?”andpossibly“inwhichpatientsubgroups/whichindicationsshoulddevelopmentcontinue?”.

FACTS6.3Capabilities 2

FACTSDoseEscalation Fortrialswhereatreatmentistobetestedatdifferentdoselevelsinadoseescalatingmannertofindwhichdoseisthemaximumlikelytobetolerated.UsedalmostexclusivelyforOncologyphase1trials,theprimaryclinicalquestionbeingaddressedis“whatisthemaximumclinicallytolerabledoseatwhichthisdrugcanbeused?”.

FACTSStagedDesigns Forevaluatingtheperformanceofthedesignsoftwoconsecutivetrials,oratrialthatseamlesslyhastwoseparatestages.Theindividualtrials/stagesaresimulatedusingtheFACTSCoreengine,FACTSStagedDesignsmanagesthesimulationofthesubsequenttrial/stageinthelightofwhat’slearnedinthefirsttrial/stage–whetherthereistobeasecondtrial/stage,whichtreatmentarmsaretobeused,howbigitshouldbe,whentoswitchfromthefirsttrial/stagetothesecondstage.Italsoallowsdatafromthefirsttrial/stagetobeincludedintheanalysisofthesecondtrial/stage.

TheresultofthisisthatforanyparticulartypeoftrialalltheoptionsinFACTSareavailableandcanbeseparatelyenabledordisabled.Thisallowstrialdesignstostartsimpleanddifferentoptionstobeintroducedoneatatime–allowingtheirindividualimpacttobeassessed.ThismakesFACTSagreatplatformforexperimentingwithandlearningaboutdifferentpossiblestatisticalapproachessuchaslongitudinalmodelling,BayesianaugmentationoftheControldata,hierarchicalmodellingofresponseacrosssubgroupsandresponseadaptiverandomization.

Common Features Across the Engines • CommonGraphicalUserInterface(GUI):theFACTSuserinterfaceisveryconsistent

acrossallthesesimulationengines,simplifyingtheadmittedlycomplextaskofmasteringFACTS.

• Noprogramming:allthestatisticalmodelsarepre-programmed,theuserselectswhichtouse,whichoptionstouse,andspecifiesthepriors.

• Fast:becausethestatisticalmodelsarepre-programmed,FACTSisabletorunsimulationsmuchfaster(10-100times)thanistypicallyachievedbyforexampleR-codedtrialsimulationscripts.Andifthatisn’tfastenough,FACTSautomaticallyspreads

FACTS6.3Capabilities 3

itssimulationsacrossalltheavailableprocessorcoresandfurthermorecanbeconfiguredtomakeuseofaseparate“computegrid”.

• VirtualSubjectSimulation:Flexiblesimulationmethodsforcorrelatedresultsacrossvisits,variationbyvisitandbytreatmentarmandpopulationsub-group.Simulationisabletosimulatefrommodelsdifferentfromtheanalysismodelsbeingused.Finally,thereistheabilitytoimportfilesofvirtualsubjectresponses–subjectresponsesarethensimulatedbyforsamplingfromthisfile,allowingthesimulationofanycorrelationdesiredovervisitsorbetweenendpoints.

• SimulatingExecution:FACTSincludestheflexibilitytosimulatecomplexaccrualrates(Poissonprocessrandomaccrual)overregionsandsites,accrualinCohorts.FACTSalsoincludestheabilitytosimulatecomplexdrop-outscenarios.

• DesignVariants:makeiteasytocomparetheperformanceofadesignatdifferentsamplesizes,notjustcomparingthetype-1errorandpowerbutalsothedesign’sabilitytomakecorrectdecisionssuchasselectionofthebestdoseandidentificationofthecorrecttreatmentpopulationorindication.

• ResultsOutput:Extensiveoutput(.csvfiles)ofallanalysesareprovided:summarizedacrosstrials,finalresultofeachtrial,eachinterimofeachtrial,andthesimulatedpatient-leveldataforeachtrial.ThisgivestheuserthefreedomtoexplorefullfrequentistorBayesiansummaries,time,patients,interims,etc..ThereisafacilitytooutputMCMCsamplesfordiagnostics.

• TrialExecution:Thesimulationengineexecutableprogramcanbeusedintheanalysisofactualrunningtrials.Theengineoutputsfullanalysisandcancreateupdatedrandomizationlistsforrecruitmentpostinterim.An“Analysis”tabwithintheFACTSGUIallowsinterimorfinalanalysestoperformed,usingthespecifiedtrialdesign,onactualorvirtualdata.

• Graphics:Thereisin-builtgraphicalpresentationofmanyoperatingcharacteristicsofthesimulatedtrials,andsummariesofallanalyses.Aswellasgraphsofsummarydata,therearegraphicalplotsofthedataandanalysisforeachinterimforeverysimulatedtrial.Finally,thereistheabilitytoimportthesimulationresultsintoRforfurtheranalysis,summaryandgraphing.

FACTS6.3Capabilities 4

FACTS Core Designs • TheseareEnginesforsimulatingdose-finding/treatmentselectiontrialswithendpoints

thatarecontinuous,dichotomousortime-to-event(TTE),ortrialwithmultipleendpoints(upto4).

• TrialTypes:Trialscanbeanalyzedforsuperiority,super-superiority(asdefinedbyasuperioritymargin),non-inferiority.Treatmentarmselectioncanbebasedonresponse,utilitycombinationofmultipleendpoints,orbesttherapeuticdosesoverefficacyandsafety.

• Dose-ResponseModels:TheseincludeNDLMs,sigmoid(EMAX),logistic,hierarchicallogistic,plateaumodel,inverted-U.SpecificationofBayesianpriors,andabilitytousehistoricaldatafortheControlarm.

• 2DDose-ResponseModels:FromFACTS6.3onwardsitispossibletospecifythatthetreatmentarmslieonagridandtoanalyzetheresponseusingfactorialor2D-NDLMresponsemodels.

• LongitudinalModels:Theseincludelinearregression,kerneldensityestimates,beta-binomials,dichotomizedcontinuousendpoints,time-to-failurefordichotomous,time-to-eventwithpiecewiseexponentialsorBayesianCoxmodeling.ForTTEendpointsthereistheabilitytomodelanearlypredictorendpoint(continuous,dichotomous,TTE).Therearevariousoptionsforhandlingincomplete/lost-to-follow-upsubjects:LOCF,BOCF,imputedfailureorsuccess.

• Adaptations:Theseincludearm-dropping,responseadaptiverandomization(RAR),earlystoppingforsuccessorfutility,earlygraduationtothenext“stage”.RARcandefinetheweightinggiventomultipleadaptationtargetsandfixedratios,allocationcanbedefinedrelativetothe“probability”ofthetargetortheamountofinformationresultingfromallocatingtothetarget.Theburn-in(pre-adaptation)periodcanbedefinedintermsoffixedallocationratios,sizeandrandomizationblocksize.

• QuantitiesofInterest(QOIs):UserscanspecifyahugerangeofdifferentQOIstobecalculatedduringeachanalysis.Thesecancompareanyspecificarm,allarmsagainstanyspecificarm,andwithanyDelta,thusmultipleprobabilitiesofbeingMED(MinimumEfficaciousDose)andED(EffectiveDose)withdifferentdeltasandquantilescanbecalculated.

• Multiple‘decision’quantitiescanbecalculatedusinganyQOIcalculatedforallarms,selectedbyaspecificarm,thearmmostlikelytobea“target”(Max,MEDorED)orarmwiththeminimumormaximumvalueofaQOI.

FACTS6.3Capabilities 5

• QOIs:ThesecanbeBayesianPosteriorprobabilities,P-valuesorpredictiveprobabilityofsuccess.Thislatterprobabilitycanbecalculatedbasedonthecurrentlyenrolledsubjects(allowingthetrialtostopenrollmentandcompletefollowupwhentheprobabilityofsuccessisveryhigh)orcompletingfullenrollment(allowingthetrialtostopforfutilitywhentheprobabilityofsuccessisverylowevenifthetrialfullyenrolls).

• Interims:Thesecanbedefinedbynumberofsubjectsenrolled,numberofsubjectscomplete,subjectscompleteuptoaparticularvisit,time-intervals,ornumberofevents(forTTE).Interimscanbeconductedduringandafterfullaccrual.Optionsfordiscontinuingfollow-upafterearlystopping.

• DesignReport:FACTSCoredesignsarenowself-documenting,attheclickofabuttonFACTSwillgenerateaMSWorddocumentthatdescribesthedesignthathasbeenspecifiedandthesimulationresults(requiresRandR-markdown).

FACTS Enrichment-Designs: • TheseareEnginesforsimulatingtrialswithtreatmenteffectsbypopulation

characteristics.Theyallowindication,populationfinding,andenrichmentdesignstobesimulated.Thereareversionsforcontinuous,dichotomous,andtime-to-eventendpoints.

• TrialTypes:Trialscanbeanalyzedforsuperiority,super-superiority(asdefinedbyasuperioritymargin),non-inferiority,withsuccessandfailuredefinedacrossallgroups,byspecificgroupsor“any”or“all”groups.Enrichmentdesignscanbesimulatedthatcandroplesssuccessfulgroupsatinterims.

• GroupModels:Theseincludetheabilitytomodelthedifferentgroupsusingindependentmodels(optionallyborrowingfromhistoricaldataonthecontrolarm)orBayesianhierarchicalmodelingorclusteredmodelling.

• LongitudinalModels:Theseincludelinearregression,kerneldensityestimates,beta-binomials,supportforcontinuousendpointsthataredichotomizedbasedonathreshold,time-to-failurefordichotomous,time-to-eventwithpiecewiseexponentialsorBayesianCoxmodeling.Variousoptionsforhandlingincomplete/lost-to-follow-upsubjects:LOCF,BOCF,imputedfailureorsuccess.

• Adaptations:Theseincludedroppinggroupsforfutilityorstoppingthewholestudyearlyforsuccessorfutility.Decisionscanbebasedonposteriorprobabilitiesorpredictiveprobabilityofsuccessinaspecifiedfuturetrial.Interimscanbedefinedbynumberofsubjectsenrolled,numberofsubjectscompleteorcompleteduptoaparticularvisit,time-intervals,ornumberofevents(forTTE).

FACTS6.3Capabilities 6

• SimulatingExecution:Abilitytosimulatedifferentaccrualratesforeachgroup,variationovertimeingroup’saccrualrates.Fullflexibilitytosimulatecomplexaccrualrates(Poissonprocessrandomaccrual)overregionsandsites.Abilitytosimulatecomplexdrop-outscenarios.

FACTS Dose-Escalation Designs • TheseEnginesareforsimulatingtrialswheredoseescalationtakesplacebasedon

observationsoftolerabilityand/orefficacy.Arangeofdesignsfromsimple(3+3,mTPI)tocomplexContinualReassessmentMethods(CRM).A2D-CRMengineincludedforphase1trialsescalatingthedoseof2drugsbeingusedincombination

• TrialTypes:Theseinclude:doseescalationtomaximumtolerateddose(MTD)basedonbinarytoxicityoutcomes,doseescalationtoMTDbasedonordinaltoxicityoutcomes,DoseescalationtoMTDandMEDbasedonajointmodelfortoxicityandefficacy,doseescalationin2“groups”whetherthegroupsaredefinedbysubject(e.g.adultorpediatric)orbytreatment(e.g.newdrugaloneornewdrugincombinationwithanexistingtreatment).FromFACTS6.2onwardsthesearenolongerseparatetrialtypesbutoptionsinthemainsimulatorthatcanbecombinedwitheachotherandwithalltheotheroptionsdescribedhere.

• An”open-enrolment”modeisincludedasfasteralternativetocohortenrolment.Theuserspecifiesasmaximumnumberofsubjectsthatcanhavebeenallocatedtoadose,nothaveafinalresponseandstillallocatesubjectstothatdose.Theusercanalsospecify“backfill”rulesothatavailablesubjectsthatcan’tbeallocatedatthetargetdosecanbeallocatedtoalowerdose.

• Dose-ResponseModels:Theseincludetwo-parameterlogistic;proportionaloddsmodelforordinaloutcomes.Bivariatemodelfortoxicityandefficacy.

• Priorspecificationtools:OnetrickyaspectoftheCRMmethodsisthespecificationofthebi-variateNormalprioronthemodelparameters,FACTSincludesanumberofmethodsforderivingthisprior.

• Informpriorwithpseudosubjects:Anaturalwaytoinclude‘informative’priorinformationtoaCRMdesignistouseaveryweakprioronthemodelparametersbutaddinasmall(fractionalsubjectsareallowed)amountofpseudodata.

• DoseRange:Explicitlydefineindividualdoselevelsoracontinuousdoserange.

FACTS6.3Capabilities 7

• Escalation:Abilitytospecifythesizeofacohort,withoptionforasmall-cohortrun-instage.Specifylimitsonescalation(e.g.maximumincrementfromonecohorttothenext,possiblydifferentindifferentpartsofthedoserange).Preventescalationtodoseswithhighprobabilityofoverdose.Variousoptionalearlystoppingrules.

• Openenrollment:Ratherthanrecruitingsubjectsincohorts,thetrialacceptsallcomers–uptoaspecifiedmaximumnumberofsubjectsrecruitedbutnotcomplete.Recruitmentcanbefasterandde-escalationdecisionstakensoonerallowingthesedesignstocompletefasterandmoreefficiently.

• TargetDoseDefinitions:ThetargetdefinitioncaneitherbetheMTDbyasinglequartileofthetoxicitycurve(e.g.25%)orbytheposteriorprobabilitythatthetoxicityrateatadoseliesinatoxicityinterval(e.g.16-25%).

FACTS Staged Design: • ThisEnginessimulatestwoFACTSCoretrialsbacktoback,eitherasseparatetrialsof

asasingletrialseamlesslycomprisedof2stages.• Thestagescanbeconnectedonascalefromcompletelyseamlesstocompletely

independent.• FACTSStagedDesigncan,forexample,beusedtosimulate:

o aPhaseIIandtheconsequentialPhaseIIItrials,oraseamlessPhaseII/IIItrialo aPhaseIIAandtheconsequentialPhaseIIBtrials,oraseamlessPhaseIIA/B

trialo aPhaseIItrialwithatreatmentarmselectionandexpansionstage

• Datainclusionoptions:TheoptionsforStage2toincludedatafromStage1include:useallofstage1data,usenostage1data,usestage1dataonthearmsretainedinthesecondstage,useallthestage1dataonthestudydrugarmsinthefirststagepooledontheonestudydrugarmretainedinthesecondstage,andusethejustthestage1datefromsubjectsfromthefirststagewhodidnotcompleteinthatstage.

FACTS6.3Capabilities 8

• Armselectionrules:rulesforselectingwhichtreatmentarmsarekeptinthesecondstageoraredroppedafterthefirststage,includingrulesonspecificarms(suchas“retainthetopdoseif…”),rulesonspecifictargetarms(suchas“retaintheMinimumEfficaciousDosewhichhasaHazardRatioofXorlesscomparedtotheControlArm”)rulesacrossallarms(suchas“retainthe2treatmentarmswiththehighestprobabilityofhavingaresponsegreaterthancontrol,aslongastheirprobabilityoftoxicityislessthan…”)andrulesappliedtogroupsoftreatmentarms(suchas“retainthetwoarmsthatareonceadaytreatmentsratherthanthetwothataretwiceadaytreatmentsif…”).

• Differentanalysismodels:Theallocationrules,interimsanddecisioncriteriacanbespecifiedseparatelyforeachstage,orthestage1rulescanbemirroredinstage2.

• PredictiveProbabilities:Possiblestage1QOIsincludepredictiveprobabilitiesoftheoutcomeofstage2.

• Output:Theoutputincludesfullsimulationoutputofbothstages.• Graphs:AswellasindependentgraphsoftheStage1andStage2outcomes,thereare

graphsofDoseSelectionandtheOverallresults.

Inter-operation with R • FACTScansimulatesubjectresponsebysamplingfromafileofvirtualpatients.This

meansthattheusercancreatesampledresponsesfromanydistributionandwithanycorrelationbetweenendpointsoracrossvisits.

• Thesimulationenginescreateeasytouse.csvfilesfromallsimulatedtrialsforveryeasyprocessingofuniqueneeds.Storesinterims,trials,andsummariesoftrials.TheseresultscanbeeasilyimportedintoRforadditionalpost-processingoftheresults.

Technology • FACTScomprisesaGUIandsimulationengines.• TheGUIrunsonallWindowsplatformsusingthe.NETenvironment • Thesimulationenginesarepre-compiledC++commandlineexecutablesthatcanrun

onWindowsorLinux,providingthefastestclinicaltrialsimulationengineavailable(byordersofmagnitude!).ThesimulationenginescanberunentirelyviatheGUI,thereisnoneedtointeractwiththemdirectly.

• Anysimulatedtrialiseasilyexecutedusingtheidenticalsimulationengineusedtosimulatethetrial.

FACTS6.3Capabilities 9

• Self-containedabilitytoparallelizesimulationstooptimizethenumberofprocessors • Fullabilitytoconnecttoagridandrunsimulatedtrialsinparallelonyourorother

grids(e.g.Amazon) • Allsupportedbyusersguides,comprehensivespecifications,numerousexamplesand

tutorials.

Formoreinformation,seewww.berryconsultants.comorcontactusat:info@berryconsultants.com