51
Introduction to Galaxy at UF HPC Oleksandr Moskalenko Assoc. Sci., UF HPC Center Biological Applications Support Matt Gitzendanner Assoc Sci., Biology/HPC Training UF Research Computing UF Research Computing Day 2011

UF Research Computing - Information Technology · PDF fileIntroduction to Galaxy at! UF HPC!! Oleksandr Moskalenko! Assoc. Sci., UF HPC Center! Biological Applications Support! Matt

  • Upload
    lekien

  • View
    217

  • Download
    0

Embed Size (px)

Citation preview

Page 1: UF Research Computing - Information Technology · PDF fileIntroduction to Galaxy at! UF HPC!! Oleksandr Moskalenko! Assoc. Sci., UF HPC Center! Biological Applications Support! Matt

Introduction to Galaxy at!UF HPC!

!Oleksandr Moskalenko!

Assoc. Sci., UF HPC Center!Biological Applications Support!

Matt Gitzendanner!Assoc Sci., Biology/HPC Training!

!

UF Research Computing

UF Research Computing Day 2011!

Page 2: UF Research Computing - Information Technology · PDF fileIntroduction to Galaxy at! UF HPC!! Oleksandr Moskalenko! Assoc. Sci., UF HPC Center! Biological Applications Support! Matt

Today’s research computing

Page 3: UF Research Computing - Information Technology · PDF fileIntroduction to Galaxy at! UF HPC!! Oleksandr Moskalenko! Assoc. Sci., UF HPC Center! Biological Applications Support! Matt

Approaches

Page 4: UF Research Computing - Information Technology · PDF fileIntroduction to Galaxy at! UF HPC!! Oleksandr Moskalenko! Assoc. Sci., UF HPC Center! Biological Applications Support! Matt

Approaches

Page 5: UF Research Computing - Information Technology · PDF fileIntroduction to Galaxy at! UF HPC!! Oleksandr Moskalenko! Assoc. Sci., UF HPC Center! Biological Applications Support! Matt

Approaches

Page 6: UF Research Computing - Information Technology · PDF fileIntroduction to Galaxy at! UF HPC!! Oleksandr Moskalenko! Assoc. Sci., UF HPC Center! Biological Applications Support! Matt

Approaches

Page 7: UF Research Computing - Information Technology · PDF fileIntroduction to Galaxy at! UF HPC!! Oleksandr Moskalenko! Assoc. Sci., UF HPC Center! Biological Applications Support! Matt

Command Line Environment

Login to head node!

Head node!

Interactive session or batch

submission!

Scheduler!

Your job runs on the

cluster!

Computing!resources!

Page 8: UF Research Computing - Information Technology · PDF fileIntroduction to Galaxy at! UF HPC!! Oleksandr Moskalenko! Assoc. Sci., UF HPC Center! Biological Applications Support! Matt

What is Galaxy?

Page 9: UF Research Computing - Information Technology · PDF fileIntroduction to Galaxy at! UF HPC!! Oleksandr Moskalenko! Assoc. Sci., UF HPC Center! Biological Applications Support! Matt

What is Galaxy?

✦ Computational biology platform!•  Open and Web-based!•  Accessible!•  Reproducible!•  Transparent!

Page 10: UF Research Computing - Information Technology · PDF fileIntroduction to Galaxy at! UF HPC!! Oleksandr Moskalenko! Assoc. Sci., UF HPC Center! Biological Applications Support! Matt

Galaxy Analysis Workspace

Page 11: UF Research Computing - Information Technology · PDF fileIntroduction to Galaxy at! UF HPC!! Oleksandr Moskalenko! Assoc. Sci., UF HPC Center! Biological Applications Support! Matt

Galaxy Analysis Workspace

Page 12: UF Research Computing - Information Technology · PDF fileIntroduction to Galaxy at! UF HPC!! Oleksandr Moskalenko! Assoc. Sci., UF HPC Center! Biological Applications Support! Matt

Galaxy Analysis Workspace

Page 13: UF Research Computing - Information Technology · PDF fileIntroduction to Galaxy at! UF HPC!! Oleksandr Moskalenko! Assoc. Sci., UF HPC Center! Biological Applications Support! Matt

Galaxy Analysis Workspace

Page 14: UF Research Computing - Information Technology · PDF fileIntroduction to Galaxy at! UF HPC!! Oleksandr Moskalenko! Assoc. Sci., UF HPC Center! Biological Applications Support! Matt

Galaxy Analysis Workspace

Page 15: UF Research Computing - Information Technology · PDF fileIntroduction to Galaxy at! UF HPC!! Oleksandr Moskalenko! Assoc. Sci., UF HPC Center! Biological Applications Support! Matt

Metadata

Page 16: UF Research Computing - Information Technology · PDF fileIntroduction to Galaxy at! UF HPC!! Oleksandr Moskalenko! Assoc. Sci., UF HPC Center! Biological Applications Support! Matt

Getting Data into Galaxy ✦ Upload a file from your computer!

•  scp to HPC and load from within Galaxy!•  Copy files to HPC using Samba!

✦ External data!•  UCSC table

browser!•  Biomart!•  interMine /

modMine!

•  EuPathDB !•  EncodeDB!•  EpiGRAPH!•  FlyMine!•  GrameneMart…!

Page 17: UF Research Computing - Information Technology · PDF fileIntroduction to Galaxy at! UF HPC!! Oleksandr Moskalenko! Assoc. Sci., UF HPC Center! Biological Applications Support! Matt

Data libraries

Page 18: UF Research Computing - Information Technology · PDF fileIntroduction to Galaxy at! UF HPC!! Oleksandr Moskalenko! Assoc. Sci., UF HPC Center! Biological Applications Support! Matt

Data Access Control

Page 19: UF Research Computing - Information Technology · PDF fileIntroduction to Galaxy at! UF HPC!! Oleksandr Moskalenko! Assoc. Sci., UF HPC Center! Biological Applications Support! Matt

Galaxy Tool Suites ✦  Text Manipulation!✦  Format Converters!✦  Filtering and Sorting!✦  Join, Subtract, Group !✦  Sequence Tools!✦  Multi-species

Alignment Tools!✦  Genomic Interval

Operation!

✦  Summary Statistics, graphing!

✦  Regional Variation!✦  EMBOSS!✦  Evolution/Phylogeny!✦  RNA-Seq!✦  ChIP-Seq!✦  GATK !

Page 20: UF Research Computing - Information Technology · PDF fileIntroduction to Galaxy at! UF HPC!! Oleksandr Moskalenko! Assoc. Sci., UF HPC Center! Biological Applications Support! Matt

A galaxy of tools

Page 21: UF Research Computing - Information Technology · PDF fileIntroduction to Galaxy at! UF HPC!! Oleksandr Moskalenko! Assoc. Sci., UF HPC Center! Biological Applications Support! Matt

Galaxy Workflows

Page 22: UF Research Computing - Information Technology · PDF fileIntroduction to Galaxy at! UF HPC!! Oleksandr Moskalenko! Assoc. Sci., UF HPC Center! Biological Applications Support! Matt

Galaxy Workflows

Page 23: UF Research Computing - Information Technology · PDF fileIntroduction to Galaxy at! UF HPC!! Oleksandr Moskalenko! Assoc. Sci., UF HPC Center! Biological Applications Support! Matt

Galaxy Workflows

Page 24: UF Research Computing - Information Technology · PDF fileIntroduction to Galaxy at! UF HPC!! Oleksandr Moskalenko! Assoc. Sci., UF HPC Center! Biological Applications Support! Matt

Visualization

Page 25: UF Research Computing - Information Technology · PDF fileIntroduction to Galaxy at! UF HPC!! Oleksandr Moskalenko! Assoc. Sci., UF HPC Center! Biological Applications Support! Matt

Sharing and publishing

Page 26: UF Research Computing - Information Technology · PDF fileIntroduction to Galaxy at! UF HPC!! Oleksandr Moskalenko! Assoc. Sci., UF HPC Center! Biological Applications Support! Matt

Sharing and publishing

Page 27: UF Research Computing - Information Technology · PDF fileIntroduction to Galaxy at! UF HPC!! Oleksandr Moskalenko! Assoc. Sci., UF HPC Center! Biological Applications Support! Matt

Sharing and publishing

Page 28: UF Research Computing - Information Technology · PDF fileIntroduction to Galaxy at! UF HPC!! Oleksandr Moskalenko! Assoc. Sci., UF HPC Center! Biological Applications Support! Matt

Galaxy pages

Page 29: UF Research Computing - Information Technology · PDF fileIntroduction to Galaxy at! UF HPC!! Oleksandr Moskalenko! Assoc. Sci., UF HPC Center! Biological Applications Support! Matt

Summary ✦ Analyze data without the CLI !✦ Visualize the results!✦ Publish histories, workflows, and

annotated pages!✦ Add new tools, get support @ HPC!✦ Focus on your science, not minutiae!✦ UF Galaxy – coming to a browser

near you!!

Page 30: UF Research Computing - Information Technology · PDF fileIntroduction to Galaxy at! UF HPC!! Oleksandr Moskalenko! Assoc. Sci., UF HPC Center! Biological Applications Support! Matt

Demo

Page 31: UF Research Computing - Information Technology · PDF fileIntroduction to Galaxy at! UF HPC!! Oleksandr Moskalenko! Assoc. Sci., UF HPC Center! Biological Applications Support! Matt

MACS demo

http://galaxy.hpc.ufl.edu!

Page 32: UF Research Computing - Information Technology · PDF fileIntroduction to Galaxy at! UF HPC!! Oleksandr Moskalenko! Assoc. Sci., UF HPC Center! Biological Applications Support! Matt

MACS demo

http://galaxy.hpc.ufl.edu!

Page 33: UF Research Computing - Information Technology · PDF fileIntroduction to Galaxy at! UF HPC!! Oleksandr Moskalenko! Assoc. Sci., UF HPC Center! Biological Applications Support! Matt

History/Shared Data

http://galaxy.hpc.ufl.edu!

Page 34: UF Research Computing - Information Technology · PDF fileIntroduction to Galaxy at! UF HPC!! Oleksandr Moskalenko! Assoc. Sci., UF HPC Center! Biological Applications Support! Matt

Shared Data

Page 35: UF Research Computing - Information Technology · PDF fileIntroduction to Galaxy at! UF HPC!! Oleksandr Moskalenko! Assoc. Sci., UF HPC Center! Biological Applications Support! Matt

MACS – Load data

Page 36: UF Research Computing - Information Technology · PDF fileIntroduction to Galaxy at! UF HPC!! Oleksandr Moskalenko! Assoc. Sci., UF HPC Center! Biological Applications Support! Matt

What’s inside

Page 37: UF Research Computing - Information Technology · PDF fileIntroduction to Galaxy at! UF HPC!! Oleksandr Moskalenko! Assoc. Sci., UF HPC Center! Biological Applications Support! Matt

MACS (NGS: Peak Calling)

Page 38: UF Research Computing - Information Technology · PDF fileIntroduction to Galaxy at! UF HPC!! Oleksandr Moskalenko! Assoc. Sci., UF HPC Center! Biological Applications Support! Matt

Submission form

Page 39: UF Research Computing - Information Technology · PDF fileIntroduction to Galaxy at! UF HPC!! Oleksandr Moskalenko! Assoc. Sci., UF HPC Center! Biological Applications Support! Matt

MACS options

•  Basic:!•  Treatment file: Your alignment file – choose BAM file!

•  Effective genome size: Human (hg19) – must set once!

•  Advanced:!•  Use model or shift size!

•  Model - fold enrichment (small and large): 10:30!

•  Bandwidth – scan bandwidth size for model or ½ window size without the model: default is 300!

Page 40: UF Research Computing - Information Technology · PDF fileIntroduction to Galaxy at! UF HPC!! Oleksandr Moskalenko! Assoc. Sci., UF HPC Center! Biological Applications Support! Matt

Submit the job to cluster

Page 41: UF Research Computing - Information Technology · PDF fileIntroduction to Galaxy at! UF HPC!! Oleksandr Moskalenko! Assoc. Sci., UF HPC Center! Biological Applications Support! Matt

Cluster job run

Page 42: UF Research Computing - Information Technology · PDF fileIntroduction to Galaxy at! UF HPC!! Oleksandr Moskalenko! Assoc. Sci., UF HPC Center! Biological Applications Support! Matt

Job completion

Page 43: UF Research Computing - Information Technology · PDF fileIntroduction to Galaxy at! UF HPC!! Oleksandr Moskalenko! Assoc. Sci., UF HPC Center! Biological Applications Support! Matt

Build a genome browser track

Page 44: UF Research Computing - Information Technology · PDF fileIntroduction to Galaxy at! UF HPC!! Oleksandr Moskalenko! Assoc. Sci., UF HPC Center! Biological Applications Support! Matt

Submit a track build job

Page 45: UF Research Computing - Information Technology · PDF fileIntroduction to Galaxy at! UF HPC!! Oleksandr Moskalenko! Assoc. Sci., UF HPC Center! Biological Applications Support! Matt

Open the track

Page 46: UF Research Computing - Information Technology · PDF fileIntroduction to Galaxy at! UF HPC!! Oleksandr Moskalenko! Assoc. Sci., UF HPC Center! Biological Applications Support! Matt

Genome Browser

Page 47: UF Research Computing - Information Technology · PDF fileIntroduction to Galaxy at! UF HPC!! Oleksandr Moskalenko! Assoc. Sci., UF HPC Center! Biological Applications Support! Matt

Add a custom track

Page 48: UF Research Computing - Information Technology · PDF fileIntroduction to Galaxy at! UF HPC!! Oleksandr Moskalenko! Assoc. Sci., UF HPC Center! Biological Applications Support! Matt

Paste track data

Page 49: UF Research Computing - Information Technology · PDF fileIntroduction to Galaxy at! UF HPC!! Oleksandr Moskalenko! Assoc. Sci., UF HPC Center! Biological Applications Support! Matt

View track

Page 50: UF Research Computing - Information Technology · PDF fileIntroduction to Galaxy at! UF HPC!! Oleksandr Moskalenko! Assoc. Sci., UF HPC Center! Biological Applications Support! Matt

Zoom in, pan, etc.

Page 51: UF Research Computing - Information Technology · PDF fileIntroduction to Galaxy at! UF HPC!! Oleksandr Moskalenko! Assoc. Sci., UF HPC Center! Biological Applications Support! Matt

•  http://wiki.hpc.ufl.edu !•  https://fisher.bioinformatics.ufl.edu!•  http://hpc.ufl.edu/support!

-  Frequently Asked Questions!

-  Account set up and maintenance!

-  Problem report submission!

-  [email protected] - Biological applications support!

-  [email protected] - Training!

Thank you!