2011-2012Stakeholder Recap

2

Message from the Executive 2Mission and Vision 7Year in Review 8Usage and Storage Graphs 12Researcher Profiles 16Staff and Committees 23Financials 25Partners 26Contact Us 27

Contents

3

A Year in Review:Building On Strengths And Supporting New Research Projects

With winter at our doorstep and a new year on the horizon, we felt it was an appropriate time to reflect on some of the milestones from 2012 and to acknowledge the individuals, teams, and research community involved in contributing to the success of the organization.

We have noted in past Annual Reports how High Performance Computing (HPC) has become more prevalent within various public and private sectors. We’ve seen this driven in part by the fact that data is now constantly being collected by everything from orbiting satellites in space to environmental monitoring devices in the field to smartphones in the palms of our hand.

Within the academic world, the possibilities for data collection across nearly every spectrum of research has uncovered new needs, increased demands, and exciting opportunities for HPC in a range of traditional and non-traditional discipline areas.

At WestGrid, we see evidence of this growth through each increase in the number of proposals submitted to Compute Canada - Calcul Canada’s annual Resource Allocation call. Looking towards 2013, we will continue to engage new users across all disciplines and work with Compute Canada and the other regional HPC divisions to ensure the national HPC platform meets the evolving needs of our user community.

Related to that, some of the final purchases from WestGrid’s portion of the Canada Foundation for Innovation’s National Platforms Fund program were completed last year and celebrated with HPC launch events in British Columbia and Alberta. These systems will support a range of big data projects, in fields ranging from genomics to particle physics to the social sciences, and will act as a valuable resource for Canadian research participation in international research endeavors, such as the ATLAS project.

Message from the Executive

4

As always, the members of our support teams play a critical role, as their expertise and assistance is critical to ensuring researchers are using the systems efficiently and effectively. This kind of support has helped a number of users make great strides in their research projects. Among those are Hirohisa Tanaka and the Tokai-to-Kamioka (T2K) Canada team (see page 18), who are part of a global experiment involving nearly 500 scientists around the world; Geoffrey Rockwell (see page 21), who is using HPC resources to change the way digital humanities researchers manipulate, analyze, and visualize electronic text; and Brian Menounos (see page 22), whose studies into the sensitive relationships between glaciers and climate change will help us predict future impacts of climatic changes. More examples of how WestGrid is supporting Canadian research can be found on pages 16 - 22 of this report.

The services and resources we provide for researchers wouldn’t be possible without the support of key partners in our community. These include our 14 partner institutions; the four provincial Optical Regional Advanced Network (ORAN) partners: MRnet (Manitoba), SRnet (Saskatchewan), Cybera (Alberta) and BCNET (British Columbia); and CANARIE, Canada’s Research and Innovation Network. Our strong working relationship with these partners is what enables us to support groups like the Canadian Advanced Network for Astronomical Research (CANFAR) project, which needed to transfer nearly 300 terabytes of astronomical research data across three provinces. Research data was copied from the National Research Council’s Herzberg Institute of Astrophysics’ (NRC-HIA) Canadian Astronomy Data Centre (CADC) in Victoria, BC, and transferred to WestGrid / Compute Canada storage facilities at the University of Saskatchewan in Saskatoon, SK to reduce the risk of a loss of data due to a possible earthquake, or other disaster, in British Columbia.

In May 2012, WestGrid collaborated with BCNET and Compute Canada to host HPCS 2012. More than 500 delegates from across Canada attended the three-day event in Vancouver, BC, which featured pre-conference workshops, peer-reviewed paper presentations, and a keynote by John Towns, Principal Investigator and Project Director of XSEDE. This event was an excellent showcase of the innovative and exciting research being supported by HPC in Canada. We look forward to attending HPCS 2013, which will take place on June 2 - 6, 2013 in Ottawa, ON.

That brings us to our users. Currently, WestGrid systems support more than 1,000 registered users and 475 project groups. The researcher profiles within this report give but a glimpse into our ever-growing diverse and dynamic user community. To each and every one of those users we say: “Keep asking questions. Keep challenging what’s out there. Keep uncovering new problems and keep pushing our resource capabilities.” Your work is what drives WestGrid, and in turn, we are here to support you and to help you succeed.

Lastly, we would like to thank the Canada Foundation for Innovation, WestGrid’s vendor partners - HP, IBM, SGI, Dell, and Appro - and our provincial government funders for their ongoing support. If you have any questions about any of WestGrid’s 2012 activities or the current infrastructure framework, please contact us at info@westgrid.ca.

WestGrid Executive CommitteeNikitas Dimopoulos, University of VictoriaMark Thachuk, University of British ColumbiaDugan O’Neil, Simon Fraser UniversityPeter Tieleman (chair), University of CalgaryPaul Lu, University of AlbertaRay Spiteri, University of SaskatchewanByron Southern, University of Manitoba

5

Mark ThachukUniversity of British Columbia

Mark Thachuk is an Associate Professor in the Department of Chemistry at the University of British Columbia. He received his PhD from the University of Waterloo. Thachuk’s research in chemistry has led to many publications, resulting in fundamental contributions to the understanding of the theory and computer simulation of the dynamics of gas phase ions.

Peter Tieleman (chair)University of Calgary

Peter Tieleman studied physical chemistry at the University of Groningen in the Netherlands, where he obtained his PhD under the supervision of Herman Berendsen, one of the pioneers of biomolecular simulation. After a year as a European Molecular Biology Organization (EMBO) fellow at the University of Oxford in Mark Sansom’s research group, Tieleman joined the University of Calgary. Since 2005, he has been a Professor in the Department of Biological Sciences. He is currently an Alberta Innovates Health Solutions Scientist and Alberta Innovates Technology Futures Strategic Chair in (Bio)Molecular Simulation, working in the areas of biomolecular simulation and computational biology. Among his distinctions are an Alfred P. Sloan Foundation Fellowship, the Royal Society of Canada’s Rutherford Memorial Medal in Chemistry, and a Natural Sciences and Engineering Research Council of Canada (NSERC) Steacie Memorial Fellowship.

Nikitas DimopoulosUniversity of Victoria

Nikitas Dimopoulos is a Professor in the Department of Electrical and Computer Engineering at the University of Victoria. His research is in the general area of computer engineering. Dimopoulos is specifically interested in computer architecture, parallel computer systems, neural networks, power aware systems, and the grid. He joined the University of Victoria in 1988 after spending a year at the Jet Propulsion Laboratory in Pasadena, USA, and served as Chair of the Department of Electrical and Computer Engineering between 1998 and 2003 and again between 2005 and 2008. Prior to this, he held academic appointments with Concordia University in Montreal. Dimopoulos is a registered Professional Engineer in British Columbia and Fellow of the Engineering Institute of Canada. He holds the Lansdowne Chair in Computer Engineering.

Dugan O’NeilSimon Fraser University

Dugan O’Neil is an Associate Professor in the Department of Physics at Simon Fraser University. His research is in particle physics at the energy frontier. He has an MSc from the University of Alberta and a PhD from the University of Victoria. In 2003, O’Neil founded a Canadian consortium on the D0 Experiment, a worldwide collaboration of scientists conducting research on the fundamental nature of matter. His group used WestGrid resources at the University of British Columbia and Simon Fraser University to process hundreds of terabytes of raw data from the experiment. The output was utilized to find the first evidence of single top quark production in 2006, and to observe the process in 2009. His research now focuses on finding new physics using the ATLAS experiment at CERN. He currently serves as Deputy Spokesperson for the ATLAS-Canada collaboration.

6

Byron SouthernUniversity of Manitoba

Byron Southern has been in the Department of Physics and Astronomy at the University of Manitoba since 1979 and has served on the WestGrid Executive since 2006 when WestGrid expanded to include all universities in Western Canada. He has served as Chair of the Executive and as a member of the National initiatives Committee for Compute Canada. His research interests are in the physics of frustrated magnetic materials and he uses HPC to simulate the magnetic properties of these systems.

Ray SpiteriUniversity of Saskatchewan

Ray Spiteri is a Professor in the Department of Computer Science at the University of Saskatchewan. He has served on the WestGrid Executive since 2006. Spiteri is a former MITACS/Mprime Project Leader (2004-2012), during which time he served as the MITACS Regional Scientific Director for the Prairie Provinces for a four-year term. His areas of research are numerical analysis, scientific computing, and high-performance computing. Spiteri lives the dream of solving differential equations for a living. His specialty is designing efficient methods for the time integration of ordinary and partial differential equations. He also has a long record of industry collaboration with companies such as IBM and Boeing. His current applications include simulation of electrical activity in the heart, fluidized-bed gasifiers, and fuel cells.

Paul LuUniversity of Alberta

Paul Lu is an Associate Professor in the Department of Computing Science at the University of Alberta. His research is in the area of high-performance computing (HPC), including algorithms, bioinformatics, virtual machines and cloud computing. In 2004, Paul’s research group created a pan-Canadian metacomputer across 19 universities and 22 administrative domains, known as the Canadian Internetworked Scientific Supercomputer. He also co-coached the University of Alberta team that won the first annual Cluster Computing Challenge at Supercomputing 2007. In 2010, his Ph.D. student contributed the ivshmem/Nahanni virtual device to the Linux KVM code base.

7

Compute Canada - Calcul Canada is Canada’s national platform of supercomputing resources, bringing together computer and data facilities, computational expertise, and hundreds of academic researchers to tackle some of Canada’s biggest research challenges.

Compute Canada - Calcul Canada serves a user community across Canada in disciplines ranging from the sciences and engineering to arts and humanities. The partner institutions and resource centres that comprise the national platform are hubs of interdisciplinary computational research. Together, these distributed computing facilities work collaboratively to provide the expertise and resources necessary to give Canada’s researchers and innovators access to world-class technologies.

As one of Compute Canada - Calcul Canada’s largest regional divisions, WestGrid encompasses 14 partner institutions across British Columbia, Alberta, Saskatchewan and Manitoba. Its user community is supported by a distributed and cohesive team of technical staff and system architects.

MISSION: To create a world-class sustained national Canadian platform of shared HPC resources and foster a community of knowledgeable personnel that is accessible by researchers in all disciplines, independent of resource or researcher location. WestGrid aims to promote HPC nationally and internationally within the Compute Canada - Calcul Canada framework.

VISION: To advance research, support and accelerate innovation and excellence, develop highly qualified personnel, and ensure a competitive advantage and economic prosperity for the well-being of Canadians, through the effective use of HPC.

UNIVERSITY OF VICTORIA

UNIVERSITY OF BRITISH COLUMBIA

UNIVERSITY OF NORTHERN BRITISH COLUMBIA

SIMON FRASER UNIVERSITY

ATHABASCAUNIVERSITY

BRANDONUNIVERSITY

UNIVERSITY OF CALGARY

UNIVERSITY OF ALBERTA

UNIVERSITY OF LETHBRIDGE

UNIVERSITY OF SASKATCHEWAN UNIVERSITY OF

MANITOBA

UNIVERSITY OF REGINA THE UNIVERSITY

OF WINNIPEG

THE BANFF CENTRE

8

Year in Review

As part of the Compute Canada - Calcul Canada national platform, WestGrid operates a grid-enabled distributed framework of HPC clusters, data storage sites, visualization resources and collaboration facilities located throughout Western Canada. The high-speed network connecting these resources is made possible through a collaborative working relationship with CANARIE and the provincial Optical Regional Advanced Network (ORAN) partners: BCNET, Cybera, SRnet, and MRnet. The items listed below include some of WestGrid’s infrastructure-related milestones from 2012, such as system launch events, resource allocation announcements, and innovative applications of the national platform.

June 16, 2011Compute Canada - Calcul Canada and Super Micro Computer announce a partnership to build an HPC platform for a new initiative, HPC for the Humanities, to enable researchers across Canada to share data resources and tools through a distributed HPC network. The initiative will enable humanities researchers to take advantage of the enormous potential of HPC to deal with large and complex sets of unstructured data in the form of books, election and financial data, archaeological information and newspapers.

June 30, 2011Compute Canada - Calcul Canada and WestGrid launch new HPC resources at both the University of British Columbia (UBC) and Simon Fraser University (SFU). Nearly $17 million was invested to expand and upgrade the UBC and SFU facilities. UBC introduced Orcinus, which provides 9,600 cores, 19.2 TB of memory and 0.5 PB of storage capacity. SFU introduced Bugaboo, which provides 414 compute nodes containing over 4,300 cores, nearly 9 TB of memory and 2.4 PB of storage capacity.Bugaboo is housed at Simon Fraser University.

9

August 11, 2011WestGrid demonstrates a newly purchased 3D, virtual data system to a private delegation visiting the University of Calgary. Called an IQ-Station, which stands for “Inexpensive Interactive Immersive Interface”, it is a virtual reality tool that allows researchers to interact with complex datasets in a three-dimensional way.

December 13, 2011Nearly 300 terabytes of astronomical research data is copied from the National Research Council’s Herzberg Institute of Astrophysics’ (NRC-HIA) Canadian Astronomy Data Centre (CADC) in Victoria, BC, and transferred to Compute Canada - Calcul Canada / WestGrid storage facilities at the University of Saskatchewan in Saskatoon, SK, as part of the Canadian Advanced Network for Astronomical Research (CANFAR) project.

February 27, 2012 As part of its annual resource allocation process, Compute Canada - Calcul Canada allocates nearly $80 million worth of state-of-the-art computing, storage, and support resources to 159 leading edge Canadian research projects across the country.

April 3, 2012Following nearly three years and $17 million in investments, Compute Canada - Calcul Canada and WestGrid co-host a celebratory launch of Alberta-based HPC and data storage resources at the University of Calgary (UofC) and the University of Alberta (UofA). The event also showcases upgrades to the research visualization and collaboration facilities at the UofA, UofC, Athabasca University, University of Lethbridge, and The Banff Centre.

The IQ station was also on display at the HPCS 2012 conference, as demonstrated above by Brian Corrie, WestGrid’s Collaboration and Visualization Coordinator.

The computing facilities at the University of Calgary were among the HPC resources showcased at the launch event.

10

Throughout the year, WestGrid supports a number of training workshops, videoconferenced Seminar Series sessions, and other events that showcase and connect its user community. Below are a few highlights of outreach activities from 2012.

April 20, 2011WestGrid joins the HPC social media scene with a new Twitter account: @WestGrid. Followers of @WestGrid can stay up to date on news and events happening within WestGrid and/or within the broader HPC community.

May 24, 2011 Keith Archer, a member of the 2011 WestGrid Executive Committee, is appointed as British Columbia’s Chief Electoral Officer. Effective September 1, 2011, his term of office is two provincial general elections plus one year.

June 15, 2011WestGrid’s 2010-2011 Annual Report is published and distributed throughout the HPC community. The report features highlights from the past year, usage statistics, and profiles of some of the users on WestGrid’s systems.

August 15-19, 2011The University of Alberta, a WestGrid partner institution, hosts a Visualization Tools and Techniques Workshop. The series of courses focused on the tools and techniques of scientific visualization and featured hands-on examples to reinforce the discussions of theory. WestGrid operates visualization resources at each of its resource sites and has visualization experts on staff to help WestGrid users interactively explore complex data sets.

September 20, 2011 The Coast to Coast Seminar Series launches its Fall 2011 theme, “Modelling of Complex Systems”, to examine the modelling of complex systems in fields such as climate change, criminology research, urban dynamics, disease dynamics, and health services delivery. The Coast to Coast Seminars are hour-long videoconferenced presentations held every two weeks discussing a range of scientific topics. For the Fall 2011 series, six sessions were held between September 20 to November 20.

October 12, 2011 WestGrid hosted its first session of its 2011 Fall Seminar Series, with Director of Operations Patrick O’Leary presenting an Introduction to WestGrid. The sessions are videoconferenced across participating WestGrid sites and cover a range of topics including how to use WestGrid, programming serial and parallel applications, using scientific data visualization, and how WestGrid and Compute Canada - Calcul Canada resources are being used to advance research investigations. For the Fall 2011 series, five sessions were held between October 12 and November 30.

WestGrid’s Twitter account currently has 165 followers.

Digital copies of the Annual Report can be found on WestGrid’s website.

WestGrid seminars are targeted at new users as well as researchers looking to take their current HPC use further.

11

October 25 - 26, 2011A delegation of WestGrid support staff participate in TECC Summit 2011, an annual meeting of the technical experts working within Compute Canada - Calcul Canada. Held at Saint Mary’s University in Halifax, NS, the three-day Summit featured presentations and working groups on a variety of topics, including security, networking, cloud, and the Compute Canada - Calcul Canada CCDB.

February 7, 2012The Coast to Coast Seminar Series launches its Spring 2012 theme, “Complex Networks in Science and Technology”, featuring talks on topics ranging from data communication networks, transportation networks, biological and gene regulation networks, and social networks. The Coast to Coast Seminars are hour-long videoconferenced presentations held every two weeks discussing a range of scientific topics. For the Spring 2012 series, five sessions were held between February 7 and April 3.

March 16, 2012 Jill Kowalchuk is appointed Interim Executive Director of Compute Canada - Calcul Canada. Kowalchuk is seconded from her role as Executive Director of WestGrid and Vice-President, Project and Partnership Development at Cybera. Lindsay Sill is appointed Interim Executive Director of WestGrid, effective March 26, 2012.

May 1-3, 2012Compute Canada - Calcul Canada, WestGrid, and BCNET host the High Performance Computing Symposium (HPCS) 2012 alongside BCNET’s annual conference. Themed Connect. Compute. Collaborate., the conference attracted more than 500 delegates from across North America to Vancouver, BC for three days of plenary sessions, panel discussions, peer-reviewed paper presentations, and networking opportunities. HPCS 2013 will be held in Ottawa, ON on June 2 - 6, 2013.

May 17, 2012 Jonathan Schaeffer, one of WestGrid’s original Principal Investigators and an active WestGrid user, is appointed as Dean of the Faculty of Science at the University of Alberta. Schaeffer was a member of WestGrid’s Executive Committee from its inception in 2002 until 2007.

WestGrid partner institutions share the Coast to Coast seminars with local audiences using on-campus videoconference facilities.

HPCS is a multidisciplinary conference showcasing new tools, techniques and interesting results in and for HPC computational research.

Jill Kowalchuk has worked with Canada’s HPC community for more than a decade.

Jonathan Schaeffer’s research area is artificial intelligence (AI). He is the author of checkers-playing program Chinook, the first computer program to win a human world championship.

12

Usage and Storage Stats

Research today demands computing resources that are fast, powerful, and grid-enabled to accommodate data-intensive calculations and facilitate collaboration across national and international borders. Day and night, WestGrid resources are processing, visualizing and storing data in support of research from well over 50 university departments, involving hundreds of users and thousands of computational jobs.

WestGrid Support Staff play a critical role ensuring its HPC resources are being used effectively and to the best advantage of the user. Support staff are available to help users get started on the systems, can lend advice on ways to leverage HPC resources within a research program, and will ensure the technical aspects of users’ projects are matched with appropriate systems.

The graphs in this section were created to share a high-level glimpse of how WestGrid’s advanced computing resources are shared across research disciplines and geographic locations. If you have any questions about the data presented in these charts, please contact support@westgrid.ca.

13

Chemistry  and  Biochemistry  37%  

Physics  and  Astronomy  29%  

Biological  and  Life  Sciences  16%  

Computer  and  InformaBon  Sciences  

7%  

Environmental  and  Earth  Sciences  

4%  

Engineering  4%  

MathemaBcs  and  StaBsBcs  1%  

Other  2%  

Compu&ng  Usage  by  Discipline  

Bri$sh  Columbia  34%  

Alberta  43%  

Saskatchewan  5%  

Manitoba  3%  

Ontario  8%  

Quebec  3%  

Nova  Sco$a  4%  

Compu&ng  Usage  by  Province  

14

Physics  and  Astronomy  77%  

Chemistry  and  Biochemistry  2%  

Engineering  4%  

Environmental  and  Earth  Sciences  

9%  

Biological  and  Life  Sciences  6%  

Computer  and  InformaCon  Sciences  

2%  

Storage  by  Discipline  

Alberta  12%  

Bri.sh  Columbia  85%  

Saskatchewan  2%  

Ontario  1%  

Storage  by  Province  

15

Biological  and  Life  Sciences  5%  

Chemistry  and  Biochemistry  64%  

Computer  and  Informa=on  Sciences  

2%  

Engineering  1%  

Environmental  and  Earth  Sciences  

3%  

Medical  Science  1%  

Physics  and  Astronomy  24%  

Compu&ng  Alloca&on  

Biological  and  Life  Sciences  3%  

Chemistry  and  Biochemistry  9%  

Computer  and  Informa<on  Sciences  

1%  

Engineering  1%  

Environmental  and  Earth  Sciences  

1%  

Physics  and  Astronomy  85%  

Storage  Alloca,on  

16

Researcher Profiles

WestGrid’s users represent a diverse, innovative, and leading-edge community of researchers who are using HPC resources, visualization tools, data storage capabilities and collaboration technologies to advance Canada’s R&D sector. The following brief research profiles offer a glimpse at some of the projects and people being supported by WestGrid and Compute Canada - Calcul Canada infrastructure and staff. WestGrid is proud to be affiliated with these leading minds breaking new ground in computational research.

Mathieu Dumberry University of AlbertaDepartment of Physics

Most of Mathieu Dumberry’s research is focused on furthering our understanding of the Earth’s interior. When he’s not studying fluid motion of our planet’s core, however, his interests expand to another environment – one just as dynamic but much further away.

“The origin of Jupiter and Saturn banded flow structures, with a large prograde equatorial jet and smaller jets of alternating direction at higher latitudes, are one of the most spectacular displays of fluid mechanics. However, they have yet to be fully explained,” says Dumberry, an Assistant Professor at the University of Alberta.

This is where WestGrid comes in. One of Dumberry’s PhD students, Daniel Laycock, is using WestGrid and Compute Canada - Calucl Canada HPC systems to run numerical simulations of zonal wind generation on Jupiter and Saturn. His high resolution models are simulating convection for a much more aggressive parameter regime than possible with traditional three dimensional (3D) simulations. Preliminary results suggest that the quasi-geostrophic model Dumberry’s team is using can reproduce the banded zonal flows of Jupiter and Saturn at a fraction of the computational cost of a 3D simulation. This will eventually allow for an investigation of regions of parameter space currently inaccessible to full 3D simulations.

“Access to WestGrid and Compute Canada - Calcul Canada computing resources is essential for this type of research because the intense computing power required for our numerical simulations is simply not possible on local cluster resources. Moreover, not having to maintain our own local cluster means that we do not have to deal with technical hardware issues, not to mention the savings in research costs. This allows us to spend more time on theoretical developments to further improve our model,” says Dumberry.

The image above shows the east-west velocity resulting from one of Daniel Laycock’s model runs using WestGrid’s HPC systems. Red indicates an eastward flow and blue is a westward flow.

17

Susan Brown University of Guelph Department of English

Canada’s literary heritage is at a critical juncture – materials are moving online, yet the management of information about Canadian cultural history still relies on tools derived from print models, which cannot accommodate the explosion of online materials. Literary studies must shift from the conventional model of solitary scholars working on small groups of texts, towards large-scale cross-disciplinary collaborative energies.

The Canadian Writing Research Collaboratory (CWRC) is funded by the Canada Foundation for Innovation to provide an open web-based environment to foster the use of digital tools and resources for literary studies in and about Canada. WestGrid and Compute Canada - Calcul Canada facilities provide an essential infrastructure backbone for this distributed project.

“The infrastructure will both facilitate digital literary research and provide a testbed for computational research ranging from text analysis and visualization to the design and use of social networking tools,” says Susan Brown, CWRC Team Leader. “WestGrid and Compute Canada - Calcul Canada are providing essential support in hardware acquisition, hosting, and backup capacity.”

The Collaboratory is being developed around active research projects and comprises two major elements, a database and a toolkit, linked through a web-based service-oriented architecture.

“CWRC’s specialized interface connects scattered and siloed data, enabling researchers to investigate links between writers, texts, places, groups, policies, and events in a collaborative online environment,” says Brown.

CWRC extends the work of the Orlando Project (http://www.ualberta.ca/orlando), a humanities computing research project that continues to publish new findings and serves as an experimental dataset. Contributing and pilot projects will contribute further data while CWRC’s technical infrastructure is in development and will provide feedback on interface and functionality.

The image above was created using a visualization tool developed using Compute Canada - Calcul Canada HPC systems.

Scott Ormiston University of Manitoba Department of Mechanical Engineering

Better designs of heat exchangers in the power system, refrigeration, and chemical processing industries are on the horizon, thanks to new modelling experiments developed by University of Manitoba Professor Scott Ormiston. Ormiston’s research is focused on developing new solution methods for modelling film condensation from gas-vapour mixtures for heat exchanger applications. These methods will have an interface tracking algorithm that accurately predicts the location of the liquid film, thereby accurately predicting the local heat and mass transfer. Current methods cannot achieve this level of accuracy because they rely on simplified governing equations or empirical correlations that are not sensitive to some design parameters.

“WestGrid and Compute Canada - Calcul Canada’s computing resources are critical to this research because of the significant effort needed to solve the heat, mass, and momentum conservation equations for many time steps for large numbers of nodes while re-computing the liquid-gas interface and computational mesh,” says Ormiston.

The results of this research have the potential to improve future designs of heat exchangers used in many basic energy conversion systems. Improved designs would increase productivity and reduce overall energy consumption.

“This research would not be possible without WestGrid and Compute Canada - Calcul Canada’s infrastructure,” says Ormiston. “If we didn’t have access to these powerful shared resources, we estimate one simulation using the full, advanced model for a heat exchanger application would take at least six weeks on a local server, rather than days using the national platform.”

18

This photograph was taken from inside of the Super-Kamiokande neutrino detector, located in eastern Japan. This image was provided by the Kamioka Observatory, Institute for Cosmic Ray Research (ICRR), The University of Tokyo.

Hirohisa Tanaka University of British Columbia Department of Physics & Astronomy

The July 2012 discovery of a Higgs boson-like particle put anti-matter on the front page of newspapers around the world and brought scientists closer to understanding and explaining the subatomic building blocks of the universe. Another international project, powered by WestGrid and Compute Canada - Calcul Canada computing systems, has similar potential for shedding light on the origin of the universe’s matter/anti-matter asymmetry.

The Tokai-to-Kamioka (T2K) experiment is a collaboration of nearly 500 scientists, including approximately 40 scientists and graduate students at seven Canadian universities (Alberta, British Columbia, Regina, Toronto, Victoria, Winnipeg, York) and TRIUMF. In this experiment, a neutrino beam produced by a proton accelerator complex in eastern Japan is directed towards Super-Kamiokande, a large neutrino detector 295 km away.

“The neutrino is a fundamental particle that plays a fundamental role in how stars like the sun produce energy and the evolution of the universe,” says Hirohisa Tanaka, Associate Professor in the Department of Physics and Astronomy at the University of British Columbia, and one of the project’s Canadian team members. “T2K-Canada has played a leading role in the experiment by contributing key components for producing and detecting neutrinos, and is now playing a critical role in the analysis of the data. Our recent studies found evidence for muon neutrinos converting to electron neutrinos, and are the first steps towards understanding how anti-matter may have disappeared from the universe.”

In 2010-2012, T2K-Canada and its Canadian HPC facilities provided nearly half of the computing resources required for data processing and simulation for the experiment.

“Thanks to WestGrid and Compute Canada-Calcul Canada, Canadian particle physicists participating in T2K are at the forefront of developing algorithms to more effectively study our neutrinos, leading to more sensitive and precise results with greater scientific impact. These exciting intellectual developments would be impossible without our HPC resources,” says Tanaka.

19

Falk Herwig University of Victoria Department of Physics and Astronomy

A University of Victoria research program investigating the nature of mixing processes of stars could play a critical role in supporting present and future observational missions. Led by Professor Falk Herwig, and in collaboration with Paul Woodward at the University of Minnesota, the studies focus on the mixing processes in very metal-poor stars that formed in the first and second burst of star formation in the very early universe.

“At that time, the overall metal content of the universe was extremely low, and stars forming out of such primordial material are known to host special types of hydrodynamic mixing processes that can be investigated through three-dimensional hydrodynamic simulations,” says Herwig. “We are using WestGrid and Compute Canada - Calcul Canada resources to specifically investigate the ingestion of H-rich material into He-shell flash convection regions in evolved primordial stars.”

This builds on earlier work of Herwig’s, which used WestGrid’s Orcinus cluster to investigate white dwarf He-shell flash stars and demonstrate excellent agreement between the simulation and observation-based predictions of the hydrodynamic mixing. The data-rich 3D simulations performed for this research are massively parallel, with typical HPC runs using between 2,000 and 4,000 compute cores for more than a week.

“Without the resources from WestGrid and Compute Canada - Calcul Canada, this type of research, which is internationally acknowledged to be of critical and increasing importance, would not happen in Canada,” said Herwig. “Our research demonstrates generating scientific knowledge through predictive science simulations, and supports the activities of critical observational missions, such as those at the Herzberg Institute of Astrophysics.”

The image above is a visualization of entrainment processes at a convection boundary in a stellar hydrodynamics simulations. Shown is the concentration of H-rich material that is mixed from a stable layer (outside the spherical circumference, not seen) into an underlying Carbon-12 rich convection layer inside a star. This run was performed on a 768^3 grid over 1.47 million time steps on the WestGrid cluster Orcinus (April 2012), running on 2056 cores for a total of 500,000 CPU hours (56 core years). This image was provided by Herwig (UVic) / Woodward (LCSE).

Stacey Wetmore University of LethbridgeDepartment of Chemistry and Biochemistry

It has been estimated that there are more than 20,000 damaged sites per typical human cell at any given time, which can lead to mutations and diseases such as cancer if left unrepaired.

Stacey Wetmore, a Professor with the Department of Chemistry and Biochemistry at the University of Lethbridge, hopes to change that. Wetmore’s research is focused on understanding deoxyribonucleic acid (DNA) damage and repair processes in our bodies in order to aid the design of new ways to diagnose, treat and prevent disease.

Wetmore’s group uses WestGrid and Compute Canada - Calcul Canada resources to power sophisticated computational chemistry routines to understand the formation mechanism of lesions and the natural mechanisms in our bodies to repair this damage.

“WestGrid and Compute Canada - Calcul Canada infrastructure is vital to this work since large-scale computer models of biosystems require significant computer resources in terms of the important factors of time, memory and disk space,” says Wetmore, a Canada Research Chair in Computational Chemistry. “The results of this research have the potential to impact Canadians by increasing our general knowledge of DNA damage and repair processes, which will allow the design of novel ways to diagnose and treat disease.”

The incorporation of calculations into traditional studies of DNA lesions (which entails making adducts, incorporating them into DNA and biological testing) will also greatly assist clinical research by providing complementary information and aiding the assessment of the biological reactivity of toxic DNA adducts.

The image above is an example of the models of DNA damage and repair Stacey Wetmore has created using WestGrid resources.

20

Sergei Noskov University of CalgaryDepartment of Biochemistry

Membrane proteins represent an essential component of life. They play an important role in everyday cell functions and act as transport channels for many small ions and organic molecules. For example, lipid membranes allow for compartmentalization of cells, but prevent uptake of nutrients and expulsion of toxins. Membrane proteins allow the cell to circumvent this problem, providing a selective and gated path for uptake and expulsion.

“Although the implications and applications of solute transport are many, our understanding of the underlying complex thermodynamics of ion and solute binding and transport across cellular membranes is very limited,” says Sergei Noskov, Associate Professor, Department of Biochemistry, University of Calgary.

Noskov’s research program aims to shed more light on this area. Using WestGrid and Compute Canada - Calcul Canada HPC facilities, Noskov and his research team are running complex computational simulations of the system of proteins embedded in membrane bilayers and solvent. One of his projects, which studied the ways proteins interact with various drugs, was profiled in WestGrid’s 2008 Annual Report.

Since then, Dr. Noskov’s lab contributed to novel methods for studies of targeted drug delivery across cellular membranes, worked on an improvement of the existing anti-arrhythmia agents and models for in-silico assessment of potentially lethal cardiotoxicity of organic molecules. Recently, he has been using computational modeling for design of nanopores with a potential utilization in DNA sequencing.

Noskov’s simulations require an immense amount of processing power, far beyond what is possible with a local computer cluster. Currently, Noskov is one of the top users of Graphics Processing Units (GPUs) on WestGrid and Compute Canada - Calcul Canada systems.

“GPU computing, with its efficient parallel performance, has enabled multiple simulations of systems for up to 500,000 atoms at atomistic resolution,” said Noskov.

Ultimately, the results of Noskov’s research program have the potential to contribute to the design of various pharmaceutical drugs and the development of a cheaper DNA sequence analysis using nano-pores.

The model pictured above is of coarse-grained DNA in the nanopore without the nanopore/environment.

Yuanming Pan University of Saskatchewan Department of Geological Sciences

Trace defects in minerals exert profound impacts on material properties such as colour, density, rheology, stability, electrical and thermal conductivity, magnetism, seismic velocity, and many more.

Electron paramagnetic resonance (EPR) spectroscopy has been and continues to be the technique of choice for quantitative studies of paramagnetic defects in minerals and other materials, because of its unparallelled sensitivity for detection and structural study of dilute defects. However, a combination of factors such as the complexity of natural materials like minerals, existence of magnetically similar defects, and lack of other complementary experimental techniques with similar sensitivities, makes the EPR data often difficult to interpret.

Yuanming Pan, a Professor of Geology at the University of Saskatchewan, is harnessing WestGrid and Compute Canada - Calcul Canada’s advanced computing resources to run first-principles theoretical models as an approach for studying the structures and properties of defects in minerals.

21

“This research would not be possible without WestGrid and Compute Canada - Calcul Canada infrastructure because our first-principles calculations using the supercell approach are extremely demanding in computing resources,” says Pan.

The calculations Pan is running on WestGrid’s systems are being used to predict the geometries, electronic structures, magnetic properties and stability of paramagnetic defects. An example of recent theoretical calculations is the new tri-vacancy model for the E’1 center in α-quartz (SiO2), which reproduces all 29Si hyperfine and superhyperfine coupling constants determined by EPR experiments and explains its common association with the [AlO4]0

center in this mineral (Li and Pan, 2012).

The results uncovered by Pan’s theoretical calculations stand to complement EPR experimental data and have a wide range of applications, including geochemical modeling, geochronology, remediation of heavy metalloid contamination, nuclear waste disposal, and mineral exploration.

Local structure (A) and electron density (B) of the E’1 center in quartz calculated from the tri-vacancy model with an Al atom at a neighboring Si site. Note the presence of a peroxy linkage (<O1-O2> = 1.48 Ǻ) and that the AlO4 group has an elongated Al-O1 bond of 1.91 Ǻ. Contours are at intervals of 0.005 e/bohr3 and from -0.01 to 0.415 e/bohr3.

Geoffrey Rockwell University of Alberta Department of Philosophy

An online text tool portal developed using WestGrid and Compute Canada - Calcul Canada resources is changing the way digital humanities researchers manipulate, analyze, and visualize electronic text.

Geoffrey Rockwell, Professor of Philosophy and Humanities Computing at the University of Alberta, is the project leader for TAPoR (Text Analysis Portal for Research), a research infrastructure project that supports digital humanities researchers working with electronic texts. Originally funded by the Canada Foundation for Innovation and now supported by SSHRC and the University of Alberta, the portal is designed so that textual scholars can discover and review analytical and visualization tools. Accompanying the portal are a set of reference text tools, TAPoRware, that are widely used.

“For TAPoR 2.0, we redesigned the portal to improve users’ engagement with the text analysis tools, focus on the discovery and social use of text analysis tools, and better communicate information about the tools,” says Rockwell.

TAPoR 2.0 is hosted by WestGrid’s “portal” service that allows research web tools to be run on a well-maintained system. The WestGrid service allows Rockwell to customize the account and run a stable web service of use to researchers internationally. Combined with the reference TAPoRware tools, this provides a useful entry into text analysis, mining and visualization. Now Rockwell and his colleagues are experimenting with the adaptation of more compute-intensive data-mining techniques to the study of literary, historical and philosophical texts.

“Support from WestGrid and Compute Canada - Calcul Canada has greatly expanded our horizons for what we can try. WestGrid staff have helped us with training and the infrastructure allows us to run an innovative portal for text analysis.”

Ultimately, Rockwell and his team hope TAPoR encourages humanities researchers to ask new questions and experiment with new interpretations of electronic texts.

One of the text tools available through TAPoR was used to create this word cloud from the WestGrid website’s RSS feed.

22

Brian Menounos University of Northern British Columbia Department of Geography

Glaciers are known to faithfully respond to changes in climate. They react to climate variability through adjustments in width, length, area, thickness, and volume, making them sensitive indicators of climatic change. For Brian Menounos, Associate Professor of Earth Sciences in the Geography Program at the University of Northern British Columbia, glaciers play a larger role than just indicators of climate change.

“In Canada, glaciers also represent a substantial source of renewable energy, contribute to the sustainability of ecosystems, and bolster the economy through tourism,” says Menounos.

In British Columbia and Alberta alone, glaciers cover 25,000 square-kilometres of the landmass and serve as frozen, freshwater reservoirs that supplement snowmelt runoff during summer and early autumn when seasonal snow cover is depleted. Electricity produced from surface runoff accounts for approximately 90% of BC’s and 17% of Alberta’s current power demand. In order to better understand the links between climate change and glaciers, Menounos and his research team are using WestGrid and Compute Canada - Calcul Canada resources to produce high resolution (1 km) climate fields for study sites in British Columbia and Patagonia. Using a state-of-the-art weather research forecasting model, these climate fields will be used to drive a glacier mass balance model and will also simulate the demise of the Cordilleran ice sheet and latest Pleistocene advances using a numerical ice sheet model. The results from Menounos’ research will contribute to documentation of past climate change and will also be vital in helping predict how alpine glaciers may change in the future.

“The sustainability of glacier runoff in the face of future climate change remains uncertain, but advanced computing resources such as those provided by WestGrid and Compute Canada - Calcul Canada is helping us to learn more about the intricate relationships between glaciers and climate and what impact glacier change will have on our natural environment,” said Menounos.

Pictured above is a MODIS image of southern British Columbia and Washington State (9 August, 2009), showing glaciers and snow-covered areas.

23

Staff and Committees

The following staff and committee lists are reflective of WestGrid’s operations during April 1, 2011 - May 31, 2012.

Member CouncilHoward Brunt University of Victoria Vice-President (Research)John Hepburn University of British Columbia Vice-President (Research)Mario Pinto Simon Fraser University Vice-President (Research)Gail Fondahl University of Northern British Columbia Vice-President (Research)Jeff Melanson The Banff Centre PresidentEd McCauley University of Calgary Vice-President (Research)Lorne Babiuk University of Alberta Vice-President (Research)Daniel Weeks University of Lethbridge Vice-President (Research) Donna Romyn Athabasca University Associate Vice President (Research) - (Acting) Karen Chad University of Saskatchewan Vice-President (Research)Dennis Fitzpatrick University of Regina Vice-President (Research)Digvir Jayas University of Manitoba Vice-President (Research)Neil Besner The University of Winnipeg Vice-President (Research)Scott Grills Brandon University Vice-President (Academic & Provost)

Executive CommitteeNikitas Dimopoulos University of VictoriaMark Thachuk University of British ColumbiaDugan O’Neil Simon Fraser UniversityPeter Tieleman (chair) University of CalgaryPaul Lu University of AlbertaRay Spiteri University of SaskatchewanByron Southern University of Manitoba

24

WestGrid Collaboration GroupBelaid Moa University of VictoriaIan Allison University of British ColumbiaBrian Corrie Simon Fraser UniversityJean Wang University of Northern British ColumbiaTodd Zimmerman UBC OkanaganKenny Lozowski The Banff CentreLuke Tymowski University of CalgaryTerry McGuire University of AlbertaTyler Heaton University of LethbridgeGraham Stewart Athabasca UniversityDuy Hoang University of SaskatchewanBarrie Parker University of ReginaBrian Greenberg University of ManitobaAnthony Tordiffe The University of WinnipegKevin Klassen Brandon University

Senior Planning CommitteeNikitas Dimopoulos University of VictoriaMark Thachuk University of British ColumbiaDugan O’Neil Simon Fraser University Martin Siegert Simon Fraser UniversitySergei Noskov University of CalgaryGino DiLabio University of AlbertaPaul Lu University of AlbertaPeter Graham University of Manitoba

Technical Site LeadsDrew Leske University of VictoriaRoman Baranowski University of British ColumbiaMartin Siegert Simon Fraser UniversityPaul Wellings University of CalgaryWaldemar Lysz University of AlbertaJason Hlady University of SaskatchewanJonatan Aronsson University of ManitobaPatrick O’Leary (Chair) WestGridRob Simmonds WestGrid

2012 Resource Allocation CommitteeAndrei Frolov Simon Fraser UniversityDugan O’Neil Simon Fraser UniversityBart Hazes University of AlbertaPeter Kusalik University of CalgaryDennis Salahub University of CalgarySeamus O’Shea University of LethbridgeYuanming Pan University of SaskatchewanKevin Stanley University of SaskatchewanBonev Stanimir Dalhousie University

StaffBrian Corrie Visualization CoordinatorJill Kowalchuk Executive DirectorPatrick O’Leary Director of Operations Lindsay Sill Interim Executive DirectorRob Simmonds Chief Technology Officer Todd Zimmerman Collaboration Coordinator

Financial CommitteeDebra Anderson University of VictoriaSandy Bligh University of VictoriaJoshua Goldberg University of VictoriaRoman Baranowski University of British ColumbiaJanet Lodge University of British ColumbiaJasmine Wong University of British ColumbiaRowena Wong University of British ColumbiaSharon Wu University of British ColumbiaCharlene Allard Simon Fraser UniversityReiko Andrew Simon Fraser UniversityMorie Der Simon Fraser UniversityHelen Doucette Simon Fraser UniversityLeann Liew Simon Fraser UniversityMoses Lo Simon Fraser UniversityPat Nickel Simon Fraser UniversityMartin Siegert Simon Fraser UniversityAneta Douglass University of Northern British ColumbiaEvelyn Evens University of Northern British ColumbiaSean Kinsley University of Northern British ColumbiaCindy Schatkoski The Banff CentreMegan Allen University of CalgarySandra Jones University of CalgaryCordana Mak University of CalgaryChris Turner University of CalgaryAnna Wang University of CalgaryLynda Brulotte University of AlbertaClaire Chen University of AlbertaAndrea Smart University of AlbertaTrudy Smith University of AlbertaNeil Sulakhe University of AlbertaDaryl Schacher University of LethbridgeMark Sera University of LethbridgeBob Hollmann Athabasca UniversitySandra Stone University of SaskatchewanJulie Tsui University of SaskatchewanJurgens Bekker University of ManitobaJackie Cooney University of ManitobaAaron Duncan University of ManitobaMaureen Eyolfson University of ManitobaDon Landry University of ReginaMeigen Schmidt University of ReginaDoug Edge The University of WinnipegAl Dunthorne Brandon UniversityErlin Zurawski Brandon University

Administrative SupportCybera is a not-for-profit, neutral organization that oversees Alberta’s ultra-high-speed research and education network, and pilots new “above-the-network” projects. Cybera provides project management to WestGrid, specifically by coordinating WestGrid’s financial, administrative, and communications activities.

25

Financials

Revenues Total Project Budget Actuals to March 31, 2012

Canada Foundation for Innovation $20,000,000 $19,933,860Government of Alberta $6,750,000 $6,750,000BC Government $9,375,000 $7,247,952Government of Manitoba $1,937,500 $1,937,500Government of Saskatchewan $1,937,500 $1,937,500Western Economic Diversification $750,350 $654,324Private Sector $10,510,674 $26,008,695Institutions $989,831 $1,194,162

$52,250,855 $65,663,993

Expenses Total Project Budget Actuals to March 31, 2012

High Performance Computing $32,110,071 $39,800,572Collaboration/Visualization $2,192,100 $1,909,901Storage $9,906,051 $12,640,934 Personnel $1,000,000 $694,374Software $1,031,000 $437,733Warranties $780,452 $4,469,996Renovations $5,240,181 $5,378,373

$52,259,855 $65,331,883

* All figures are unaudited.* Total project budget amended April 2011 - approved by the Canada Foundation for Innovation.

26

Partners

www.westgrid.ca | info@westgrid.ca | @WestGrid