Embed Size (px)
Citation preview
Research Methods
What is Computer Science?
Origins mathematics, engineering, and commercial practice.
Evolved intotheoretical, experimental and design (or user) orientated aspects.
balance and synthesize these aspects
Research Methods
Research Tools
TheoryAbstraction (= experimentation)Design
Experimentation
Theory(Maths) DesignCS
Research Methods
What is Research?
New Stuff?What sort of stuff?Only for academics?What is good research?Who makes a good researcher?
Research Methods
Introduction to CS Research
Getting started Research proposalsLiterature reviewsPresentationsMSc & PhD work
A research lifecycle Flavours of researchEthics in researchPitfalls
Research Methods
The Research Lifecycle
Research Activities: Literature Search (survey previous work)Do the Work (elaborate ideas and get results)Write Up (plan and write a draft)3! = 6 orderings
ProblemIdentified
ProblemIdentified
ResearchActivities
ResearchActivities
Solution Adopted
Solution Adopted
Research Methods
Is Research a Linear Progression?
Progress?Circular?Evolutionary?
Research Methods
Orders: The Classics
LIT WORK WRITEDon’t know when to stop the literature searchCan inhibit innovation
WORK LIT WRITEMight get a nasty shock (someone else beat you to it)
CONCLUSION: safe options
Research Methods
Orders: quick and dirty
WORK WRITE [LIT]Gamble that the referees (examiners) know less than you
WRITE WORK [LIT]Suits speculative workSometimes used to drag the referees in as unwitting collaborators
CONCLUSION: living on the edge
Research Methods
Orders: paperchasers
WRITE LIT WORKWriting serves as a plan of work focused to produce a single publication
LIT WRITE WORKGood for an idea out of your normal line of research
CONCLUSION: unconventional
Research Methods
Flavours of Research
TheoreticalDevelop new theories
EngineeringDevelop better mechanisms to improve current practice
ExperimentalEvaluate a theory/mechanism – usually via implementation and testingCan include human factors
Some research projects cover the spectrum
Research Methods
Research Outputs
TheoreticalTheoremsModelsAnalyses of existing research
EngineeringSystem architectures & prototypesCode librariesKnowledge basesOntologiesHardware specifications
Research Methods
Experimental Research Outputs
Human factorsSurveys (questionnaires, interviews)Experimental results (lab tests, field studies, case studies)Measurements and opinions (quantitative and qualitative results)
SystemBenchmarksTest programs & measurementsComparative analyses
Research Methods
Ethics in Research
A research lifecycleFlavours of researchEthics in researchPitfalls Getting startedResearch proposalsLiterature reviewsPresentationsMSc & PhD work
Resource: “On being a Scientist: responsible conduct in research”
www.nap.edu/readingroom/books/obas
Research Methods
Avoid conflicts of interest
Science relies on professional judgement which can be compromised by:
Financial conflicts (e.g., undisclosed shares in a company with interest in the outcome of research)Social and personal beliefs (e.g., Einstein’s “God does not play dice”)Pressures of competition (e.g., reviewing a paper with similar work)
Solutions:DisclosureSelf knowledgePeer review
Research Methods
Give credit where its due
Authorship (and order of authors):Conventions can vary considerablyBest to decide upfrontAvoid “honorary” authors – must make a direct and substantial contributionEstablishes accountability as well as credit
Acknowledgements:The place to give credit for less substantial assistance
Citations:Part of the reward system – connected to funding and reputation
Research Methods
Case Study: Pulsars
Credit is a sensitive issue when researchers are of different seniority:
In 1967 Jocelyn Bell, a 24-year-old graduate student, discovered pulsarsSupervised by Anthony Hewish she was in charge of operating and analyzing data from a 4.5 acre radiotelescopeOne day Bell noticed "a bit of scruff" on the data chart Together Bell and Hewish analyzed the signal and found several similar examples elsewhere in the skyWith 3 others they published a paper announcing the discovery
Research Methods
Judgement on Pulsars?
Hewish got a Nobel Prize, Bell did notAgainst:
Bell’s recognition of the signal was the crucial act of discovery
For:Bell didn’t deserve a Nobel Prize for doing what is expected of a graduate student in a project conceived and set up by others
Research Methods
Shut Down the Paper Mill
The publish or perish paper mill:Research careers seem to depend on quantity of publications not quality
Consequences:1. Haste and negligence
But progress relies on a trust in previous results
2. MPUs (minimum publishable units)But this dilutes contribution and forces wading through masses of literature
Research Methods
Case Study: nanotechnology
Jan Schön:Worked in condensed matter physics and nanotechnologyClaimed he could replace silicone-based transistors with organic dye moleculesIn 2001, averaged 1 paper every 8 daysOn track for a Nobel Prize
Found Out:Results seemed suspiciously preciseA researcher spotted identical graphs in two separate papersWhole constructed data sets reused in different experiments
Research Methods
Judgement on Nanotechnology
Outcome:Schön was fired from his position at Bell Labs after an internal investigationMany of his papers were rescindedHe was banned from applying for funding in Germany
Other Consequences:For his co-authors?For reviewers of his papers?
Research Methods
Photo Manipulation
Recently many journals (Cell Biology, Science, Nature) have begun testing for photo manipulationThe following manipulations are not allowed:
Splicing together different images to represent a single experimentChanging brightness and contrast of only a part of the imageAny change that conceals information, even when it is considered to be aspecificShowing only a very small part of the photograph so that additional information is not visible
Research Methods
Misconduct
Fabrication (making up results)Falsification (modifying results)Plagiarism (copying without credit)Suppresion (not reporting negative results)Other deviations from accepted research practice:
Covering up misconduct, misuse of research funds, etc.
Consequences:Harm to individuals, squandering of public funds, attracts criticism of Science
But how can scientists expect to get away with it?
Research Methods
Case Study: Dealing with misconduct
Francine is finishing her Ph.D. and Sylvia is a fellow grad student. Both have the same supervisor.Francine realizes there are problems with Sylvia’s work, she:
Is rarely in the labNever shows anyone her codeHas performance results that seem too “clean” to be real
Also:Francine needs a reference from her supervisor and Sylvia is one of her favouritesBoth Francine and her supervisor are using Sylvia’s results for their own research
Research Methods
Judgement on Misconduct
Should Francine first try to talk with Sylvia, with her supervisor, or with someone else entirely? Does she know enough to be able to raise concerns? Where else can Francine go for information that could help her decide what to do?
Research Methods
Final Do’s and Don’t’s
Do: Get ethics clearance if your research is potentially hazardous to human subjectsrespect:
• IP rights and confidentiality • Patents• The ACM code of ethics
Don’t:Publish the same thing in more than one placeInform the media of results before peer-reviewed publication
Research Methods
Pitfalls
A research lifecycleFlavours of researchEthics in researchPitfalls Getting startedResearch proposalsLiterature reviewsPresentationsMSc & PhD work Source: “The Researcher’s Bible”
Homepages.inf.ed.ac.uk/bundy/how-tos/resbible.html
Research Methods
Solving the World
Easy to pick research goals that are too ambitious
Especially in Artificial Intelligence
Instead:Allow the main burden of scoping to fall on your supervisorFind out where the state of the art liesLook to the future work section of papersCan also redo bad work, properly
Research Methods
Manna from Heaven
Don’t expect inspiration to strike, staring at a blank piece of paper“Science is 99% perspiration and 1% inspiration”Instead:
Read the literature with a question in mindTalk to people - your project partner and supervisor - and explain your ideasTackle a simplified version of your problemWrite down your ideas in a working form
Research Methods
Boondoggling
The appearance of work without actual productivity
Surprisingly seductiveCoding for its own sakeWriting Yet Another Programming Language (YAPL)Gathering unnecessary experimental data
Instead:Make sure your programming and experimentation contributes directly to the research
Research Methods
Ivory Tower
Focus on your topic is good but don’t shut out the rest of the world completely
Because it prevents cross-pollenisation of ideas
Instead:Keep in touch with the state of the art in related fields - attend colloquia and talk to other students about their researchSet aside a part of the week for reading abstracts and skimming papers
Research Methods
Misunderstood Genius
Easy to believe that no one understands your ideas because you are a geniusMore likely:
Love of jargon. CS is full of jargon. Try to rephrase your ideas using ordinary EnglishIf I can do it, it's trivial. Once you have seen the solution to a problem it appears simple Love of complexity. It’s not a virtue to make an unnecessarily complicated program - it’s just a nuisance to other people. Occam and Einstein were right!
Research Methods
Starting Research
A research lifecycleFlavours of researchEthics in researchPitfalls Getting startedResearch proposalsLiterature reviewsPresentationsMSc & PhD work
Research Methods
Finding a Research Question
What problems to tackle?:What matters to you Anything messy or difficultNew technologies New users A paper you enjoyed A paper you disliked
Example question: “How can the new technology <T> be adapted to run on Cell Phones/PDA’s?”
Research Methods
Research Fit
What research are you suited to?what interests you?what expertise exists around you?What are your skills?
Don’t neglect methodology:Look at similar research to decide what kind of methodology is best for your research questionMake sure you are willing to apply that methodology
Research Methods
It’s not easy so why do it ?
Satisfaction & thrill of being the first to ever create/understand something
Famous Eureka moment
Privilege and recognition of adding to human knowledgeMeet/work with passionate, deep thinkersFreedom – what to do, how, when, with whom
Research Methods
Refining the Research Question
Choose an initial objectiveRead the literatureRefine the objective:
Narrow it Write it as a question Describe it in a single sentence Decide on the measure of success
Do a quick first prototype/experiment
Research Methods
Research Proposal: Preparation
Ask yourself:Am I familiar with related research in this area Do I have a good understanding of the steps that will be involved in achieving these goals Do I have the ability to successfully conduct each of these steps Am I sufficiently motivated and enthusiastic about all the steps in this project Am I convinced that the results of this research will be useful to others
Research Methods
Research Proposal: Structure
Honours proposal structure:Project DescriptionRelated WorkOutcomes (system, questions tackled, expected impact, key success factors)Work Detail (timeline, resources required, deliverables, milestones, work allocation)References
More general proposals might include:Budget, CV, Dissemination plan
Research Methods
Research Proposal: Evaluation
Have you answered these questions:What you are planning to do ?Why ?What the difficulties are?Is it feasible for you ?Do you have a plan of how to do it ?Have you done your homework ?
Research Methods
Literature Reviews
Getting started Research proposalsLiterature reviewsPresentationsMSc & PhD work
A research lifecycle Flavours of researchEthics in researchPitfalls
Research Methods
What is a Literature Review?
As a process:Reading, taking notes, organising, documentingStart with general, broad, textbook works Move toward specialised, recent papers
As a document:Not just a string of article summariesRather a coherent discussion of previous related workIncludes intro, conclusion, references
Don’t confuse the document and process
Research Methods
The Literature Review ProcessBroad
Narrow
Understand the Field
Understand the Field
Find a SurveyFind a Survey
Find FocusedResearch
Find FocusedResearch
Back Chain to References
Back Chain to References
Forward Chain to Citations
Forward Chain to Citations
• Lecture Notes • Text Books
• Text Books • Survey Papers• Theses
• Research Papers
Research Methods
Resources
Experts: lecturers, supervisors, librariansACM Digital Library
http://portal.acm.org/dl.cfm Includes most ACM pubs (but not IEEE)UCT has a subscription
Google Scholarhttp://scholar.google.com/Good all-round resource
CiteSeerhttp://citeseer.ist.psu.edu/ Digital library and search engine Heavily linked meta-data allows chaining through citations
Research Methods
Critical Reading Required
Being able to read rapidly and critically is a vital skillFirst Skim:
Abstract, section headings and figures
Then Dive into Detail:May require reading references for a full understanding
Take Notes:Complete citationMain research question & conclusionsResearch methodologyKey ideas or results relevant to your researchFuture workGaps/problems
Research Methods
Presentations
A research lifecycleFlavours of researchEthics in research Getting startedResearch proposalsLiterature reviewsPresentationsMSc & PhD work
Research Methods
Communication of Ideas
Feedback is important:To learn of new developmentsTo share responsibilityTo get support and adviceTo develop communication skills
Teamwork is important:As a forum for feedbackTo tackle larger problemsTo learn interpersonal skills
Modern Science is advanced by sharing ideas and working in teams
Research Methods
Presentations
Present your main idea & its significanceOmit complicated & old ideasStructure:
Intro, context, body, some detail, conclude
Know your audiences’ backgroundMake sure that both non- & experts benefitDon’t gloss over problems with your ideasAnticipate questions
Research Methods
Slide Layout
Allow about 2-3 minutes per slideAvoid too much text
Just cuesAbout 7-15 bullet points at most
Include graphs/charts/pictures Avoid code/maths
Research Methods
Nervous ?
Prepare well, run through with your supervisor Time yourself Contract stomach muscles & breathe out hardSpeaking Skills:
A pause is better than an interjection Speak more slowly than you think necessaryRepeat questions if they are inaudible
Research Methods
Postgraduate Work
A research lifecycleFlavours of researchEthics in research Getting startedResearch proposalsLiterature reviewsPresentationsMSc & PhD work
Research Methods
Postgraduate Research
Masters (MSc):Must demonstrate that you are familiar with the state of the art in the field and are capable of working at that level
Doctorate (PhD):A PhD additionally requires that you are able to contribute new knowledgeNeed not be a major contribution
Research Methods
Examiner’s Instructions
A Candidate must demonstrate that he/she:Principles: Understands the nature, objectives and scientific principles underlying the investigationLiterature: Is adequately acquainted with the relevant literatureMethodology: Has mastered appropriate techniques and analytical methodsFindings: Assesses the significance of findings in a thorough and logically coherent mannerIndependence: Shows evidence of critical and independent thoughtWriting: Reports on the study in an acceptable scientific format that is satisfactory in both presentation and literary style
Research Methods
Considering an MSc?
Choose your supervisor carefullyTalk to their current studentsBe aware of their supervision style
Be prepared to change your topic if necessary
But do this early rather than late
Organise finances upfront and prepare a budget
Ask about Bursaries, Tutoring, etc.
Beware of part-time research. Consider the opportunity cost
