How to write Thesis in Computer Science

how to write a computer science thesis proposal and how to write master thesis and phd thesis computer science and how to write a computer science thesis proposal
Dr.JohnParker Profile Pic
Dr.JohnParker,Singapore,Researcher
Published Date:02-07-2017
Your Website URL(Optional)
Comment
Justin Zobel Writing for Computer Science Third Edition Chapter 1 Introduction This writing seemeth to me … not much better than the noise or sound which musicians make while they are in tuning their instruments. Francis Bacon The Advancement of Learning No tale is so good that it can’t be spoiled in the telling. Proverb Writing plays many roles in science. We use it to record events and clarify our thinking. We use it to communicate to our colleagues, as we explain concepts and discuss our work. And we use it to add to scientific knowledge, by contributing to books, journals, and conference proceedings. Unfortunately,manyresearchersdonotwritewell.Bacon’squotegivenabovewas made four hundred years ago, yet applies to much science writing today. Perhaps we shouldnotalwaysexpectresearcherstocommunicatewell;surelytheskillsrequired for science and writing are different. But are they? The best science is based on straightforward,logicalthinking,anditisn’trich,artisticsentencesthatweexpectin aresearchpaper—weexpectreadability.Ascientistwhocanconceiveofandexplore interestingideasinarigorouswayshouldbeabletousemuchthesameskillstosolve the problem of how to explain and present those ideas to other people. However, many researchers undervalue the importance of clarity, and underesti- matetheeffortrequiredtoproduceahigh-qualitypieceofwriting.Someresearchers seemcontenttowritebadly,andperhapshaven’tconsideredtheimpactofpoorwrit- ing on their readers, and thus on their own careers. A research paper can remain relevant for years or even decades and, if published in a major journal or conference, may be read by thousands of students and researchers. Everyone whose work is affected by a poorly written paper will suffer: ambiguity leads to misunderstanding; omissions frustrate; complexity makes readers struggle to reconstruct the author’s intention. Effort used to understand the structure of a paper or the syntax of its sentences is effort not used to understand its content. And, as the proverb tells us, no tale is so goodthatitcan’tbespoiledinthetelling.Irrespectiveoftheimportance andvalidity ofapaper, itcannot be convincing ifitisdifficult tounderstand. The moreimportant © Springer-Verlag London 2014 1 J. Zobel, Writing for Computer Science, DOI 10.1007/978-1-4471-6639-9_12 1 Introduction the results—or the more startling or unlikely they seem—the better the supporting arguments and their presentation should be. Remember that, while you have months or years to prepare your work, reviewers and examiners often have no more than hours and may have rather less. You need to help them to spend their time well. For writing about science to be respected, a researcher must have something of valuetosay.Apaperorthesisreportsonresearchundertakenaccordingtothenorms of the field, to a standard that persuades a skeptical reader that the results are robust and of interest. Thus the written work rests on a program of activity that begins with interesting questions and proceeds through a sound methodology to clear results. Fewresearchersareinstinctivewriters,andfewpeopleareinstinctiveresearchers. Yetitisnotsodifficulttobecomeagoodwriter.Thosewhodowritewellhave,largely, learnt through experience. Inexperienced researchers can produce competent papers by doing no more than follow some elementary steps: create a logical organization, useconcisesentences,reviseagainstchecklistsofpossibleproblems,seekfeedback. Likewise, the skills of research must be learnt, and early attempts at investigation and experimentation are often marked by mistakes, detours, and fumbling; but, as for writing, competent work can be produced by appreciating that there is a more or less standard template that can be followed, and then using the template to produce a first research outcome. Most researchers find that their work improves through practice, experience, and willingness to continue to reflect and learn. This observation certainly applies to me. I’ve continued to develop as a writer, and today produce text much more quickly— and with better results—than when I wrote the second edition a decade ago. I’m also a better scientist, and, looking back just a few years, am aware of poor research outcomes that are due to mistakes I would not make today. In my experience, most scientists develop a great deal as they proceed through their careers. Kinds of Publication Scientific results can be presented in a book, a thesis, a journal article, a paper or extended abstract in a conference or workshop proceedings, or a manuscript. Each kind of publication has its own characteristics. Books—the form of publication that undergraduates are the most familiar with—are usually texts that tend not to contain new results or provide evidence for the correctness of the information they present. The main purpose of a textbook is to collect information and present it in an acces- sible, readable form, and thus textbooks are generally better written than are papers. The other forms of publication are for describing the outcomes of new research. A thesis is usually a deep—or even definitive—exploration of a single problem. Journals and conference proceedings consist of contributions that range from sub- stantial papers to extended abstracts. A journal paper is typically an end product of the research process, a careful presentation of new ideas that has been revised (sometimesoverseveraliterations)accordingtoreferees’andcolleagues’suggestions and criticisms. A paper or extended abstract in conference proceedings can likewise be an end-product, but conferences are also used to report work in progress. ConferenceKinds of Publication 3 papers are usually refereed, but with more limited opportunities for iteration and revision, and may be constrained by strict length limits. There is no universal defin- ition of “extended abstract”, but a common meaning is that the detail of the work is omitted. That is, an extended abstract may review the results of a research program, but may not include enough detail to make a solid argument for the claims. In contrast to books—which can reflect an author’s opinions as well as report on established scientific knowledge—the content of a paper must be defended and justified. This is the purpose of reviewing: to attempt to ensure that papers published in a reputable journal or conference are trustworthy, high-quality work. Indeed, in a common usage a published paper is distinguished from a mere paper by having been refereed. Atypicalresearchpaperconsistsofthearguments,evidence,experiments,proofs, andbackground requiredtosupportandexplain acentralhypothesis. Incontrast,the process of research that leads to a paper can include uninteresting failures, invalid hypotheses, misconceptions, and experimental mistakes. With few exceptions these do not belong in a paper. While a thesis might be more inclusive, for example if the author includes a critical reflection on how the work developed over the course of a Ph.D., such material would usually be limited to mistakes or failures that are genuinelyilluminating.Apaperorthesisshouldbeanobjectiveadditiontoscientific knowledge, not a description of the path that was taken to the result. Thus “style” is not just about how to write, but is also about what to say. Writing, Science, and Skepticism Science is a system for accumulating reliable knowledge. Broadly speaking, the process of science begins with speculation, observation, and a growing understand- ing of some idea or phenomenon. This understanding is used to shape research questions, which in turn are used to develop hypotheses that can be tested by proof or experimentation. The results are described in a paper, which is then submitted for independent review before (hopefully) being published; or the results are described in a thesis that is then submitted for examination. Writing underpins the whole of the research cycle. A key aspect of writing is that the discipline of stating ideas as logical, organized text forces you to formulate and clarify your thoughts. Concepts and ideas are made concrete; the act of writing sug- gestsnewconceptstoconsider;writtenmaterialcanbesystematicallydiscussedand debatedwithcolleagues;andtheonlyeffectivewaytodevelopcomplexargumentsor threadsofreasoning,andevaluatewhethertheyarerobust,istowritethemdown.That is,writingisnottheendoftheresearchprocess,butinsteadshapesit.Onlythestyling of a paper, the polishing process, truly takes place after the research is complete. Thus the ability to write well is a key skill of science. Like many aspects of research,writingcanonlybethoroughlylearntwhileworkingwithotherresearchers. Toooften,however,theonlyhelpanovicereceivesisanadvisor’sfeedbackondrafts of papers. Such interaction can be far from adequate: many researchers have little4 1 Introduction experience of writing extended documents, and may be confronting the difficulties of writing in English when it is not their first language. It is not surprising that some researchers struggle. Many are intimidated by writing, and avoid it because describing research is less entertaining than actually doing it. For some advisors, the task of helping a student to write well is not one that comes naturally, and can be a distraction from the day-to-day academic work of research and teaching. Yet writing defines what we consider to be knowledge. Scientific results are only accepted as correct once they are refereed and published; if they aren’t published, 1 they aren’t confirmed. Each new contribution builds on a foundation of existing concepts that are known and, within limits, trusted. New research may be wrong or misguided,buttheprocessof reviewing eliminates someworkofpoor quality,while thescientificcultureofquestioningideasandrequiringconvincingdemonstrationsof their correctness means that, over time, weak or unsupported concepts are forgotten. Aunifyingprincipleforthescientificculturethatdeterminesthevalueofresearch is that of skepticism. Within science, skepticism is an open-minded approach to knowledge: a researcher should accept claims provisionally given reasonable evi- dence and given agreement (or at least absence of contradiction) with other provi- sionally accepted claims. A skeptic seeks the most accurate description or solution that fits the known facts, without concern for issues such as the need to seek favour withauthorities,whilesuspendingjudgementuntildecisiveinformationisavailable. Effective research programs are designed to seek the evidence needed to convince a reasonable skeptic. Absolute skepticism is unsustainable, but credulity—the will- ingness to believe anything—is pointless, as, without some degree of questioning, it is impossible for knowledge to progress. Skepticismiskeytogoodscience.Foranideatosurvive,otherresearchersmustbe persuaded of its relevance and correctness—not with rhetoric, but in the established framework of a scientific publication. New ideas must be explained clearly to give themthebestpossiblechance ofbeing understood,believed, remembered, andused. Thisbeginswiththetaskofexplainingourideastothepersonatthenextdesk,oreven toourselves.Itendswithpublication,thatis,anexplanationofresultstotheresearch community. Thus good writing is a crucial part of the process of good science. Using This Book There are many good general books on writing style and research methods, but the conventions of style vary from discipline to discipline, and broad guidance on science writing can be wrong or irrelevant for a specific area. Some topics—such as algorithms, mathematics, and research methods for computer science—are not discussed in these books at all. Theroleofthisbookistohelpcomputerscientistswiththeirwritingandresearch. Fornovices, itintroduces theelements ofascientificpaper andreviews awiderange 1 Which is why codes of scientific conduct typically require that scientists not publicize their discoveries until after the work has been refereed.Using This Book 5 of issues that working researchers need to consider. For experienced researchers, it provides a reference point against which they can assess their own views and abili- ties, and is an exposure to wider cultures of research. This book is also intended to encouragereflection;somechaptersposequestionsaboutresearchthataresponsible researcher should address. Nobody can learn to write or become a researcher just by reading this book, or indeed any book. To become competent it is necessary to practice, that is, to do research and write it up in collaboration with experienced researchers. However, familiarity with the elements of writing and research isessen- tial in scientific training. Style is in some respects a matter of taste. The advice in this book is not a code of law to be rigidly obeyed; it is a collection of guidelines, not rules, and there are inevitablysituationsinwhichthe“correct”stylewillseemwrong.Butgenerallythere aregoodreasonsforwritinginacertainway.Almostcertainlyyouwilldisagreewith some of the advice in this book, but exposure to another opinion should lead you to justify your own choice of style, rather than by habit continue with what may be poor writing. A good principle is: By all means break a rule, but have a good reason for doing so. Most computer scientists can benefit from reading a book about writing and research. This book can be used as the principal text for a senior research meth- ods subject, or for a series of lectures on the practice of research. Such a subject would not necessarily follow this book chapter by chapter, but instead use it as a resource. In my own teaching of research methods, lectures on writing style seem to workbestasintroductionstothekeytopicsofgoodwriting;talkingstudentsthrough the detailed advice given here is less effective than getting them to read the book while they write and undertake research for themselves. That said, for a range of topics—figures, algorithms, presentations, statistics, reading and reviewing, draft- ing, ethics, and experimentation, for example—the relevant chapter can be used as the basis of one or two lectures. This book covers the major facets of writing and experimentation for research in computer science: � Commencingaresearchprogram,includinggettingstartedontheresearchandthe writing (Chap.2), reading and reviewing (Chap.3), and principles of hypotheses, research questions, and evidence (Chap.4). � Organization of papers and theses, and the practice of writing (Chap.5). � Good writing, including writing style (Chaps.6–8), mathematical style (Chap.9), presentation of algorithms (Chap.10), design of figures and graphs (Chap.11), expert writing for other professional contexts (Chap.12), and final editing (Chap.13). � Research methodology, including experimentation (Chap.14) and statistical prin- ciples (Chap.15). � Presentations, including talks and posters (Chap.16). � Ethics (Chap.17). There are also exercises to help develop writing and research skills.6 1 Introduction If you are new to research, Chaps.2–5 may be the right place to begin. Note too that much of the book is relevant to writing in computer science in general, in particular Chaps.6–13. While the examples and so on are derived from research, the lessonsarebroader,andapplytomanyofthekindsofwritingthatprofessionalshave to undertake. This book has been written with the intention that it be browsed, not memorized or learnt by rote. Read through it once or twice, absorb whatever advice seems of value to you, then consult it for specific problems. There are checklists to be used as a reference for evaluating your work, at the ends of Chaps.2, 4, 5, and 12–17, and, to some extent, all of the chapters are composed of lists of issues to check. Some readers of this book will want to pursue topics further. There are areas where the material is reasonably comprehensive, but there are others where it is only introductory, and still others where I’ve done no more than note that a topic is important. For most of these, it is easy to find good resources on the Web, which is where I recommend that readers look for further information on, for example, statistical methods, human studies and human ethics, and the challenges that are specific to authors whose first language is not English. Earlier editions of this book included bibliographies. These rapidly dated, and, with many good reading lists online—and new materials appearing all the time— Isuggestthatreaderssearchfortextsandpapersontopicsofinterest,usingtheonline reviewforumsasguides.Therearemanyhomepagesforresearchmethodssubjects, on research in general and in the specific context of computing, where up-to-date readings can be found. Spelling and Terminology British spelling is used throughout this book, with just a couple of quirks, such as use of “program” rather than “programme”. American readers: There is an “e” in “judgement” and a “u” in “rigour”—within these pages. Australian readers: There is a “z” in “customize”. These are choices, not mistakes. Choiceofterminologyislessstraightforward.Anundergraduate isanundergrad- uate, but the American graduate student is the British or Australian postgraduate. The generic “research student” is used throughout, and, making arbitrary choices, “thesis”ratherthan“dissertation”and“Ph.D.”ratherthan“doctorate”.Theacademic staffmember(facultyinNorthAmerica)whoworkswith—“supervises”—aresearch student is, in this book, an “advisor” rather than a “supervisor”. Collectively, these people are “researchers” rather than “scientists”; while “computer scientists” are, in a broad sense, not just researchers in the discipline in computer science but people who are computational experts or practitioners. Researchers write articles, papers, reports, theses, extended abstracts, and reviews; in this book, the generic term for theseformsofresearchwritingisa“write-up”,while“paper”isusedforbothrefereed publications and for work submitted for reviewing, and, sometimes, for theses too.Spelling and Terminology 7 Some of the examples are based on projects I’ve been involved in. Most of my research has been collaborative; rather than use circumlocutions such as “my col- leagues and I”, or “together with my students”, the simple shorthand “we” is used to indicate that the work was not mine alone. Many of the examples of language use are drawn from other people’s writing; in some cases, the text has been altered to disguise its origin.Chapter 2 Getting Started Science is more than a body of knowledge; it is a way of thinking. Carl Sagan The Demon-Haunted World There are as many scientific methods as there are individual scientists. Percy W. Bridgman On “Scientific Method” There are many ways in which a research project can begin. It may be that a conversationwithacolleaguesuggestedinterestingquestionstopursue,orthatyour generalinterestinatopicwascrystallizedintoaspecificinvestigationbysomething learnt in a seminar, or that enrollment in a research degree forced you to identify a problem to work on. Then definite aims are stated; theories are developed or exper- iments are undertaken; and the outcomes are written up. Thetopicofthischapterisaboutgettingstarted:findingaquestion,workingwith an advisor, and planning the research. The perspective taken is a practical one, as a workingscientist:Whatkindsofstagesandeventsdoesaresearcherhavetomanage in order to produce an interesting, valid piece of work? This chapter, and Chaps.3, 4, 14, and 15, complement other parts of the book—which are largely on the topic of how research should be described—by considering how the content of paper is arrived at. Thus this chapter concerns the first of the steps involved in doing a research project, which broadly are: � Formation of a precise question, the answer to which will satisfy the aim of the research. � Developmentofadetailed understanding,throughreadingandcriticalanalysisof scientific literature and other resources. � Gathering of evidence that relates to the question, through experiment, analysis, ortheory.Theseareintendedtosupport—ordisprove—thehypothesisunderlying the question. � Linking of the question and evidence with an argument, that is, a chain of reasoning. � Description of the work in a publication. © Springer-Verlag London 2014 9 J. Zobel, Writing for Computer Science, DOI 10.1007/978-1-4471-6639-9_210 2 Getting Started Learning to do research involves acquisition of a range of separate skills. It takes experiencetoseetheseskillsaspartofasingleintegrated“processofresearch”.That is, many people learn to be researchers by working step by step under supervision; only after having been through the research process once or twice does the bigger picture become evident. Some newcomers try to pursue research as if it were some other kind of activity. For example, in computer science many research students see experimentation as a form of software development, and undertake a research write-up as if they were assemblinganessay,ausermanual,orsoftwaredocumentation.Partoflearningtobe ascientistisrecognitionofhowtheaimsofresearchdifferfromthoseofcoursework. A perspective on research is that it is the process that leads to papers and the- ses, because these represent our store of accepted scientific knowledge. Another perspective is that it is about having impact; by creating new knowledge, successful researchchangesthepracticesandunderstandingsofotherscientists.Ourworkmust beadoptedinsomewaybyothersifitistobeofvalue.Thusanotherpartoflearning to be a scientist is coming to understand that publication is not an end in itself, but is part of an ongoing collaborative enterprise. Beginnings The origin of a research investigation is typically a moment of insight. A student attending a lecture wonders why search engines do not provide better spelling cor- rection.Aresearcherinvestigatingexternalsortingisataseminaronfilecompression, andponderswhetheronecouldbeofbenefittotheother.Anadvisorisfrustratedby network delays and questions whether the routing algorithm is working effectively. A student asks a professor about the possibility of research on evaluation of code reliability; the professor, who hadn’t previously contemplated such work, realises that it could build on recent advances in type theory. Tea-room arguments are a rich source of seed ideas. One person is idly speculating, just to make conversation; another pursues the speculation and a research topic is created. Or someone claims thataresearcher’sideaisunworkable,andalistenerstartstoturnoverthearguments. What makes it unworkable? How might those issues be addressed? This first step is a subjective one: to choose to explore ideas that seem likely to succeed, or are intriguing, or have the potential to lead to something new, or that contradict received wisdom. At the beginning, it isn’t possible to know whether the work is novel or will lead to valuable results; otherwise there would be no scope for research. The final outcome is an objective scientific report, but curiosity and guesswork are what establish research directions. It is typically at this stage that a student becomes involved in the research. Some studentshaveaclearideaofwhattheywanttopursue—whetheritisfeasible,rational, or has research potential is another matter—but the majority are in effect shopping for a topic and advisor. They have a desire to work on research and to be creative, perhapswithoutanydefiniteideaofwhatresearchis.TheyaredrawnbyaparticularBeginnings 11 area or problem, or want to work with a particular individual. Students may talk through a range of possible projects with several alternative advisors before making a definite choice and starting to work on a research problem in earnest. Shaping a Research Project How a potential research topic is shaped into a defined project depends on context. Experienced scientists aiming to write a paper on a subject of mutual interest tend tobefairlyfocused:theyquicklydesignaseriesofexperimentsortheoreticalgoals, investigate the relevant literature, and set deadlines. Forstudents,doingaresearchprojectadditionallyinvolvestraining,whichaffects how the work proceeds. Also, for a larger research program such as a Ph.D., there are both short-term and long-term goals: short-term goals include the current spe- cific explorations, which may be intended to lead to an initial research paper; the long-termgoals are thewider investigation that willeventually formthe basisof the student’s thesis. At the beginning of a research program, then, you need to establish answers to two key questions. First, what is the broad problem to be investigated? Second, what are the specific initial activities to undertake and outcomes to pursue? Having clear short-term research goals gives shape to a research program. It also gives the studenttrainingintheelementsofresearch:planning,reading,programming,testing, analysis, critical thinking, writing, and presentation. Forexample,inresearchinthe1990sintoalgorithmsforinformationretrieval,we observed that the time to retrieve documents from a repository could be reduced if theywerefirstcompressed;thecostofdecompressionafterretrievalwasoutweighed by savings in transfer times. A broad research problem suggested by this topic is whether compression can be of benefit within a database even if the data is stored uncompressed.Pursuingthisproblemwitharesearchstudentledtoaspecificinitial researchgoal:givenalargedatabasetablethatiscompressedasitisreadintomemory, isitpossibletosortitmorerapidlythanifitwerenotcompressedatall?Whatkinds of compression algorithm are suitable? Success in these specific explorations leads to questions such as, where else in a database system can compression be used? Failure leads to questions such as, under what conditions might compression be useful? When developing a topic into a research question, it is helpful to explore what makes the topic interesting. Productive research is often driven by a strong moti- vating example, which also helps focus the activity towards useful goals. It can be easy to explore problems that are entirely hypothetical, but difficult to evaluate the effectiveness of any solutions. Sometimes it is necessary to make a conscious deci- sion to explore questions where work can be done, rather than where we would like to work; just as medical studies may involve molecular simulations rather than real patients,roboticsmayinvolvetheartificeofsoccer-playingratherthantherealityof planetary exploration.12 2 Getting Started In choosing a topic and advisor, many students focus on the question of “is this themostinterestingtopiconoffer?”,oftentotheexclusionofotherquestionsthatare equally important. One such question is “is this advisor right for me?” Students and advisors form close working relationships that, in the case of a Ph.D., must endure for several years. The student is typically responsible for most of the effort, but the intellectualinputisshared,andtherelationshipcangrowovertimetobeapartnership of equals. However, most relationships have moments of tension, unhappiness, or disagreement. Choosing the right person—considering the advisor as an individual, not just as a respected researcher—is as important as choosing the right topic. A charismatic or famous advisor isn’t necessarily likeable or easy to work with. Thefactthatatopicisinafashionableareashouldbeatmostaminorconsidera- tion;thefashionmaywellhavepassedbeforethestudenthasgraduated.Sometrends are profound shifts that have ongoing effects, such as the opportunities created by the Web for new technologies; others are gone almost before they arrive. While it isn’t necessarily obvious which category a new trend belongs in, a topic should not be investigated unless you are confident that it will continue to be relevant. Another important question is, is this project at the right kind of technical level? Some brilliant students are neither fast programmers nor systems experts, while others do not have strong mathematical ability. It is not wise to select a project for which you do not have the skills or that doesn’t make use of your strengths. A single research area can offer many different kinds of topic. Consider the fol- lowing examples of strengths and topics in the area of Web search: Statistical. IdentifypropertiesofWebpagesthatareusefulindeterminingwhether they are good answers to queries. Mathematical. Provethattheefficiencyofindexconstructionhasreachedalower bound in terms of asymptotic cost. Analytical. Quantifybottlenecksinqueryprocessing,andrelatethemtoproperties of computers and networks. Algorithmic. Develop and demonstrate the benefit of a new index structure. Representational. Proposeandevaluateaformallanguageforcapturingproperties of image, video, or audio to be used in search. Behavioural. Quantify the effect on searchers of varying the interface. Social. Link changes in search technology to changes in queries and user demo- graphics. As this list illustrates, many skills and backgrounds can be applied to a single problem domain. An alternative perspective on the issue of how to choose a topic is this: most projects that are intellectually challenging are interesting to undertake; agonizing over whether a particular option is the project may not be productive. However, it is also true that some researchers only enjoy their work if they can identify a broadervalue:forexample,theycanseelikelypracticaloutcomes.HighlyspeculativeShaping a Research Project 13 projectsleavesomepeopledissatisfied,whileothersareexcitedbythepossibilityof a leap into the future. When evaluating a problem, a factor to consider is the barrier to entry, that is, the knowledge, infrastructure, or resources required to do work in a particular field. Sometimesitjustisn’tpossibletopursueacertaindirection,becauseofthecosts,or becauseno-oneinyourinstitutionhasthenecessaryexpertise.Anothervariantofthe sameissueistheneedforacodebase,orexperienceinacodebase;ifinvestigationof acertainqueryoptimizationproblemmeansthatyouneedtounderstandandmodify the source code for a full-strength distributed database system, then possibly the project is beyond your reach. As research fields mature, there is a tendency for the barrier to entry to rise: the volume of background knowledge a new researcher must master is increased, the scope for interesting questions is narrowed, the straightforward or obvious lines of investigation have been explored, and the standard of the baselines is high. If a field is popular or well-developed, it may make more sense to explore other directions. Project scale is a related issue. Some students are wildly ambitious, entering research with the hope of achieving something of dramatic significance. However, major breakthroughs are by definition rare—otherwise, they wouldn’t be major— while,asmostresearchersdiscover,evenaminoradvancecanbeprofoundlyreward- ing.Moreover,anambitiousprojectcreatesahighpotentialforfailure,especiallyin a shorter-term project such as a minor thesis. There is a piece of folklore that says that most scientists do their best work in their Ph.D. This is a myth, and is certainly not a good reason for tackling a problem that is too large to resolve. Mostresearchistosomeextentincremental:itimproves,repairs,extends,varies, or replaces work done by others. The issue is the magnitude of the increment. A trivialstepthatdoesnomorethanexploreanobvioussolutiontoasimpleproblem— achange,say,tothefieldsinanetworkpackettosaveacoupleofbits—isunlikelyto beworthinvestigating.Thereneedstobechallengeandthepossibilityofunexpected discovery for research to be interesting. For a novice researcher, it makes sense to identify outcomes that can clearly be achieved; this is research training, after all, not research olympics. A principle is to pursue the smallest question that is interesting. If these outcomes are reached early on, it should be straightforward, in a well-designed project, to move on to more challenging goals. Someresearchisconcernedwithproblemsthatappeartobesolvedincommercial or production software. Often, however, research on such problems can be justified. Inatypicalcommercialimplementationthetaskistofindaworkablesolution,while inresearchthequalityofthatsolutionmustbemeasured,andthusworkonthesame problem that produces similar solutions can nonetheless have different outcomes. Moreover, while it is in a company’s interests to claim that a problem is solved by their technology, such claims are not easily verified. In some cases, investigation of a problem for which there is already a commercial solution can be of as much value as investigation of a problem of purely academic interest.14 2 Getting Started Research Planning Students commencing their first research project are accustomed to the patterns of undergraduate study: attending lectures, completing assignments, revising for exams. Activity is determined by a succession of deadlines that impose a great deal of structure. In contrast, a typical research project has just one deadline: completion. Admin- istrative requirements may impose some additional milestones, such as submission ofaprojectoutlineoraprogressreport,butmanystudents(andadvisors)donottake these milestones seriously. However, having a series of deadlines is critical to the success of a project. The question then is, what should these deadlines be and how should they be determined? Some people appear to plan their projects directly in terms of the aspects of the problemthatattractedtheminthefirstplace.Forexample,theydownloadsomecode or implement something, then experiment, then write up. A common failing of this approach to research is that each stage can take longer than anticipated, the time for write-up is compressed, and the final report is poor. Yet the write-up is the only part of the work that survives or is assessed. Arguably, an even more significant failing is that the scientific validity of the outcomes can be compromised. It is a mistake, forexample,toimplementacompletesystemratherthanaskwhatcodeisneededto explore the research questions. A strong approach to the task of defining a project and setting milestones is to explicitlyconsiderwhatisneededattheend,thenreasonbackwards.Thefinalthing required is the write-up in the form of a thesis, paper, or report; so you need to plan in terms of the steps necessary to produce the write-up. As an example, consider researchthatisexpectedtohaveasubstantialexperimentalcomponent;thewrite-up islikelytoinvolveabackgroundreview,explanationsofpreviousandnewalgorithms, descriptions of experiments, and analysis of outcomes. Completion of each of these elements is a milestone. Continuingtoreasonbackwards,thenextstepistoidentifywhatformtheexper- iments will take. Chapter14 concerns experiments and how they are reported, but priortodesigningexperimentstheresearchermustconsiderhowtheyaretobeused. What will the experiments show, assuming the hypothesis to be true? How will the results be different if the hypothesis is false? That is, the experiments are an evalua- tion of whether some hypothesised phenomenon is actually observed. Experiments involvedata,code,andsomekindofplatform.Runningofexperimentsrequiresthat all three of these be obtained, and that skeptical questions be asked about them: whether the data is realistic, for example. Experiments may also involve users. Who will they be? Is ethics clearance required? Computer scientists, accustomed to working with algorithms and proofs, areoftensurprisedbyhowwide-rangingtheiruniversityethicsrequirementscanbe. Many research activities do not have an experimental component, and instead concern principles, or fresh analysis of data, or qualitative interpretation of a case study, or a comparative reflection, or any of a wide range of other kinds of work.Research Planning 15 However, milestones can always be identified, because (obviously, I hope) any substantialprojectcanbemeaningfullydescribedasacollectionofsmalleractivities. Twopointshereareworthemphasizing.First,whilethecomponentsofaresearch projectshouldbeidentifiedinadvance,theydonotnecessarilyhavetobecompleted in turn. Second, we should plan research with the following attitude: what evidence mustwecollecttoconvinceaskepticalreaderthattheresultsarecorrect?Asuccessful research outcome rests on finding a good answer to this question. Having identified specific goals, another purpose of research planning is to esti- matethedatesatwhichmilestonesshouldbereached.Oneoftheaxiomsofresearch, 1 however,isthateverythingtakeslongerthanplannedfor. Atraditionalresearchstrat- egyistofirstreadtheliterature,thendesign,thenanalyseorimplement,thentestor evaluate, then write up. A more effective strategy is to overlap these stages as much as possible. You should begin the implementation, analysis, and write-up as soon as it is reasonable to do so. For the long-term research activity of a Ph.D., there are other considerations that becomesignificant.AtypicalconcerninthelaterstagesofaPh.D.iswhetherenough researchhasyetbeendone,orwhetheradditionalnewworkneedstobeundertaken. Oftenthebestresponsetothisquestionistowritethethesis.Onceyourthesisismore or less complete, it should be easy to assess whether further work is justified. Doing suchadditionalworkprobablyinvolvesfillingawell-definedgap,ataskthatismuch better defined than that of fumbling around for further questions to investigate. Thus, rather than working to a schedule of long-term timelines that may be unre- alistic, be flexible. Adjust the work you are doing on a day-to-day basis—pruning yourresearchgoals,givingmoretimetothewriting,addressingwhateverthecurrent bottleneck happens to be—to ensure that you are reaching overall aims. Students and Advisors Advisors are powerful figures in their students’ lives, and every student–advisor relationshipisdifferent.Someprofessorsatthepeakoftheircareersstillhavestrong views—oftenoutrageoramazement—about theirownadvisors,despitemanyyears ofexperienceontheothersideofthefence.Talesincludethatofthestudentwhosaw his advisor twice, once to choose a topic and once to submit; and that of the advisor who casually advised a student to “have another look at some of those famous open problems”. Thankfully these are rare exceptions. Thepurposeofaresearchprogram—aPh.D.,masters,orminorthesis—isforthe universitytoprovideastudentwithresearchtraining,whilethestudentdemonstrates thecapabilitytoundertakeresearchfromconceptiontowrite-up,includingsuchskills asworkingindependentlyandproducingnovel,criticalinsights.Aside-benefitisthat thestudent,oftenwiththeadvisor,shouldproducesomepublishableresearch.There 1 Even after taking this axiom into account.16 2 Getting Started are a range of approaches to advising that achieve these aims, but they are all based on the strategy of learning while doing. Someadvisors,forexample,settheirstudentsproblemssuchasverifyingaproof inapublishedpaperandseeingwhetheritcanbeappliedtovariantsofthetheorem, thus,ineffect,gettingthestudenttoexplorethelimitsatwhichthetheoremnolonger applies. Another example is to attempt to confirm someone else’s results, by down- loading code or by developing a fresh implementation. The difficulties encountered insucheffortsareafertilesourceofresearchquestions.Otheradvisorsimmediately start their students on activities that are expected to lead to a research publication. It is in this last case that the model of advising as apprenticeship is most evident. Typically, in the early stages the advisor specifies each small step the student should take: running a certain experiment, identifying a suitable source of data, searching the literature to resolve a particular question, or writing one small section ofaproposedpaper.Asstudentsmatureintoresearchers,theybecomemoreindepen- dent,oftenbyanticipatingwhattheiradvisorswillask,whileadvisorsgraduallyleave morespacefortheirstudentstoassertthisindependence.Overtime,therelationship becomes one of guidance rather than management. The trade-offs implicit in such a relationship are complex. One is the question of authorship of work the student has undertaken, as discussed in Chap.17. Another is the degree of independence. Advisors often believe that their students are either demanding or overconfident; students, on the other hand, can feel either confined by excessive control or at a loss due to being expected to undertake tasks without assistance. The needs of students who are working more or less alone may be very different to those of students who are part of an extended research group. Anareawheretheadvisor’sexpertiseiscriticalisinscopingtheproject.Itneeds tostandsufficientlyalonefromothercurrentwork,yetberelevanttoagroup’swider activities.Itshouldbeopenenoughtoallowinnovationandfreedom,yethaveagood likelihood of success. It should be close enough to the advisor’s core expertise to allow the advisor to verify that the work is sufficiently novel, and to verify that the appropriate literature has been thoroughly explored. The fact that an advisor finds a topic interesting does not by itself justify asking a student to work on it. Likewise, a student who is keen on a topic must consider whether competent supervision is available in that area. Advisors can be busy people. Prepare for your meetings—bring tables of results or lists of questions, for example. Be honest; if you are trying to convince your advisorthatyouhavecompletedsomeparticularpieceofwork,thentheworkshould have been done. Advisors are not fools. Saying that you have been reading for a week sounds like an excuse; and, if it is true, you probably haven’t spent your time effectively. The student–advisor relationship is not only concerned with research training, but is a means for advisors to be involved in research on a particular topic. Thus students and advisors often write papers together. At times, this can be a source of conflict,when,forexample,anadvisorwantsastudenttoworkonapaperwhilethe student wants to make progress on a thesis. On the other hand, the involvement ofStudents and Advisors 17 theadvisor—andtheincentivefortheadvisortotakeanactiverole—meansthatthe research is undertaken as teamwork. OvertheyearsIhavenoticedthatthereareseveralcharacteristicsthataresharedby successfulresearchstudents.First,theyshowawillingnesstoreadwidely,toexplore the field broadly beyond their specific topic, to try things out, and to generally take part in the academic community. Second, they have the enthusiasm to develop their interestinsomearea,andthenaskforadviceonhowthatinterestcanbeturnedintoa thesisproject.Third,theyhavetheabilityandpersistencetoundertakeadetailed(and even gruelling) investigation of a specific facet of a larger topic. Fourth, they take the initiative in terms of what needs to be done and how to present it, and gradually assume responsibility for all aspects of the research. Fifth, they are systematic and organized, and understand the need for rigour, discipline, stringency, quality, and high standards. Sixth, they actively reflect on habits and working practices, and seek to improve themselves and overcome their limitations and knowledge gaps. Seventh,theirwork looksplausible;ithastheformandfeelofhigh-qualitypublished papers. Last, they have the strength to keep working despite some significant failed or unsuccessful activity; in a Ph.D., loss of months of work is not unusual. Note that neither “brilliance” nor “genius” is in this list. Intellectual capacity is important, but many bright people do not become outstanding Ph.D. students— sometimes,becausetheyunderestimatethechallengeofextendedstudy.Indeed,I’ve supervisedseveralstudentswhosepreviousacademicrecordwasuninspiringbutwho nonetheless produced a strong thesis, in particular because they were persistent and resilient enough to pursue their work despite setbacks and obstacles. A “Getting Started” Checklist � Is your proposed topic clearly a research activity? Is it consistent with the aims and purposes of research? � How is your project different from, say, software development, essay writing, or data analysis? � In the context of your project, what are the area, topic, and research question? (How are these concepts distinct from each other?) � Is the project of appropriate scale, with challenges that are a match to your skills andinterests?Isthequestionnarrowenoughtogiveyouconfidencethattheproject is achievable? � Istheprojectdistinctfromotheractive projectsinyour researchgroup?Isitclear that the anticipated outcomes are interesting enough to justify the work? � Is it clear what skills and contributions you bring to the project, and what will be contributed by your advisor? What skills do you need to develop? � What resources are required and how will you obtain them? � What are the likely obstacles to completion, or the greatest difficulties? Do you know how these will be addressed?18 2 Getting Started � Canyouwritedownaroadmap,withmilestones,thatprovidesaclearpathtothe anticipated research outcomes? � Do you and your advisor have an agreed method for working together, with a defined schedule of meetings?Chapter 3 Reading and Reviewing You know, we have little bits of understanding, glimpses, a little bit of light here and there, but there’s a tremendous amount of darkness. Noam Chomsky And diff’ring judgements serve but to declare, That truth lies somewhere, if we knew but where. William Cowper Hope The more that you read, the more things you will know. The more that you learn, the more places you’ll go. Dr. Seuss I Can Read With My Eyes Shut A novice researcher can believe that the doing of research is primarily about investigation—running experiments, developing theory, or doing analysis. With experience, though, researchers discover the importance of developing an under- standing. It has been argued that many experimental researchers do their best work aftertheyhavebeeninafieldforfiveyearsormore,becauseittakestimetoacquire a deep, thorough appreciation of the area, and of existing knowledge and its lim- itations. To acquire this understanding, you need to become an effective reader of research papers. A successful reader can identify the contributions and value of a paper, while recognizing its flaws; and uses critical scrunity to identify the extent to which the flaws in a paper are serious. This reading then informs new work, directly as a source of knowledge and indirectly as a guide to how to produce work that will be appreciated. A particular application of reading, moreover, is to become a reliable 1 referee or thesis examiner. This chapter, then, is about both reading and reviewing. Itisanintroductiontotheelementsofeffectivereading,andisaguidetoreviewing. 1 I’ve used “referee” rather than “reviewer” in this book (but “reviewing” rather than the slightly awkward“refereeing”),butinmyexperiencethetermsareusedtomeanmuchthesamething.Here, though, I am also using “referee” to mean “examiner”, because thesis examination involves many of the same skills as paper reviewing, and an understanding of the reviewing process will help you prepare your thesis. © Springer-Verlag London 2014 19 J. Zobel, Writing for Computer Science, DOI 10.1007/978-1-4471-6639-9_320 3 Reading and Reviewing Reviewing is a central part of the scientific process. Criticism and analysis of paperswrittenbyotherscientistsisthemainmechanismforidentifyinggoodresearch andeliminatingbad,andisarguablyasimportantanactivityasresearchitself.Many people are intimidated by the task of reviewing, perhaps because it is a kind of intellectual test: you have been asked to demonstrate a thorough understanding of someone else’s work. It is also intimidating because it brings responsibility; you wanttoneitherwronglycriticizesolidworknorrecommendthatflawedresearchbe published.Unhelpfully,thequalityofreviewingishighlyvariable—mostresearchers havestoriestotellofgoodworkrejectedwithonlyafewhastywordsofexplanation, or of referees who haven’t read the work at all. Reviewingcanbeachore,butdeservesthesameeffort,care,andethicalstandards as any other research activity. And it has rewards, beyond the gratitude of editors and authors. It can lead you to look at your own work from a fresh perspective, and exposesyoutodifferentkindsoferrororfailureinresearch—theaveragestandardof work submitted for publication is well below that of work that gets published. And, while you may not be asked to referee a paper as part of your research, your own work will be reviewed or examined, and thus this chapter provides a perspective on the standards expected of a submitted paper. Research Literature By the time your research is complete, you need to be confident that you have read and understood all of the scientific literature that has a significant connection to your work. Your reading achieves several aims. It establishes that your work is indeednovelorinnovative;ithelpsyoutounderstandcurrenttheory,discoveries,and debates;itcanidentifynewlinesofquestioningorinvestigation;anditshouldprovide alternative perspectives on your work. This reading will ultimately be summarized in the background sections and the discussions of related work in your write-ups. The literature on which your work will rely is usually expected to be papers that have been refereed and published in a reputable venue, theses that have been under- takenandexaminedatareputableinstitution,andbooksthatarebasedontheinforma- tionpresentedinrefereedthesesandbooks.Thesearethedocumentsthatareaccepted bytheresearchcommunityasasourceofknowledge;indeed,theycanberegardedas beingtheentiretyofourscientificknowledge.Theliteraturedoesnotincludeprimary sourcessuchaslabnotebooks,responsestoasurvey,oroutputsfromanexperiment. Whattheselackisinterpretationofthecontentsinlightofaspecifichypothesis.Other literature—newsarticles,sciencemagazines,Wikipediapages,ordocumentation,for example—mayalertyoutotheexistenceofreputablework,butisrarelyworthciting. That is, your learning may be built on a wider literature, but the arguments in your write-up should be based on knowledge that is from a refereed source. A thorough search of the literature can easily lead to discovery of hundreds of potentially relevant papers. However, papers are not textbooks, and should not be treatedastextbooks.Aresearcherreadingapaperisnotstudyingforanexam;there