Taking a Deep Breath Before Jumping into

Download PDF
0 downloads 0 Views 218KB Size Report
Comment
design; literature review; research problem; computer science; eomputer engineering ..... including learning curve, GUI support, documentation, on-line help, etc. ..... Formatting the manuscript as per the journal or conference style takes ...
Published by IEEE Published version is available at http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=7079094 DOI: 10.1109/ACCT.2015.93

Taking a Deep Breath Before Jumping into Research in Computer Science and Engineering Harshadkumar B. Prajapati

Vipul K. Dabhi

C. K. Bhensdadia

Department of Information Technology Dharmsinh Desai University Nadiad, INDIA e-mail: [email protected], [email protected]

Department of Information Technology Dharmsinh Desai University Nadiad, INDIA e-mail: [email protected]

Department of Computer Engineering Dharmsinh Desai University Nadiad, INDIA e-mail: [email protected]

Abstract—Research has substantially increased in modern times due to availability and search facility of literature sources through Internet. Carrying out research and proving that it is done is a necessity for excellent academic career and for satisfying requirements of Master or PhD degrees in most universities. There is no recipe book for producing the research with desired taste and quantity; however, general methodology does exist. This paper presents crucial parts concerning research process in Computer Science and Engineering (CSE). The paper discusses the important parts of research methodology in CSE including research problem formulation, literature review, research design, and research communication. The paper presents each phase of the research process with essential ingredients; moreover, wherever required, the paper also provides pointers for further reading. The presented discussion is in the context of research applicable in CSE field. Therefore, we expect that this paper would become a good starting point for newbie researchers in CSE. Keywords—research; research methodology; research design; literature review; research problem; computer science; eomputer engineering

I. INTRODUCTION Research is essential to cope up with the changing climate conditions, to enhance life style of human kind, to do progress of a country, and so on. In rapidly changing era of machines, a computing machine, i.e. a computer, has become a helping hand of human kind; for example, the computer that existed only in universities or research organizations during its realization has now come, as part of the cell phone, in a pocket of a common person. This change has become possible due to continuous research and development of improved devices and new technology. Though the research outcome of academic research might not help the human kind directly, its realization in form of technology by manufactures does. Computer Science and Engineering (CSE) has much broader applicability

than anticipated at the time of the conception of the computer. The computer and computing are used in almost all other fields of science and engineering. As in research, something new is to be produced or proposed, knowing how to do research is very important. Research plays a major role for recognition of a university and for the progress of an individual academician. Moreover, higher academic degrees such as Master or PhD in Science and Engineering require that the candidate go through research process and produce a thesis. Research involves generating new knowledge or doing creative work, for which no cookbook exists. The research methodology [1] provides us a systematic way of doing the research. The general steps of research methodology are following: define research problem, review the literature, formulate hypothesis, design research, collect data, analyze data, and interpret and report [1]. In CSE, the research generally involves producing novel algorithm, approach, technique, metric, architecture, protocol, data structure, or concept representation and processing. The research in CSE may demand various skills such as writing programs, debugging programs, troubleshooting problems, and testing software. Generally, before we start any work, we anticipate what is expected from our work [2]. Analogously, before carrying out research work, the researcher must understand what is expected from the research work and what is the way to do it. The most effective way of learning research methodology is through experience. For beginners or novice researchers, understanding the research methodology and related concepts is essential before jumping into research. The research work started without following research methodology may result into failure of the work or excessive delay in completing the research work. Thus, understanding of research methodology and following it systematically is always better than blindly doing trial and error. The

understanding of research methodology can help in producing good results and can reduce the time spent in finishing the research work on a particular topic; the saved time otherwise could be accumulated for solving other important research problems. Therefore, research methodology should not be taken as burden, rather it should be learned with interest. The steps of research methodology help a lot at various stages during research, and can help in producing significant and impactful results. The work in [3] presents scientific method of new knowledge development and discusses what is computer science? with focus on the relation of computer science (CS) with science. Moreover, the work also discusses scientific methods of CS. The work in [4] provides brief overview of research methodology and discusses characteristics of qualitative research and quantitative research. The work in [5] presents the framework of research and shows that the system development can be used as research methodology. The book [1] by Kothari details research methodology with focus on data, statistical analysis, and hypothesis testing. However, research in CSE involves a crucial step of producing a new artifact, on which experiments are later executed to collect data. The work in [6] presents status of research in Computer Science (CS), containing examination of 628 papers published between 1995 and 1999 in 13 leading research journals of CS, covering various questions concerning research such as what topics, research approaches, and research methods are addressed by the researchers in the published papers. In this paper, we first briefly introduce research and research methodology (in Section II) with focus on the characteristics of research, research approaches, and research process. Next, we present the characteristics of a research problem, and discuss research problem formulation and its refinement (in Section III). Next, we discuss the importance of the literature review and present the process of carrying out the literature review (in Section IV). The literature review helps the researchers in putting their work on a solid foundation. If a researcher wants to claim that whatever is being presented is novel, the researcher needs to show that the concerning gaps do exist in the works done by others. Next, we present research in action containing design, evaluation, and validation of research (in Section V). Next, we discuss research communication, which covers writing a research paper, getting a research a paper accepted, and research publication (in Section VI). Finally, we present the conclusions and further directions (in Section VII).

II. INTRODUCTION TO RESEARCH AND RESEARCH METHODOLOGY Research is defined as a systematic inquiry or search for new knowledge on a selected topic [1], [7]. A research problem generally involves producing unknown facts from known facts. To systematically solve a research problem requires following of wellestablished steps, which is called research methodology. This section presents the essential characteristics of research, the research approaches, and the process of carrying out research. A. Characteristics of a Good Research A good research possesses following essential characteristics [1]. Systematic: A good research is performed with the specified steps in predetermined sequence according to a specific set of rules. • Logical: A good research is headed by the rules of logical reasoning, e.g., induction and deduction. • Empirical: A good research should be based on experiments and observations, which should be related to real situations and real data. • Replicable: A good research should allow other researchers to replicate the study to verify or reproduce the results. In logic reasoning [8], deductive reasoning works from general observations to specific observations, which is called top-down approach. In this reasoning, conclusion follows from facts. The inductive reasoning works from specific observations to broader generalizations and theories, which is called bottom-up approach. As this approach generates a theory or generalized facts based on specific results, it involves degree of uncertainty. •

B. Research Approaches Depending upon the nature of a research problem, a research approach guides about using particular methods or techniques [1] in conducting the research. We emphasize on two approaches depending upon the kind of parameters and data involved in the research problem: qualitative and quantitative approaches [9]. 1) Qualitative Approach: Qualitative approach [10], [4] deals with analysis of characteristics of some phenomena, or concept, or entity, or the approach deals with an aim of establishing relation among associated variables. In this approach, variables are unknown and the approach is used to discover the variables of interest; for example, it can be used to discover what gaps exist in the work on some topic. This approach mainly uses non-numeric

techniques such as clarification, understanding, discovery, analogy, analysis, and synthesis. In this approach, the form of the final result or conclusion is “This is a new way of solving this problem” or “These are the parameters not addressed by the existing work” or “This is the problem not yet solved by others”. 2) Quantitative Approach: Quantitative approach [4] contains analysis of data, either generated through experimental approach or simulation approach. The approach generally involves use of statistical, formula, or numerical analysis to generate results. The main focus of this approach is on the data related to known variables with focus on analysis, causal determination, prediction, and generalization of findings. In this approach, the form of the final result or conclusion is “This solution performs N% better than the current best solution(s)”. C. Research Process The generalized process of carrying out research is shown in Fig. 1. The research process involves various tasks or operations at different stages of the research, in which different research methods or techniques [1] can be applied. The research methods in CSE include writing programs, developing systems, developing software architecture, developing hardware architecture or design, designing data structures, developing ontologies, developing knowledge bases, proving theorem, designing new measure, analyzing time/space complexity, analyzing strengths and weaknesses of existing work, synthesizing the approaches of existing work, and developing a testbed. This paper discusses each major step of the research process in a separate section.

Fig. 1. Generalized process of carrying out research

III. RESEARCH PROBLEM FORMULATION The most important part in research is identifying the research problem. If a research problem is not available at hand, e.g., for a research student or a newbie researcher, then finding the research problem becomes the most difficult part. In a project driven research work, a student may not need to worry about identifying the problem, as, normally, the guide or supervisor communicates the research problem related to the project to the student. However, in general, Master or PhD students must undergo the process of how to find out a research problem. The process of identifying a research problem helps them in their future research endeavors. This section addresses the formulation of a research problem. A. Characteristics of a Research Problem Any problem in the world is related to either optimization or control, and for the research problem also same is true. Following are essential characteristics of a research problem. Many journals decide about submitted manuscripts based on whether the addressed problem in the manuscript possesses these characteristics. •

Valid (logical): The research problem should be valid with reference to general rules of logic and should not violate context or topic or domain specific rules or concepts.

•

Original: The addressed problem should be original in some way, depending upon intended use of research work, e.g., in a conference paper, or in a journal article, or in a PhD thesis. For example, originality can be achieved by investigating the parameters other than the investigated by others, considering different or broad scenarios, and targeting different objectives.

•

Sufficient: The research problem should be sufficient in breadth and depth to justify the place of the presentation of the research results.

•

Impact: The research problem should have a broad impact on the field or topic that is being investigated.

The most important characteristics of a research problem are originality and sufficiency. There are different ways of achieving originality in a problem and/or its solution. The readers are directed to the work in [11], [12] for further readings on originality. Regarding sufficiency, in experiment based research work, for example, a conference paper may demand around three to four months work, a mediocre journal article may demand around a year’s work, and a PhD

thesis may require work consisting two journal publications, all excluding writing time. B. Search for a Research Problem

measures that can be used to evaluate the proposed solution. In step 3, the researcher should define a feasible problem based on the results of step 2. This step consists of description of the methodology used for solving the problem and expected results and their significance. For a large research problem, e.g. in PhD research, even at the end of step 3, further refinement of the problem would be needed, which is presented in next subsection. C. Refining The Research Problem

Fig. 2. Moving from the Subject to Research Problem

The search direction for a research problem is shown in Fig. 2. Search for a research problem is started with searching and reading of the preliminary literature in the subject area. At this stage, a general topic or topics of interest are decided. While choosing a topic, the researcher should be cautious about the type of the problem [13], for example, the topic should not be not too narrow and not too wide. We present three major steps of identifying the research problem, which are as follows. •

Step 1: Identify the apparent or seeming problem.

•

Step 2: Study and analyze the causes of the problem.

•

Step 3: Define the feasible problem.

In step 1, there are two possibilities: (1) new problem and (2) existing problem. A problem is new if a researcher discovered it, i.e., it has not been mentioned in the literature. For a new problem, the researcher needs to pay much attention along with analytical and background study and need to take experts’ suggestions, as the new problem could be messy, ambiguous, or vague. A problem is an existing problem, if it has been mentioned in the literature. A researcher can find the existing problem from the further work section of the published research papers. For both the types of the problem, the researchers need to apply their own creativity and critical thinking. For the existing problem, the researcher needs to mold the problem in such a way that its solution and approach would generate impact on other such problems and their solutions. In step 2, the researcher should try to find out the causes of the problem, issues involved, and behavior of associated entities. Moreover, the researcher should also try to find out parameters that need to be considered in the definition of the problem. The researcher should try to find out relations among those parameters. Furthermore, the researcher should try to find out

Refinement of the research problem is generally done after exhaustive literature review is done. By doing exhaustive literature review, a researcher comes to know about what is already done and where do the gaps exist. The refinement of the problem includes (1) formulating the hypothesis, (2) detailing the research questions with focus on objectives, and (3) deciding the scope of the research problem with focus on time availability and feasibility of the solution. The refinement of the research problem should be detailed at such a level that the planning and ordering of remaining phases of the research work can be done. In refining the research problem, creativity and brainstorming help a lot. The creativity refers to the ability to look at the problem with many associated attributes and with different viewpoints. Thus, the creativity helps to realize the problem. Creativity can help in making the research problem original. The strength of the hypothesis determines the impact of the research problem. Scope of the research problem determines whether the problem or the refinement of the problem is sufficient for the intended research work.

We provide a few important pointers or references that would become very useful in developing a research problem. The work in [14] provides some guidelines related to finding a research problem. Turning a topic into research question is addressed in [15]. The work in [15] presents a framework for developing a research problem, which is an essential guide to beginners. IV. KNOWING THE LITERATURE The main focus of a literature review is to perform an objective study, produce a thorough summary, and do critical analysis of the relevant literature on the topic being studied [16, 17]. The important characteristic of a research is that it should be novel. Therefore, researchers need to make sure that the problem on which they are working will produce new results or knowledge. However, to prove or to show that the results are new, the researcher needs to study and survey existing knowledge. Knowing the literature helps in two ways: to understand the concepts and to show the novelty of the

attempted work. In this section, we discuss various aspects related to the literature review. A. Why to do Literature Review To a novice researcher, a question could come that when we are doing our own research work, why we need to worry about what others have done. However, doing a research work without knowing the state-of-theart is a wrong practice. Starting the development or implementation work in hurry, without knowing the literature, may lead to the work that has already been done by others and you are not aware of that. Thus, the literature review becomes an inherent part of a research work, presented in either a paper or a thesis (or dissertation). Table I shows various purposes of literature review. TABLE I.

PURPOSES OF LITERATURE REVIEW. ADAPTED FROM [16], [18], [19]

Research Problem Related 1) Delimiting (creating boundary of) the research problem; 2) Seeking new lines of inquiry; 3) Establishing the context of the topic or problem; 4) Rationalizing the significance of the problem; 5) Identifying application areas in which similar research problems exist. Fundamental Concepts Related 1) Knowing background theory; 2) Gaining methodological insights; 3) Enhancing and acquiring the subject vocabulary; 4) Understanding the structure of the subject. Solution Related 1) Avoiding fruitless approaches; 2) Identifying recommendations for further research; 3) Distinguishing what has already been done from what needs to be done; 4) Discovering important factors relevant to the topic; 5) Synthesizing and gaining a new perspective; 6) Identifying relationships between ideas and practices; 7) Relating ideas and theory to applications; 8) Identifying benchmark problems (input and scenario); 9) Identifying performance measures; 10) Identifying related/available tools used by practioners.

The systematic literature review is generally used to understand current knowledge and to find out the gaps that exist, which can provide directions for further research. The literature review related to research in CSE can encompass the review of (1) solution approaches, (2) methodology (experiment methods), (3) tools/environments, and (4) measures or metrics. The solution approaches in CSE cover algorithm, method, technique, data structure, rules, information representation, etc. The researchers need to choose appropriate tools depending upon their technical requirements. The main decisive factor is the complexity involved in extending the available tool for desired research purposes. The other factors for the decision about tools include supported parameters of study, supported output measures or metrics, and usability, including learning curve, GUI support, documentation, on-line help, etc. C. Process of Literature Review The process of literature review, shown in Fig. 3, involves six major steps. This is a general process used in both traditional and systematic literature reviews. The traditional approach of literature review selects topics that are to be reviewed in the first step: problem formulation. The selected topics should be focused and narrow. However, the systematic approach formulates the problem, which generally involves specific questions to be addressed in the survey.

Other uses 1) Identifying standard journals or conferences; 2) Identifying related projects; 3) Identifying researchers who contributed most in the literature; 4) Identifying possible reviewers for evaluating the research work.

B. Types of Literature Review Based on the objective of performing literature review, it can be categorized into two major types: (1) traditional and (2) systematic. The traditional literature review studies, critics, and summarizes the relevant body of literature. The systematic literature review [20] uses a well defined approach, and studies the available literature exhaustively. The traditional literature review can be used in writing the related work section in a research article, whereas the systematic literature review can be used in preparing and writing a survey article.

Fig. 3. Process of Literature Review

In collection of sources step, generally, sources of last ten years are considered. In CSE, the sources may include conference papers, journal papers, technical reports, books, and web materials. The work presented in the recently available sources would be sometime older; for example, the work presented in a conference paper would be around 4 to 6 months old, in a journal article it would be 1 year, and in a book it would be around 2 years old. The technical reports become available earlier and they contain details as well. Web materials also become available earlier; however, as web

materials do not have any formal publication process, their authenticity is questionable. While searching the literature, a researcher should keep record of the search keywords already tried with the date on which search was done. Search results would return many articles or papers, but not all the articles or papers might be important. The citation counts of papers, which can be found from Google Scholar search engine, may be used for estimating the importance of a particular paper. As earlier published work would have higher citation counts, it is better to decide the importance based on total citation counts of the paper divided by the number of years since the paper’s publication. In CSE field, widely accepted article databases include from the publishers such as IEEE [21], ACM [22], Springer [23], Elsevier [24], and Wiley [25]. A researcher needs to subscribe, through either personal or university subscription, to these digital libraries to access the research papers or articles. If access to such databases is not available, however, the initial version of a paper can be retrieved from a pre-print server, if the authors have uploaded it. A researcher can also try to search the paper on author’s home page. Even, if it is not found there, a request to the corresponding author through email can be sent, which may work out as the author would be interested in someone take interest in his or her work and give citation to the work. Not each available source will be useful for carrying out research work, or maybe due to time constraint, we might not have enough time to read each one fully. Evaluation of collected sources helps in reducing the materials to read and to decide which sources should be paid much attention and for which sources superfluous reading would be sufficient. The evaluation of sources can be done based on the content, context, methodology, author, and relevance of the presented work [26]. Interpretation of the studied literature helps in generating description, classification, categorization, and analysis of literature. The synthesis of the results shows major issues addressed and solution approaches tried, and suggests gaps that need to be filled. D. Writing Literature Review For writing literature review, two important concerns are (1) what is to be written and (2) how is it to be written. For (1), the content could be descriptive or graphical/tabular. When writing literature review in the related work section in a research article, a researcher needs to include the seminal (landmark) work, the most recent work, and progress made between seminal and the recent work. For (2) a researcher needs to use appropriate tense. We have two choices of tenses for writing the statements related to literature review: (1) present perfect tense and (2) past tense. Present perfect tense is used to write about past-to-present progress

made on a topic. Past tense is used to write about findings made by a particular author. In writing of descriptive literature review, a researcher needs to show the relationships between different related work. In case of writing literature review in the related work section in a research article, the researchers need to show how the existing work relates with their proposed work. V. RESEARCH IN ACTION In CSE, Research in Action phase could involve many activities to be done. In this phase, a researcher feels that the work on the research problem is progressing. This section presents three main steps of the research in action phase: design the research, evaluate the research, and validate the research. A. Design the Research The design of the research includes two major steps: (1) propose new work and (2) design the workloads. 1) Propose new work In CSE, there are many domains such as Artificial Intelligence, Software Engineering, Computer Hardware, Network, Distributed Computing, and many more. However, any research in CSE produces one or more of following: new algorithm, new architecture/software design, new Hardware/components, new metric, new approach, or new concepts (logic, semantics, data structure, etc). There are three ways of proposing a new algorithm: (1) propose a very new algorithm containing creative idea, (2) apply a technique or algorithm that was invented for one domain into another domain, (3) combine more than one technique to form a new technique to optimize an existing algorithm. Similarly, for development of new architecture/software [5], hardware/components, and concepts these three ways can be applied. Proposing a new work involves a lot of brainstorming. As activities related to brain are involved, there is no guarantee of outcome and the result is not time bound. However, use of mind tools can provide substantial input in this process. For example, SCAMPER [27], which stands for Substitute, Combine, Adapt, Modify, Put to another use, Eliminate, Reverse, is an appropriate mind tool to introduce novelty in the existing work. 2) Design the Workloads In design of the workloads, various parameters of the study are varied in appropriate proportion; and to carryout it principles of experiment design are used. The principles of experiment design are (1) randomization, (2) replication, and (3) local control. The workloads to be used for performance evaluation of the proposed work depend on the field of study and type of the problem

being addressed. However, irrespective of domain, the types of workloads include test, real, and synthetic workloads [28]. Properly designed workloads allow observing effect of independent parameters on dependent parameters. The best way to design the workloads is to collect the benchmark problems (input and scenario) considered by other researchers. For example, the work in [29] presents how benchmarking and comparison of task graph (DAG) scheduling algorithms can be done; the work in [30] provides various scientific workflows that can be used for evaluation of workflow scheduling algorithms. B. Evaluate the Research In CSE, the kinds of evaluation of the proposed work could be one or many of the following: (1) theoretical, (2) simulation, (3) experimental, and (4) implementation. The evaluation of the research includes four major steps (1) implement the proposed work (e.g., algorithm), (2) configure environment for designed workload scenarios (e.g., script, input data files, and run-time parameters), (3) execute the proposed work and gather the outcomes of the experiments (e.g., through log files, output files, instrumented code, etc.), and (4) analyze the results. In evaluation of the research, the proposed work is compared with other recent best algorithms. The evaluation metrics [31] or measures depend on the type of study and the domain of the problem. The evaluation measures fall in following broad categories: performance (end result), solution complexity (space and time), and impact to theory/facts (whether conclusions are generalized or specialized). The evaluation of the research should lead to new facts. The researcher should search for scenarios in which the proposed solution provides best results and scenarios in which the solution provides worst or negative results. The researcher should try to understand and analyze the causes of the best results and negative results, which may produce new fact or gaps that still need to be filled. C. Validate the Research For validating the research, a researcher needs to provide enough evidences to convince other researchers about the validity of the research. The validation can be experimental [32] or theoretical. The experimental validation includes comparing the results of the proposed research with the current best results available in the literature. The theoretical validation can include providing mathematical or analytical proof of the proposed research. The theoretical validation may consider providing proof of new approach or concepts by considering various cases, i.e., proof by cases. Publishing the research work in a reputed journal also establishes the validation of the proposed work.

VI. RESEARCH COMMUNICATION Research communication includes three major steps: (1) writing a research paper, (2) getting the research paper accepted, and (3) research publication. A. Writing a research paper 1) Structure of a Research Paper and Essential Aspects of its Sections The general structure of the body of a research paper in Science follows IMRAD (Introduction, Methods, Results, and Discussion) structure, which is also true for CSE. The work in [33] discusses and compares Hourglass model and King’s model of the structure of a scientific paper. We provide essential aspects of each section of a scientific paper. Title: The title of a paper should be clear, concise, thought provoking, and meaningful. While deciding about the title, we should imagine how someone would search for our paper. Abstract: The abstract of the searched paper is read by around 87% [34] of the people who look at our paper. Therefore, while writing the abstract, it is important to remember that we have to sell our paper [34]. The abstract should focus on (1) problem context, usually one or two statements, (2) the problem itself, (3) why to attempt this problem? (why it is interesting/challenging/new), (4) how you achieve solution (methodology in brief), (5) what your solution achieves (major results in brief), and (6) significance of the work (what follows from your solution/results). The word limit for the abstract is generally 100 to 250 words, but in CSE the limit of around 150 words is most common. Index Terms: Index terms are used to classify our paper. They are used by search engines or indexing databases for creating index in order to facilitate searching of research papers by keywords. While writing index terms we have to remember that using which keywords, in searching through search engine, others would arrive at our paper. Introduction: The introduction of the paper is read by around 43% [34] of the people who look at our paper. The introduction section explains the background and significance of the paper. The introduction section starts with a short introduction to the research area or topic or problem. The second paragraph should focus on the problem definition and motivations. This paragraph can include why it is useful and challenging to solve the problem. This paragraph can discuss the nature of the problem and the complexity of the problem; moreover, it can also highlight the significance of the problem. The

third paragraph should focus on the applied methods, major results, and their validation. If the discussion of the methods, the results, and the validation is lengthy, the third paragraph can be split into two. In the Introduction section, there is no need to write detailed results, for which the appropriate place is the Results section. However, the paragraph should present generalized result, which could discuss broad view, trend lines, or causal relationships of the results. The section can be ended with a summary of the organization of the paper, which is generally found in papers of conferences and journals in CSE. Method: The method section of a paper is read by around 12% [34] of the people who look at our paper. It should include the details so that other can reproduce the same result by following what you mentioned in this section. It generally focuses on how you carried out your research. The methodology should discuss about procedures, methods, experiments, processes, equipment, data structures, algorithms, etc. Moreover, it can also discuss about performance measures, benchmark problems tested, and situations/scenarios considered. Result: This section generally contains results in form of graphs or plots or tables and discusses the presented results. In CSE, the results generally discuss effect of parameters on the performance of the proposed work. The performance measures could be related to time/space complexity, reliability, scalability, and efficiency of the proposed work. We must pay much attention to the results section, as most reviewers read this section first. Discussion: This section is also important, as reviewers read it along with the result section. For reviewers, the results and discussion sections reveal the strength and impact of the proposed work quantitatively. Both the results and discussion sections play major role in deciding the acceptance of the paper. The discussion section should be organized such that the presented information flows from more specific to more general, i.e., from detailed results to generalized results to generalized conclusions. This section should answer and discuss the questions mentioned in the Introduction section, and it should discuss the strength of the proposed work (results). In addition, the discussion section should mention the limitations of the work and should discuss unanswered or unaddressed related questions, which can indicate future directions to other researchers. Conclusion (or Summary): The conclusion section of the paper is read by around 55% [34] of the people who look at our paper. The conclusion should present (1) interpretation of presented work with focus on pros and cons in terms of the performance, (2) rejected

alternatives in the proposed work, (3) limitations of the proposed solution, and (4) suggestions for “Future Work”. In CSE research, the proposed work generally contains a novel approach for solving the research problem. Therefore, the approach itself and its implementation need a separate section or two. If the discussion on the method is not lengthy, then the Method and Results sections can be merged and can be placed under the Evaluation section. We direct the readers to [35] for example based understanding of the structure of a scientific paper. The work in [36] discusses about how to write research articles for computing and engineering disciplines. The work in [37] surveys schematic structure of research articles in Computer Science field. 2) Scientific writing Bunch of sources exist related to writing for publication work, which is referred using different terms such as technical writing, scientific writing, journal article writing, and research writing. A short style guide for an effective writing is [38]. An effective writing is concise; therefore, a researcher needs to acquire skill of contracting the writing [39]. A researcher has to know which tenses are appropriate for writing different sections of the paper. Paraphrasing skills, to stay away from plagiarism [40], is also essential for scientific writing. The quality of writing improves by experience, not by just reading some books or articles. Therefore, a researcher needs to write a lot of papers, to review others’ work, to understand phrases or writing style used by the authors of reputed journal articles, such as published by IEEE transactions, Springer, Elsevier, and others. The best and fast way of learning an effective writing style could be to pick up an effective, highly cited, and closely related paper on your research topic and to use it as a model for structuring the content as well as phrases. We provide one example of it; for example, on the topic of scheduling in heterogeneous environment the work in [41] has been used as the model for the work in [42]. 3) Writing tools Formatting the manuscript as per the journal or conference style takes valuable time of a researcher. Writing tools are very useful to save our formatting time. In CSE, most journals and conferences provide templates for LaTeX and MS-Word. LaTeX can save our valuable time in preparing a manuscript, as the formatting instructions are included in class file (.cls) and bibliography style (.bst) file. LaTeX is very quick in formatting a manuscript for various journal or conference styles; whereas, as MS-Word is WYSIWYG (What You Say is What You Get) editor, we need to

format the content or manipulate individual styles that are being used in the paper. Though, we can save time in formatting the references using bibtex4word [43] macro along with LaTeX installation, such as MikTeX [44]. Microsoft Office 2007 onward, the Microsoft Word supports the feature of citing sources and generating references, which saves our time that is otherwise spent in renumbering and reordering the references. It is always better to produce the list of references using some tool to avoid typos and manual formatting. We can collect bibliography information of each literature source either from citeseer [45] or Google Scholar [46] or from publisher’s web-site, e.g., IEEE and ACM provide bibliography in various formats such as BibTeX, EndNote, RefMan, and Refworks. Reference manager tools, e.g. JabRef [47], are also available for editing bibliography. Another important requirement is to have high quality diagrams or images in a manuscript. OpenOffice Draw is a generalized drawing tool supporting various geometric shapes with different styles. A specialized diagram editor tool can also be used to prepare illustrations of high print quality. For example, using Dia [48], we can prepare illustrations related to engineering. Dia supports almost all categories, including flowchart, UML, ER, Networking, Database, and many more, of diagrams related to CSE. Moreover, Dia supports exporting the prepared illustration into various bitmap and vector graphics formats. Using vector graphics formats such as .eps or .pdf, LaTeX can produce high quality images in a manuscript. 4) Journal or Conference paper? A conference paper focuses on the research problem, the solution approach, and the results, whereas a journal paper focuses on the research problem, solution approach, the results, and the validation of the approach. Decision about publishing a conference paper or journal paper depends upon many factors [49]. An important concern is that in conference submission, the author does not get a second chance of submission and there is a deadline for submission, whereas in a journal submission there is no deadline for submission and author may get second chance of submission by improving the manuscript with incorporating the major or minor revisions asked by the journal. In a conference, unlike in journal, researchers get opportunity of communicating their work and getting comments from audience and session chairs during oral or poster presentations; moreover, the researchers also get opportunity of discussing their work with other researchers, during free or lunch time. In CSE, the conferences ask for papers of 6 to 10 pages whereas for most journals the limit is 14 to 20 pages. The limit of 18 pages is most common in IEEE

and Elsevier journals. For a survey paper, generally the limit is relaxed; for example, ACM computing survey papers can be up to 50 pages. Before submitting to a journal, a researcher must find out average time taken by the journal for giving the decision, as journals usually take a longer time in decision due to rigorous review process. In CSE, reputed journals take around 6 to 8 months time in giving the review decision. However, if the manuscript is out of scope, or not of significant value, or is not meeting journal’s standard, the manuscript is rejected administratively by Editor-inChief (EIC), in which case the manuscript does not undergo the review process. B. Getting a Research Paper Accepted Researchers must know based on which criteria their submitted papers are reviewed. Generally, review of a paper in a conference submission is not as stringent as that is done in journal submission. In reputed journals, the emphasis in review process is on turning a manuscript into a high quality research article, for which suggestions provided by reviewers, and may be by Editor-in-Chief or Associate Editor, are very useful. Though reviewers do not contribute directly in writing a manuscript, but incorporating their suggestions or comments into the manuscript can improve the quality of the manuscript substantially. For a conference submission, the author does not get such second chance of submission if the paper gets rejected. However, in journal submission, the author has an opportunity of revising a research paper if the reviewers found it to be reconsidered after major or minor revision. If major or minor revisions are suggested, then by most journals in CSE the time of around 8 weeks or 2 weeks, respectively, is given to authors for revising and resubmitting the manuscript. For most journals the review criteria are related to impact, originality, and significance. The criteria could be (1) reader interest, (2) content, and (3) presentation. The reader interest focuses on (i) the type of the manuscript, whether it is of type research, application, case study, or experience report, and (ii) whether the submitted manuscript is relevant to the journal. The content focuses on (i) how the manuscript advances the field or how the manuscript adds new knowledge to the existing literature, and (ii) whether the manuscript is scientifically sound. The presentation includes (i) appropriateness about title, abstract, and keywords, (ii) quality of introduction section and conclusion section, (iii) organization of the manuscript with focus on the length of the manuscript, (iv) appropriateness of references, and (v) readability of the manuscript focusing on style, grammar, and effectiveness of the text.

C. Research Publication After the paper is accepted at conference, the committee will asks the authors to submit camera ready copy (CRC) and expect the presentation of the selected papers. If the paper gets accepted in a journal, generally, the corresponding author is asked to submit source files (.doc or .tex), image files, bib files, and any other files that are needed to build production quality manuscript. Most journals ask the corresponding author to go through the galley proof for any typos or corrections and to approve it, after which an online early version of the selected paper can be published by the publisher. VII. CONCLUSIONS This paper has discussed the importance of research methodology and presented minimal number of concepts that beginners must know before jumping into research. The paper has presented research process and discussed it with emphasis on research problem formulation and its refinement, performing literature review, and carrying out research. The paper also discussed the characteristics of a research problem and its solution, which researchers can use to evaluate their own work based on these characteristics. The paper not only discussed the characteristics but also showed how these characteristics can be achieved in a research problem. Moreover, the paper provided directions or pointers at various places in this paper. The understanding and practice of presented suggestions could help in producing research results timely. Future direction includes how comparison of two or more works can be done using statistical tests, specifically focusing on the significance of the tests, understanding of various tests, selection of test based on situation, and taking a decision on when to apply which test.

[8]

[9]

[10]

[11]

[12] [13]

[14]

[15]

[16] [17]

[18]

[19] [20]

[21] [22] [23] [24]

REFERENCES [1] [2] [3]

[4]

[5]

[6]

[7]

C. R. Kothari, Research methodology: methods and techniques. New Age International, 2004. S. R. Covey, The 7 habits of highly effective people. Simon & Schuster, 1991. G. Dodig-Crnkovic, “Scientific methods in computer science,” in Proceedings of the Conference for the Promotion of Research in IT at New Universities and at University Colleges in Sweden, Skövde, Suecia, 2002, pp. 126–130. P. Goubil-Gambrell, “What do practitioners need to know about research methodology?” in Professional Communication Conference, 1991. IPCC’91. Proceedings. The Engineered Communication., International, vol. 1. IEEE, 1991, pp. 243–248. J. F. Nunamaker Jr and M. Chen, “Systems development in information systems research,” in System Sciences, 1990., Proceedings of the Twenty-Third Annual Hawaii International Conference on, vol. 3. IEEE, 1990, pp. 631–640. V. Ramesh, R. L. Glass, and I. Vessey, “Research in computer science: an empirical study,” Journal of systems and software, vol. 70, no. 1, pp. 165–176, 2004. P. D. Leedy and J. E. Ormrod, “Practical research,” Planning and design, vol. 8, 2005.

[25] [26]

[27]

[28] [29]

[30]

[31]

J. H. Gallier, Logic for Computer Science: Foundations of Automatic Theorem Proving. New York, NY, USA: Harper & Row Publishers, Inc., 1985. S. F. Bahari, “Qualitative versus quantitative research strategies: contrasting epistemological and ontological assumptions,” Jurnal Teknologi, vol. 52, no. 1, pp. 17–28, 2012. O. Hazzan, Y. Dubinsky, L. Eidelman, V. Sakhnini, and M. Teif, “Qualitative research in computer science education,” ACM SIGCSE Bulletin, vol. 38, no. 1, pp. 408–412, 2006. E. Phillips, “The concept of quality in the phd,” Quality in PhD Education. Canberra: Centre for Educational Development and Academic Methods (CEDAM), 1993. E. Phillips, D. Pugh et al., How to get a PhD: A handbook for students and their supervisors. McGraw-Hill International, 2010. P. Eades. How to get a PhD in Information Technology. Last accessed on 28 November 2014. [Online]. Available: http://sydney.edu.au/engineering/it/~peter/howtogetphdusyd.pps J. Eisner. How to Find Research Problems. Last accessed on 28 November 2014. [Online]. Available: http://cs.jhu.edu/~jason/advice/how-to-find-research-problems.html T. J. Ellis and Y. Levy, “Framework of problem-based research: A guide for novice researchers on the development of a researchworthy problem,” Informing Science: International Journal of an Emerging Transdiscipline, vol. 11, pp. 17–33, 2008. C. Hart, Doing a literature review: Releasing the social science research imagination. Sage, 1998. P. Cronin, F. Ryan, and M. Coughlan, “Undertaking a literature review: a step-by-step approach,” British Journal of Nursing, vol. 17, no. 1, p. 38, 2008. J. J. Randolph, “A guide to writing the dissertation literature review,” Practical Assessment, Research & Evaluation, vol. 14, no. 13, p. 2, 2009. M. D. Gall, J. P. Gall, and W. R. Borg, “Educational research: An introduction (8th edition),” 2006. S. Keele, “Guidelines for performing systematic literature reviews in software engineering,” Technical report, EBSE Technical Report EBSE-2007-01, Tech. Rep., 2007. IEEE Xplore Digital Library. Last accessed on 28 November 2014. [Online]. Available: http://ieeexplore.ieee.org/Xplore/home.jsp ACM Digital Library. Last accessed on 28 November 2014. [Online]. Available: http://dl.acm.org/ Springer. Last accessed on 28 November 2014. [Online]. Available: http://www.springer.com/ ELSEVIER. Last accessed on 28 November 2014. [Online]. Available: http://www.elsevier.com/ Wiley Online Library. Last accessed on 28 November 2014. [Online]. Available: http://onlinelibrary.wiley.com/ Literature Reviews Student Learning Centre Flinders University. Last accessed on 28 November 2014. [Online]. Available: http://www.flinders.edu.au/slc_files/Documents/Blue%20Guides/Literature%20Review.pdf Mind Tools Creativity Tools. Last accessed on 28 November 2014. [Online]. Available: http://www.mindtools.com/pages/main/newMN_CT.htm R. Jain, The art of computer systems performance analysis. John Wiley & Sons, 2008. Y.-K. Kwok and I. Ahmad, “Benchmarking and comparison of the task graph scheduling algorithms,” Journal of Parallel and Distributed Computing, vol. 59, no. 3, pp. 381–422, 1999. S. Bharathi, A. Chervenak, E. Deelman, G. Mehta, M.-H. Su, and K. Vahi, “Characterization of scientific workflows,” in Workflows in Support of Large-Scale Science, 2008. WORKS 2008. Third Workshop on. IEEE, 2008, pp. 1–10. N. E. Fenton and S. L. Pfleeger, Software metrics: a rigorous and practical approach. PWS Publishing Co., 1998.

[32] M. Zelkowitz and D. Wallace, “Experimental models for validating technology,” Computer, vol. 31, no. 5, pp. 23–31, May 1998. [33] M. Derntl, “Basics of research paper writing and publishing.” [34] (2012) How to write scientific papers KP/JG. Last accessed on 28 November 2014. [Online]. Available: http://www.idt.mdh.se/kurser/ct3340/ht11/Paper-Writing-Publication-1-Overview.pdf [35] U. Khedker. (2009, October) How to Write a Good Paper? Last accessed on 28 November 2014. [Online]. Available: http://www.cse.iitb.ac.in/~uday/soft-copies/writing-a-good-paper.pdf [36] I. Stojmenovic and V. Milutinovic, “How to write research articles in computing and engineering disciplines,” Singidunum journal of applied sciences, vol. 9, no. 1, pp. 42–50, 2012. [37] S. Posteguillo, “The schematic structure of computer science research articles,” English for Specific Purposes, vol. 18, no. 2, pp. 139–160, 1999. [38] W. Strunk, The elements of style. Penguin, 2007. [39] L. Blaxter, C. Hughes, and M. Tight, How to research. McGraw-Hill International, 2010. [40] M. Roig, “Avoiding plagiarism, self-plagiarism, and other questionable writing practices: A guide to ethical writing,” 2011. [41] H. Topcuouglu, S. Hariri, and M.-y. Wu, “Performance-effective and low-complexity task scheduling for heterogeneous

[42]

[43]

[44] [45] [46] [47] [48] [49]

computing,” Parallel and Distributed Systems, IEEE Transactions on, vol. 13, no. 3, pp. 260–274, March 2002. H. Arabnejad and J. Barbosa, “List scheduling algorithm for heterogeneous systems by an optimistic cost table,” Parallel and Distributed Systems, IEEE Transactions on, vol. 25, no. 3, pp. 682– 694, March 2014. M. Brookes. Bibtex4Word - Home. Last accessed on 28 November 2014. [Online]. Available: http://www.ee.ic.ac.uk/hp/staff/dmb/perl/ MiKTeX. Last accessed on 28 November 2014. [Online]. Available: http://miktex.org/ CiteSeerX. Last accessed on 28 November 2014. [Online]. Available: http://citeseerx.ist.psu.edu/ Google Scholar. Last accessed on 28 November 2014. [Online]. Available: http://scholar.google.co.in/ JabRef reference manager. Last accessed on 28 November 2014. [Online]. Available: http://jabref.sourceforge.net/ Dia Diagram Editor. Last accessed on 28 November 2014. [Online]. Available: http://dia-installer.de/ M. Ernst. (2006, December) Choosing a venue: conference or journal? Last accessed on 28 November 2014. [Online]. Available: https://homes.cs.washington.edu/~mernst/advice/conferencesvs-journals.html