TDDD89 Scientific Method

Seminar 2

Seminar 2

Purpose

To practice formulating your own research questions and introduction.
To practice formulating proper academic English.

Reading material

The reading material here pertains to several common themes of final theses. By themes we refer here to the specific sample master's thesis topic areas and method types being read in the different topic groups. Within your teams, choose a theme and the two papers pertaining to that theme based on their applicability to the thesis that you will be working on during the course. Many will find the guidelines provided by Kitchenham, as well as Runeson and Höst to be generally applicable for theses in many industrial settings. However, if you have already read these references earlier or believe that your particular theses will have a different focus than what is targeted by these two guidelines papers, you also have an option to read references relevant for a number of other types of theses.

• You will need to read the introductions (and, where already available, background) of the draft extended thesis plans written by the other 2 or 3 teams in your seminar topic group, as well as one of the following references on how to conduct certain types of studies.
• For those who plan to conduct studies on the effects of software systems in industrial settings (primarily students in themes 1, 3):
  • B. A. Kitchenham, S. L. Pfleeger, L. M. Pickard, and P. W. Jones. "Preliminary guidelines for empirical research in software engineering". IEEE Transactions on Software Engineering, 28(8):721-734, August 2002.
  • P. Runeson and M. Höst. "Guidelines for conducting and reporting case study research in software engineering". Empirical Software Engineering 14(2):131-164, Apr. 2009.
• For those who plan to study usability aspects of software systems (primarily students in theme 3):
  • D. Alonso-Rios et al.: "Usability: a critical analysis and a taxonomy." International Journal of Human- Computer Interaction 26.1 (2009): 53-74.
  • M. Matera, F. Rizzo, and G. T. Carughi, Web Engineering, Chapter on Web Usability: Principles and Evaluation Methods, pp. 143-180. Springer, 2006.
• For those who plan to study Machine Learning topics (primarily students in theme 4):
  • Vanschoren, Joaquin, et al. "Experiment databases." Machine Learning 87.2 (2012): 127-158.
  • Caruana, Rich, and Alexandru Niculescu-Mizil. "An empirical comparison of supervised learning algorithms." Proceedings of the 23rd international conference on Machine learning. ACM, 2006.
• For those who plan to make use of internal code quality evaluations (primarily students in themes 2, 3, 5):
  • Raimund Moser, Witold Pedrycz, and Giancarlo Succi. "A comparative analysis of the efficiency of change metrics and static code attributes for defect prediction." Proceedings of the 30th International Conference on Software engineering (ICSE). ACM, 2008.
  • Sjøberg, Dag IK, et al. "Quantifying the effect of code smells on maintenance effort." IEEE Transactions on Software Engineering 39.8 (2013): 1144-1156
• For those who plan to do experiments, esp. performance evaluations, in their thesis project (primarily theme 6):
  • T. Hoefler, R. Belli: Scientific Benchmarking of Parallel Computing Systems - Twelve ways to tell the masses when reporting performance results. Proc. SC'15, ACM.
• For those who plan to do hardware-, compiler- or systems programming related theses (primarily students in theme 2 and 6):
  • Kuon, Ian, and Jonathan Rose. "Measuring the gap between FPGAs and ASICs." IEEE Transactions on computer-aided design of integrated circuits and systems 26.2 (2007): 203-215.
  • Gilberto Reynoso-Meza et al.: "Controller tuning by means of multi-objective optimization algorithms: A global tuning framework." IEEE Transactions on Control Systems Technology 21.2 (2013): 445-458.
• For those who plan to do security evaluation theses (primarily students in theme 5):
  • Hannes Holm, Mathias Ekstedt, and Dennis Andersson. "Empirical analysis of system-level vulnerability metrics through actual attacks." IEEE Transactions on dependable and secure computing 9.6 (2012): 825-837.
  • Hossain Shahriar and Mohammad Zulkernine. "Mitigating program security vulnerabilities: Approaches and challenges." ACM Computing Surveys (CSUR) 44.3 (2012): 11.
• For those who plan to do theoretical theses:
  • D. Knuth et al.: Mathematical Writing, Stanford University, 1987.
  • F. E. Su: Guidelines for Good Mathematical Writing, Harvey Mudd College, Mathematics department.

Preparations

1. (Individually)

   Read the material specified in the Reading material section above pertaining to seminar 2. Each student needs to read one of the papers given above (for that, each pair team needs to divide the material appropriate for your theses so that you have two different papers to discuss as you come to the seminar).

   Prepare answers for the Questions A-B below.

2. (Per team)

   Starting from your one-page thesis topic outline, write the beginning of your thesis plan (thesis plan introduction):

   Use the appropriate document template for master's theses (i.e., not the one for thesis plans!) at IDA* / ISY from the start, for the sake of practising using it, even though you are writing what is going to become an extended thesis plan and not the thesis ("report") itself.
   ^* Note (2022): This slightly revised variant of the IDA LaTeX template available on Overleaf can be used as an alternative.

   The most important part of your thesis plan is defining research questions. Try out a few tentative questions and write them all down as part of your introduction. Later, you will get to remove and revise them. Make sure that the questions are somehow possible to answer and relate to the effects of that which you expect to produce during the thesis project. Take inspiration from the material that you are to read as preparations. Do not fear writing down too many questions at the start as you will have ample opportunities to revise them and drop some of them later.

   Aim to write 2 pages plus references as appropriate, not more. (If you already have some text about background description beyond what is necessary to understand the research questions, put it into Chapter 2 Background. Remove all unused chapters copied from the template.)

Submission: see below. Deadline: Tuesday in the third week.

Preparation Questions (by Wednesday)

For the paper in the reading material above that you have read, answer these questions:

A. What are the main results, or guidelines, of the paper that you read? Provide a summary and give some concrete examples of either what the authors suggest when writing a research paper, or how they themselves formulate research questions and try to answer them.

B. How can you make use of the results or advice provided by the paper that you read, in order to make an assessment of the tentative research questions in your own team's extended thesis plan introduction?

Submissions

• (Per team) Each pair needs to upload the introduction section of their extended thesis plan to LISAM (in folder Seminar2/(groupname)/) no later than Tuesday in the third course week (14/11/2023). Make sure to have a Word/PDF file in your group folder for everyone to read.
• (Individually) Each individual needs to upload answers to questions A-B above, to be available on LISAM before the seminar.

Review (Wednesday)

(Individually)

After the submission deadline, each individual needs to read the other^* thesis plans (introduction and background sections) from the other members of your group and prepare answers to the questions C-D below.
Have your preparatory answers to C-D ready in the Lisam folder for your group before the beginning of the seminar.

C. Are the research questions easy to find, clear and with a reasonable scope compared to the master's thesis you read before Seminar 1? Justify your answer.

D. How would you assess the introduction of the thesis plan based on the grading rubric (attributes Introduction, Organization, and Language and form)? Justify your assessment by referring to the specific formulations in the rubric.

The seminar

During the seminar, you will first present the papers that you read and compare answers to questions A and B, and then review each others' submissions based on questions C and D. Make sure to outline concrete suggestions for improvement. Be the critic you wish to have. Your seminar leader will help you during your discussions.

After the seminar

After the seminar you have about 1.5 days (by friday in the third week, 17/11/2023) to revise your thesis plan, in particular the research questions, taking the feedback that you got from your group in the seminar discussion into account.

Submit the document with your revised introduction (via the first submission link in LISAM) for feedback on the introduction. Note that this submission is a mandatory moment towards passing UPG2.
Note: you will have to form spontaneous groups in Lisam for the submission, then you will both receive the feedback (on Academic English) via Lisam.

You will get feedback on the introduction from two sources:

• from Shelley / Brittany for feedback on Academic English, shortly before the last lecture (9 december). Individual feedback will be given through LISAM, and general feedback will be given in the last lecture.
• from your seminar leader about the contents of the introduction, in particular about the context, motivation, and the research questions. This (individual) feedback will be sent as PDF annotations by email. You can then ask questions about the feedback in the Feedback Seminar.

Page responsible: Christoph Kessler
Last updated: 2023-10-30