Link Search Menu Expand Document
CS8803-MDS

Draft Paper

DUE WEDNESDAY, NOVEMBER 29 AT 4:00PM.

Table of contents

It’s time to start putting everything together into the paper. This is an exciting moment — this is when all the work you’ve been doing starts to take shape into a paper that you can potentially submit for publication. This won’t be a complete draft, but it will put together the bones of the paper and incorporate text for a substantial proportion of the paper. You can continue to iterate on the final results. However, a common failure point for researchers is not to save enough time to write a convincing paper before the deadline, which is why we’re starting now.

In class, your draft paper will be passed to other students for a mock peer review, and you’ll do a mock peer review on their paper. We expect that you will be able to address the feedbacks from the TAs and the peer review in the final submission.

Expected Length: The final paper should be at least 4 pages (excluding references) for solo teams, 6 pages for others. In your draft paper, aim for at least two-thirds of the minimum page requirement for the final paper.

Download the LaTeX template here: CS8803-MDS: Final Paper.

Part 1: Make Your Paper Outline

Make an outline by planning out the sections and subsections, as well as figures and tables that you are planning to include in your final paper.

  • Sections and subsections: What sections/subsections are you planning to include in the paper? What are the main points that you are trying to cover in each section?
  • Figures, tables, graphs: For each figure and table you will have in your paper, what role does it play? Why is it included in that section/sub-section?

Here’s an example paper outline to show you the level of detail you should describe your paper’s structure. You should have about 1 page for this part, but don’t go overboard. It is more important that you spend your time this week writing a good draft paper, but we’d still like you to explicitly do this meta-analysis so that you have something to reference when writing. This outline will also help your peers to get an overall sense of the paper during the peer review.

Part 2: Write your draft paper

Now, it’s time to flesh out your outline into a full draft paper. You are encouraged to use content that you generated in previous assignments, e.g., the introduction and progress reports. You may need to tweak your previous writing as your project has evolved.

At this stage, your draft paper can leave the Related Work, Discussion and Conclusion sections in outline form. Method and Evaluation sections can be incomplete or just pilot data. We expect that you will be focusing most of your effort on writing out the details of your approach, and reporting any results you have so far. Include a References section with citations at the bottom. Include any graphs that you are going to want to include in your final writeup. You will, of course, be able to update this for your final submission as your project progresses.

Your draft paper will be traded with another group for peer review in section after the deadline. We, the staff, will also send feedback on the draft.

How to Design Your Evaluation

To convince people that your idea is correct, you’ll need some way to convince an expert that you have evaluated it fairly and correctly. The evaluation section will also be an important component of how we evaluate your final project. Below is a scaffold that you can follow to plan out your evaluation.

Step 1: Articulate Your Thesis

The first step in planning an evaluation is to articulate the main thesis of your work. (Remember from Assignment 1, Project Proposal, that the main thesis of the work is likely embedded in the topic sentence of your bit flip paragraph.) Go back and reflect on that statement — tweak it if necessary based on what you’ve learned from your project so far.

Step 2: Derive Your Claim

Theses typicall imply a claim. For example, “x > y”-type (“X is better than Y”) theses imply a claim that x is in fact better/faster/more performant/more usable than y, and “∃ x”-type (“there exists an X”) theses imply a claim that whereas x could not exist before, that it does with your system. Discuss with your team the claim implied by your thesis.

Step 3: Design Your Evaluation

Now, you need to work from your claim to design a specific evaluation plan. How do you prove what you have claimed? The evaluation typically specifies:

  • Dependent Variable (DV): what is your dependent variable? (This is the variable you measure as the outcome, such as runtime and accuracy.)
  • Independent Variable (IV): what is your independent variable? (This is the variable you manipulate for comparison to create conditions, such as the algorithm or the interface used.)
  • Task: what is the specific task that is being performed in order to measure the DV? (This might include executing a benchmark, a known ML classification task, or a specific sequence of behaviors that a user must perform.)

You don’t need to re-invent the wheel here. Often papers in your related work establish an evaluation paradigm that you can import to your paper. In fact, this is often preferred, since then you don’t need to convince a reader that your approach is valid, since it’s already in the literature. So, go review the evaluations used in your prior work and use those to develop a few possible plans. Then, share those plans with your team and work together to develop a variant that works well for your project.

Next, run the following unit test on your proposed design: does it directly test the thesis you articulated above? Imagine a few possible outcomes from your evaluation. Depending on how it comes out, does it directly prove or disprove your thesis, or only obliquely shed light on whether your thesis is correct?

Submission and Grading

Submit a PDF with (1) your outline and (2) your draft paper on canvas before class. Your submission will be used in class for peer review.

Your submission will be graded on the following criteria:

  • Outline: does your outline provide a clear and comprehensive plan of the paper? (5pt)
  • Completeness:
    • Method Section: Does your draft provide an overview of the key elements of your methodology? (5pt)
    • Evaluation Section: Is there a well-defined plan for your evaluation, demonstrating how it directly tests your thesis? At this stage, only preliminary results are expected. (5pt)
  • Clarity: is the writing overall clear and easy to follow for a technical expert in the field? (5pt)