Final project

Final project

Individual students will work on a project of their own choice and design over the course of the semester, culminating with a class presentation followed by a final project delivery. The goal of this project is to make a linguistic discovery through application of data-intensive methods.

Components

A project consists of three main components: data, analysis, and presentation.

Data

  • Start with public data. Many linguistics research projects begin with a targeted data collection effort: field work, surveys, elicitation, human subjects, and more. But the underlying assumption of data science is that data already exists—often, though not always, “in the wild”—and it is up to a data scientist to harness it. True to this assumption, we will have you start with data that already exists and is public. (This will also save you a lot of headaches trying to figure out how to share non-public data ethically!)

  • Add value. You should not, however, be content with data as it is packaged and presented to you. In many cases, your data will need a lot of work – sourcing, cleaning up, and reorganizing. In other cases, you may be dealing with published data that’s more or less ready for analysis. You are, then, expected to add value: augmenting, annotating and leveraging multiple data sets are all potential avenues.

  • Follow best data practices. Throughout this semester, we will be learning about best data practices, both emerging and firmly established in data science circles. Make sure your own data efforts and the output are in compliance.

Analysis

  • Linguistic analysis. You will have designed your data with a research question in mind. Your data should make a suitable empirical basis for your linguistic inquiry; your research question should be properly motivated and addressed in a theoretically and methodologically sound manner. Your interpretations of the findings should likewise be rigorously supported by your data. Even with meticulous preparation, however, your data in the end may not prove fruitful grounds for your original research question. Pivoting is therefore allowed up to a certain point; whether or not this move is ultimately successful, reasons for pivoting and/or failure of the original research agenda must be thoroughly probed and documented, since this sort of outcome is all part-and-parcel of research efforts deeply grounded in real-life data and, further, provides valuable insight.

  • Computational methods. In your linguistic analysis, you are expected to employ various computational methods. Proper techniques should be used in accordance with your research question and the specifics of your data. At the same time, you should demonstrate mastery of these techniques by justifying your choice of computational methods and thoroughly evaluating the outcome, rather than blindly applying them and accepting the returned output. As with linguistic analysis, failed experimentation should not be brushed aside, but rather receive proper investigation and documentation, as this is all part of the discovery process. One concrete way this will likely show up is which R package(s) you use; you might need a package that we haven’t discussed, and you might need to try a few before finding the right one.

Since this is an R-based class, your data wrangling and analysis should all happen in R code. Because I don’t use Python, I can’t help you write or troubleshoot Python code; it’s okay if you use an R package that relies on Python code (e.g., spacyr, which wraps the popular and powerful Python spaCy library), but you shouldn’t be interacting with Python code directly. Likewise, you should never edit data files in Excel, as Excel doesn’t force you to create a reproducible record of what you’ve done to the file. If you’re not sure about what’s okay and not-okay, don’t hesitate to ask!

Presentation

This component encompasses all audience-facing aspects of your project, which include but are not limited to:

Weight distribution. Ideally, a project will have the three components in perfect balance: a total of say 18% (of the final grade total) will be equally split between data/analysis/presentation as 6%-6%-6%. In reality, everyone’s project will be different: some will have ambitious and challenging data curation plans, while others might wish to focus their efforts on extensive use of advanced computational methods. To accommodate this, a limited amount of trade-off is provisioned between the “data” and the “analysis” components: more data-focused projects therefore may have up to 7%-5%-6% distribution with more data-side contribution, while projects heavily focused on analysis are allowed to go easier on data-related efforts, with up to 5%-7%-6% split.

Submission

Your project should be initiated and developed in the form of a GitHub-hosted public repository. The final deliverables should include:

Project milestones

The term project is worth 40% of your final grade, which you accrue over the course of the semester through meeting several structured milestones. Refer to the links below for due dates.

  Milestone Weight Distribution: Data D; Analysis A; Presentation P Task
1 Project ideas 2% DDAA Send instructor 1-2 project ideas.
2 Project plan 2% DAPP Finalize project plan, create a GitHub project repository.
3 Progress report 1 4% DDDDDDPP Focus on data curation, report progress.
4 Progress report 2 4% DDDDAAPP Continue with data curation, attempt analysis.
5 Progress report 3 4% DDAAAAPP Data-side effort should be done; ramp up analysis.
6 Project presentation 6% AAAAAAPPPPPP Oral presentation of your work (last week of class).
7 Repository submission 18% DDDDAAAAPPPP
DDDDAAAAPPPP
DDDDAAAAPPPP		 Turn in final project in the form of a GitHub repository.

Project ideas

Due by 12pm Tuesday, October 4

You should come up with one or two project ideas. Include these details:

Since we haven’t discussed all the possible techniques you might want to use, I don’t expect you to necessarily know it off the bat. It’s okay to leave “blanks to be filled-in” like “I’ll perform some sort of web scraping”

Not sure what’s the proper scope for a project? Check out some projects from previous versions of this class. Of course, these projects used Python, which we don’t cover in this class; read them not for the code but for their scope. (Last year’s R Data Science class didn’t use public data, so their projects aren’t as instructive for our purposes.)

Submission: In the project_ideas/ directory of Class-Exercise-Repo, create project_ideas_YOURNAME.md. Commit, push to your fork, and create a pull request for me.

Project plan

Due by 12pm Thursday, October 13

Launch your project as a GitHub repository and publish a project plan. (If you were choosing between two project ideas, now is the time to pick one!)

Submission: Your project repo counts as your submission.

Progress report 1

Due by 12pm Thursday, November 3

For the 1st progress report, focus on your data. Goals:

Contents:

  1. progress_report.md
    • Create a section called “1st Progress Report”, and then provide a summary of what you accomplished. Keep it short (a screen-full), and provide links to related documents, including your knitted Markdown and data samples.
    • Include a subsection where you outline a couple of options (or a single option, if you are fairly sure) regarding the “sharing plan” for your data. You should plan out how much and what you will be sharing. Make sure to include a justification.
  2. At least one R Markdown document, knitted to Markdown, with an overview of your data:
    • Provide an overview of your data. Clearly document each step of your data processing pipeline.
    • Compile some basic stats on your data: the size (as of now) and the makeup are the bare minimum.
    • Bullet points have their uses, but let’s see some written summaries and explanations too.
    • Remember: your Markdown file is how you’re communicating your project to people who aren’t in your head. Make it easy for me and your classmates to understand what you are doing. Explain your goals, show your data and your processes.
    • This can be a single document, or multiple documents, depending on what works for you. Name your document(s) in some way that makes sense to you.
  3. Some form of your data. If all of your data is currently stored in a git-ignored directory, make a small sample available in a directory called data_samples/.

Above are the minimum requirements, but do feel free to impose additional organization as you see fit. This is your project, after all! Down the line, you’ll be using your README.md as a landing page for your project, where you can direct readers to the appropriate folders and files.

Some of you may have discovered that your project is not panning out as you had hoped and you need to start over. This is your last chance to do so; you will have to launch a viable project quickly. As far as your project repository is concerned, you should keep the old one (along with its Git history) but alter it to fit your new project:
  • Change your GitHub repository's name. That changes its URL, so you will need to update your local Git's remote setting.
  • Update your progress_report.md file with an explanation of what happened and why this change of course was necessary. You'll need your 1st progress report too.
  • Edit your README.md, project_plan.md, and any other files accordingly to fit your new project direction. They shouldn't contain references to your old plan. (Remember: You're not throwing the old versions away, since they're part of your Git history.)

Submission: Your project repo counts as your submission.

Progress report 2

Due by 12pm Tuesday, November 15

For the 2nd progress report, ease up your focus on data and start working on analysis. Goals:

As for the progress report itself, these should be the content:

  1. Your progress report: progress_report.md
    • Create a section entitled “2nd Progress Report”, and then provide a summary of what you accomplished. Again keep it short (a screen-full), and provide links to related documents, including your R Markdown document and other folders/documents.
    • Include two subsections:
      • Sharing scheme for the “found” portion of your data. You had already made some tentative plans as part of the previous progress report; you are finalizing the scheme here.
      • Your decision on licensing for your project and reasons/justification. See “Your license” item below.
  2. Your code in the form of an R Markdown file, knitted to Markdown. You have three options:
    • EXISTING: the existing script file which was part of your 1st progress report. You continue to update and add to it.
    • NEW REPLACEMENT: a whole new script file that replaces the earlier one. The script you submitted earlier as part of the 1st progress report is now regarded as initial exploration and is no longer part of your work pipeline.
    • NEW CONTINUING: a new script file that’s part of a pipeline. The earlier script you submitted for the 1st progress report accomplishes PART 1 of your work pipeline, and this new file is PART 2 that picks up where PART 1 left off.
    • On top of your script, specify which type it is so we will have a sense of how the script fits in your project. Make a note of this in your progress report section as well.
  3. Your data: include it in a designated folder. Suggested name: data/. Be careful not to commit anything that you cannot publish.
    • If including the found portion of your data in its entirety, make sure it’s within your right to do so. Present a justification in progress report.
    • If you are including samples, make sure it’s within fair use. Document your sampling method and justification in progress report.
    • Are you including derived data? Again provide justification.
    • Are you including some new data you created yourself, like annotation? Again, document it.
  4. Your license: LICENSE.md.
    • This is a binding licensing document, intended as audience-facing. This is where you lay out your licensing terms for your future visitors wanting to use your data and code.
    • Do not confuse this with the license of the dataset you downloaded: this document is about YOU specifying a license for YOUR PROJECT REPOSITORY.
    • You may adopt popular, existing licensing standards: revisit Lauren Collister’s materials, and consult this quick guide and also this one from GitHub Help.
    • Include reasons/justifications in the appropriate subsection in your progress report.

Submission: Your project repo counts as your submission.

NOTE: After ‘submission’, don’t hold yourself back from pushing more updates and changes thinking you should freeze the repo until grading is done. There’s no need: I have access to your repo at every stage it moves through.

Progress report 3

Due Thursday, December 1

For the 3rd and last progress report, you should focus on analysis. Goals:

As for the progress report itself, these should be the content:

  1. Your progress report: progress_report.md
    • Create a section entitled “3rd Progress Report”, and then provide a summary of what you accomplished. Again keep it short (a screen-full), and provide links to related documents, including your R Markdown file and other folders/documents.
  2. Your code in the form of R Markdown, knitted to Markdown. The same three options:
    • EXISTING: the existing script file which was part of your earlier progress report. You continue to update and add to it.
    • NEW REPLACEMENT: a whole new script file that replaces something earlier. The script you submitted earlier as part of the 1st progress report is now regarded as initial exploration and is no longer part of your work pipeline.
    • NEW CONTINUING: a new script file that’s part of a pipeline. The earlier script you submitted for a previous progress report accomplishes PART 1 of your work pipeline, and this new file is PART 2 that picks up where PART 1 left off.
    • On top of your script, specify which type it is so we will have a sense of how the script fits in your project. Make a note of this in your progress report section as well.
  3. Your data:
    • Some of you have worked on your data files. Make sure to note it in your progress report.
    • Are your data files finished as of last progress report? No new changes since? If so, make a note of it in your progress report.
  4. README.md file:
    • If you haven’t already, make sure your project has a proper title (not just your repo name like Inaugural-Address-Analysis but something human readable)

Submission: Your project repo counts as your submission.

Presentation Guidelines

Format

Content

Evaluation

Presenter schedule (randomly generated)

Repo submission guidelines

Due by 2:15 Thursday, December 15 (one week after the end of our last class meeting)

You’ve worked hard through many project milestones, and it’s time to prepare your project for final submission. Unlike the three progress reports where the focus was firmly on the process, the final submission should highlight the results and your interpretation of them. The process should still get a fair and clear illustration, but you should prune out from your production code any “branches” representing trial-and-error that led to a dead end. (You are encouraged to move any old code bits into a designated “scratchpads” subfolder.) All in all, your GitHub repo should present a coherent picture of your project, from start to finish—someone looking at it from the outside should perceive a product, not a work-in-progress.

Your repo: files and folders

Below are the required files with predetermined file names. Objects that are entirely/substantially new in this submission are in orange, with more details linked below.

In addition, you should have:

Lastly, some of you might have extra files and directories serving some purpose: perhaps a “scratchpads” folder containing some old code that is no longer relevant, or something like that. Make sure to explain what these are in your README.md document.

README.md

Revamp your README document and give it a proper structure. This document is what greets your visitors, so its goal should be to give them a short but proper orientation. It should include:

final_report.md

Think of this as a usual “final report” that is in a markdown format instead of MS Word. Details:

Images folder and files

Your final_report.md file will need figures and graphs for illustration.

Your code: R Markdown

The same usual guidelines for your R Markdown files continue to apply: your code should work correctly while walking the audience through the whole process. This time around, however, your code should be in a streamlined form: you should prune your code of any unsuccessful bits and experiments that have since been abandoned. In other words, your code files should demonstrate your project in a lean and coherent manner. Some important points: