The document titled “Datasheets for Datasets” introduces the concept of providing detailed documentation for datasets used in machine learning, similar to datasheets in the electronics industry. This proposal aims to enhance transparency, accountability, and reproducibility in the machine learning community by ensuring datasets are accompanied by comprehensive descriptions covering their motivation, composition, collection process, recommended uses, and maintenance. The authors argue that such datasheets can help mitigate biases, facilitate better decision-making for dataset consumers, and improve the overall quality and fairness of machine learning models. The document outlines the development process for these datasheets, presents a structured set of questions for dataset creators to answer, and discusses the potential impact and challenges of implementing this practice. The authors also provide an example datasheet and acknowledge the contributions of various individuals and organizations to the development of this proposal.

Authors: Timnit Gebru, Jamie Morgenstern, Briana Vecchione, Jennifer Wortman Vaughan, Hanna Wallach, Hal Daumé III, Kate Crawford

Pages: 18

The document “Fields of Gold: Scraping Web Data for Marketing Insights” explores the growing practice of using web scraping and APIs to gather web data for marketing research. It emphasizes the importance of addressing validity concerns throughout the data collection process, from selecting sources to designing and extracting data. The authors propose a framework that integrates technical, legal, and ethical considerations to enhance the validity of web-scraped data. They review over 300 articles from top marketing journals to illustrate how web data has advanced marketing thought and provide a typology of its uses. The document also discusses future research directions, encouraging the exploration of underutilized data sources and innovative approaches to capture evolving marketplace realities. The goal is to facilitate broader adoption and more rigorous use of web data in marketing research.

Authors: Johannes Boegershausen, Hannes Datta, Abhishek Borah, Andrew T. Stephen

Pages: 20

Slides for Week 1 of the course. The slides provide an overview of web scraping and APIs, detailing their uses in capturing data from websites and obtaining official data programmatically. It outlines the steps for setting up a web scraper in Python using requests and BeautifulSoup and discusses the process of making API requests. The text highlights the opportunities web data offers for businesses and researchers, noting the increasing importance and impact of web data in marketing. It emphasizes the need for specialized knowledge in web data collection due to its complexities and challenges, such as legal and ethical considerations and the need for tailored strategies. Frameworks and hierarchical approaches for effective web scraping and data collection are also presented, underscoring the importance of structured, responsible, and efficient practices in web data utilization.

Authors: Hannes Datta

Pages: 39

Slides for Week 2 of the course. The slides provide comprehensive guidance on web data collection, highlighting the differences between web scraping and APIs, including scaling, cost, and data access. It details the structured approach proposed by Boegershausen et al. (2022) for online data collection, emphasizing three stages: Source Selection, Collection Design, and Data Extraction, each with specific questions and legal and ethical considerations. The text outlines practical steps for web scraping using HTML, CSS selectors, and XPATH, and stresses the importance of wrapping code in functions for reusability and converting results to JSON for flexibility. It discusses challenges in sampling, extraction frequency, and data processing during collection, offering solutions for handling sensitive information and ensuring data quality. The overall emphasis is on the technical, ethical, and operational aspects necessary for effective web data collection and utilization in research and business.

Authors: Hannes Datta

Pages: 39

Slides for Week 3 of the course. The slides provide detailed instructions and considerations for web data extraction, including handling relative and absolute links, saving data in JSON format, and preventing array misalignment by preserving data's original structure. It discusses the frequency of data extraction and the importance of retaining raw data during collection. For dynamic websites, it recommends using Selenium for simulated browsing, combined with BeautifulSoup for parsing and extracting data from HTML content. The approach varies based on the site's complexity: using the requests library and BeautifulSoup for static sites, and Selenium for dynamic sites with JavaScript content, ensuring comprehensive and effective web scraping techniques.

Authors:

Pages: 45

Slides for Week 4 of the course. The slides explain the nature and functionality of APIs, highlighting their role as connectors between platforms and databases and their availability to developers and researchers after registration and approval. It contrasts APIs with web scraping, noting that APIs provide structured JSON data and clearer permissions, while web scraping involves handling unstructured HTML and potential legal ambiguities. It covers discovering API documentation, particularly authentication and endpoint exploration using OpenAI as an example. The text also explains how to interpret JSON data using tools like JSONviewer.stack.hu, the differences between .append() and .extend() methods in Python, and how to submit parameters in API calls via URLs or request headers. Additionally, it distinguishes between for loops and while loops in programming, emphasizing their use cases based on the predictability of their termination conditions.

Authors: Hannes Datta

Pages: 26