Skip to content
Proyectos

CU-ITSS/Web-Data-Scraping-S2019

Ver en GitHub

CU-ITSS/Web-Data-Scraping-S2019 has 312 GitHub stars and 69 forks. GitHot measured 0 stars in 24 hours. Last checked 2026-11-22 02:55 UTC. Last push 2022-04-25.

Cómo se miden estas cifras

Crecimiento de estrellas medido

in 24h
0
in 7 days
Recopilando historial
in 15 days
Recopilando historial
in 1 month
Recopilando historial

Cambios netos en períodos que se solapan, no totales diarios. Los datos no disponibles se dejan en blanco. Las estrellas indican interés, no calidad.

Acerca de

Spring 2019 ITSS Mini-Course — ARSC 5040Brian C. Keegan, Ph.D.Assistant Professor, Department of Information ScienceUniversity of Colorado Boulder

Herramientas de desarrollo

README.md

Original

Web Data Scraping

Spring 2019 ITSS Mini-Course — ARSC 5040
Brian C. Keegan, Ph.D.
Assistant Professor, Department of Information Science
University of Colorado Boulder

Copyright and distributed under an MIT License

Course description

This is a five-week one-credit "mini-course" on retrieving ("scraping") data from the web. The course is intended for researchers in the social sciences and humanities with computational instincts but limited or no prior programming experience. Each class will be 2.5 hours long: we'll take a break mid-way for biological input and output. Lectures will use a combination of lecture-by-notebook as well as hands-on exercises. The end of each class will have links to resources and additional take-home exercises. Students will have the option of presenting their solutions to the take-home exercises at the beginning of the next class.

Although many programming languages offer libraries for web information retrieval and analysis, we will be focusing on the Python data analysis ecosystem given its popularity and capabilities. I would strongly recommend that students download the latest Python 3.7 or above version of the Anaconda distribution which includes the Jupyter Notebook environment we're currently in, most of the data libraries we will use, and other conveniences.

Learning objectives

Students will:

  • Be able to navigate and access structured web data like HTML, XML, and JSON
  • Develop strategies for identifying relevant structures in semi-structed data using browser console tools
  • Utilize Python-based libraries to make request and parse web data
  • Retrieve data from platforms' application programming interfaces (APIs)
  • Critically reflect about the technological and ethical constraints on web scraping

Class outline

  • Week 1: Introduction to Jupyter, browser console, structured data, ethical considerations
  • Week 2: Scraping HTML with requests and BeautifulSoup
  • Week 3: Scraping an API with requests and json, Wikipedia and Reddit
  • Week 4: Scraping web data with Selenium, ethics of screen-scraping
  • Week 5: Scraping Twitter

Evaluation

To be determined based on enrollments, distribution of skills, etc. but will primarily involve regular attendance, participation, and upwards trajectory in skill and confidence.

Acknowledgements

This course will draw on resources built by myself and Allison Morgan for the 2018 Summer Institute for Computational Social Science, which were in turn derived from other resources developed by Simon Munzert and Chris Bail.

Thank you also to Professors Bhuvana Narasimhan and Stefanie Mollborn for coordinating the ITSS seminars.

Explorar proyectos relacionados

Cargando…