Dominic's Homepage!
My name is Dominic Teo and I'm currently an analytics manager with the Ministry of National Development (MND). I'm interested in the intersection of public policy and technology as well as the application of Big Data and Civic Tech in the public sphere.
I graduated Cum Laude from Sciences Po Paris with a BA in Social Sciences & Economics followed by a MS in Computational Analysis and Public Policy (MSCAPP) at the University of Chicago. The MSCAPP degree is a 2 year dual degree offered by the Schools of Computer Science and Public Policy.
Take a look at my resumé and various projects! (click on titles of projects for more details)
Hosted on GitHub Pages — Theme by orderedlist
Netflix Top 10 - Choosing What To Watch Next (2022)
Built using: Python - Flask and BeautifulSoup, Javascript, HTML and hosted on Render
Background
Recently, Netflix began publicizing its global top 10 most watched films or TV shows. This provides a list of content for people to watch yet besides the title, Netflix’s website does not provide any more information on these films/shows that will help people decide if these films/shows are for them.
This website/program helps scrapes the top 10 lists from Netflix’s website for the following countries:
| Countries covered in this program |
|---|
| Australia |
| Brazil |
| France |
| Hong Kong |
| India |
| Japan |
| Singapore |
| Taiwan |
| United States |
| United Kingdom |
Users can either select the country in the drop-down list or access the results directly via URL. There are 2 aspects to this program:
- Web Scraping Netflix Top 10 data from Netflix’s website.
- Using the Netflix Top 10 data as input for the OMDb API to access the film/show’s details as well as YouTube trailer.
The OMDb API and YouTube trailer will then be shown.
Web Scraping the Netflix Top 10 data from Netflix’s website
- I’ve created a Flask API that utilises BeautifulSoup to scrape Netflix’s Top 10 website to return the Top 10 shows and films for the country selected.
Utilising OMDb API to get film/show’s info and web scraping the YouTube trailer
- Loop through the web scrapped list of top 10 films and tv shows as input for a Flask API that utilises the OMDb API to return the show/film’s IMDB rating (out of 10) and a short description of the film.
- There will be cases where the film/show title does not match the OMDb API database and will hence return no results. I’ve tried to mitigate this through the following steps:
- Oftentimes the title scrapped is too long (for titles with “:” e.g. 13 Hours: The Secret Soldiers of Benghazi) and the title in the OMDb database is its simplified version (e.g. 13 Hours) hence I will try to split the scrapped title on “:” and will use the first part of the title as input for the OMDb API again.
- For TV shows, just search using the title without the season number.
-
Then the Flask API will use the film title to web scrape the URL for the film trailer on YouTube which will then embedd it.
-
Note: on Javascript, accessing the Flask API that loops through the web scrapped list means that it will return the results not in the Top 10 order but in the order in which the results are returned. Hence, I have created 10 unique ID divs for Films and another 10 unique ID divs for TV shows. Their ID contains their rank thus regardless of the order in which results are returned, they will populate the div that contains their rank.