Web Scraping and Analyzing YoYo Tricks

Web Scraping and Analyzing YoYo Tricks

Overview

Happy Sunday everyone! Today I’m going to skim over a recent passion project of mine that involves yoyo tricks. I have been throwing around yoyos since I was a teenager. Proof: Me, junior year, in the high school talent show (I’m no better today):

But don’t worry; I won’t be demoing yoyo tricks today. The purpose of this post is to provide a high-level overview of web scraping and network visualization. For me, the combination of yoyo tricks + web scraping + network graphs was exciting enough to convince me to dive in.

The motivation here was to obtain/create a catalog all yoyo tricks and how they relate to one another. This could be useful in determining the most foundational yoyo tricks, the most complex yoyo tricks, or to even in creating a recommendation engine to give users a selection of new tricks to learn based on what they know.

Where to Start

So where does one obtain a catalog of yoyo tricks? The guys at YoYoTricks.com do an amazing job of teaching a wide array of yoyo tricks, so that’s where I started. and I was ecstatic to see just how clean and organized their site is. Honestly, their meticulous design eliminated a lot of leg-work from this project. In addition to providing transcripts for each video, they provide links to related tricks, and an easily accessible index of their entire site.

So, the first step in the process was scraping this index to obtain a list of all yoyo trick names and the link to each trick tutorial. For this, I used R (rvest for HTML parsing, dplyr for transformations) and SelectorGadget to help define what to CSS classes to collect.

read_html('https://yoyotricks.com/sitemap/') %>% html_nodes('.cat-item-1 a') %>% html_attrs() %>%
  lapply(function(row) {
    data.frame(title=row['title'], link=row['href'], stringsAsFactors = FALSE)
  }) %>%
  bind_rows() %>%
  filter(grepl(x=link, pattern='https://yoyotricks.com/yoyo-tricks/.+')) %>%
  head(5) %>% kable()
title link
Large Haugan Collider https://yoyotricks.com/yoyo-tricks/large-haugan-collider/244899/
Dorito https://yoyotricks.com/yoyo-tricks/tyler-severance-collection/dorito/243432/
Reverse Brent Stole https://yoyotricks.com/yoyo-tricks/reverse-brent-stole/239362/
Vanish Grind https://yoyotricks.com/yoyo-tricks/vanish-grind/233896/
Daniel Day-Lewis https://yoyotricks.com/yoyo-tricks/daniel-day-lewis/232973/

From here, I won’t be supplying the verbose code here for two reasons:

  1. The code is long, messy, and uninteresting.
  2. I don’t want YoYoTricks.com overwhelmed with requests.

However, for those that are yearning to experiement, I have hosted a SQLite database of the trick catalog and references here:

www.kaggle.com/dm3ll3n/yoyotricks

Next Steps

From here, I iterated over each trick to obtain the tutorial transcript and enrichment data, such as trick category, trick tags, and the names of tricks included in the “Tricks You Need to Know” tab. While doing so, I accummulated a list of all references between tricks.

id title author category tags link
1 Vanish Grind Adam B. Yoyo Tricks;Yoyo String Tricks (1A) Green-Triangle;grind https://yoyotricks.com/yoyo-tricks/vanish-grind/233896/
2 Daniel Day-Lewis Adam B. Yoyo Tricks;Long String Tricks https://yoyotricks.com/yoyo-tricks/daniel-day-lewis/232973/
3 Bouncy Castle Jake E. Yoyo Tricks;Yoyo String Tricks (1A) Hop;Green-Triangle https://yoyotricks.com/yoyo-tricks/bouncy-castle/228525/
4 Monochrome Cory H. Yoyo Tricks;Yoyo String Tricks (1A) Slack-String https://yoyotricks.com/yoyo-tricks/monochrome/224191/
5 Wax On, Wax Off Jake E. Yoyo Tricks;Yoyo String Tricks (1A) Repeater;chopsticks https://yoyotricks.com/yoyo-tricks/wax-on-wax-off/224190/
6 Three Open Mount Slack Binds Cory H. Yoyo Tricks;Yoyo String Tricks (1A) Slack-String;Binds https://yoyotricks.com/yoyo-tricks/three-open-mount-slack-binds/224189/

When building a table of references between tricks, I considered tricks referenced in the “Tricks You Need to Know” tab, and I also parsed the transcript of each video for the names of yoyo tricks.

In the table above, each number corresponds to a trick in the catalog table. The column weight indicates how many times the from trick references the to trick.

from to weight
1 294 1
2 90 2
2 281 2
2 235 2
2 335 2
2 335 1

So, with a catalog of tricks in hand, and a tally of references between tricks, we can now visualize and explore the network with the R package visNetwork. In the graph below, the larger nodes refer to tricks that are referenced more often than others. Upon visual inspection, we see that a few of these foundational tricks include:

  • Man on the Flying Trapeze
  • Binds
  • Flips
  • The Breakaway

Note, too, that the graph below allows selection based on the yoyo trick “style”, which is one of 1A, 2A, 3A, 4A, and 5A. When a novice thinks of yoyoing, they’re likely to think of 2A (two-handed looping tricks). However, 1A tricks are much more popular in the community, based on my experience and the prevalence of 1A tricks.

While this visual is fun and interesting (IMO), it is busy and tough to perform analyses on. Thankfully, we have a data frame of edges (links), which is all that’s required to leverage graph theory with the powerful igraph library. As an example, let’s measure betweenness of the nodes. This measure will allow us to quantify which tricks are crucial to the structure of the graph. In this case, nodes high in betweenness are either very foundational (many references to) or very complex (many references from).

g <- igraph::graph_from_data_frame(df_edges, vertices=df_nodes)

betweenness(g) %>% set_names(V(g)$title) %>% sort(decreasing = TRUE) %>% head() %>% kable()
x
Binds 1312.6182
The Breakaway 864.3170
Man on the Flying Trapeze 621.9813
Brain-Scrambler 453.3278
One-and-a-Half Side-Mount 416.6106
Split-Bottom Mount 357.0000

Indeed, the tricks with the top betweenness are foundational tricks. Most tricks are unresponsive and involve a “bind”. The “Breakaway” is a foundational trick, as is “Man on the Flying Trapeze”. In fact, you can’t do a “Trapeze” without a “Breakaway”. Similarly, various popular “mount” tricks make the list, as mounts are often the building blocks for complex tricks.

It is now possible to identify the most fundamental or most complex trick, quantify relationships between tricks, and even develop a recommendation system that recommends new tricks to learn given a set of known tricks. Of course, the guys over at YoYoTricks.com already do an excellent job of guiding beginners through the ranks of yoyo tricks. If you want to start up yoyoing, I would look no further than YoYoTricks.com.

If you’re interested in further data exploration, I encourage you to spin up a kernel on Kaggle with the YoYoTricks dataset:

https://www.kaggle.com/dm3ll3n/yoyotricks

If you find something interesting, let me know in the chat!

Until next time,
Donald

Related

comments powered by Disqus