A Dash of Data

Fun-sized bites of data analysis

February 18, 2015
by Alice Zhao
0 comments

The Data Behind ‘The Bachelor’ & ‘The Bachelorette’

The year is 2002. I’m sitting in front of a TV along with a group of girls excited to watch a new reality TV show that’s come on air. There are twenty-some beautiful women all trying to win the love of a successful Harvard grad (who happened to be both homecoming king and valedictorian of his high school as well). Over the course of 7 weeks, he dates the women, gives roses to the ones he’d like to keep and in the end, chooses the woman he ultimately wants to marry — aka the winner of The Bachelor.

Thirteen years later, the show is still steadily bringing in an average of 8 million viewers per episode (myself being one of them!). And so, I dedicate this post to my favorite Monday night guilty pleasure, The Bachelor.

One of the reasons I keep coming back to the show every season is for the hope of a happy ending. The most famous success story is of Trista and Ryan who have been together for over ten years and have two kids together. I also love Jason and Molly who didn’t end up together at the end of their season, but got married later on. So just how many of the 28 seasons have ended in success?

success_funnel

It turns out not that many do. In fact, only two-thirds of the seasons end with proposals. Then out of those proposals, only 5 have led to marriage, with the Bachelorette having a better success rate at 30% versus the Bachelor at 11%. The first successful season occurred in 2003 (Trista & Ryan), the second one happened six years later in 2009 (Jason & Molly) and there have been quite a few more happy endings in recent years as well.

While I watch the show in part for the love stories, the other part is for the contestants (and all of their drama), of course! I decided to take a closer look at all of the contestants who go on the show, starting with what they do for a living. I looked at the professions of the contestants in the final four, since I figured these were the more desirable occupations as they had made it so far into the show.

occupations

I saw that the most desirable females tend to be models, assistants, students and teachers, while the top men are professional athletes or in finance. There were a few jobs that were shared by both men and women such as realtors and sales reps. Out of the 200+ male contestants I looked at, I found that none of them had “assistant” in their title and only 2 were students (both eliminated in episode 1). Out of the 300+ female contestants, there were no engineers and they often had job titles like “dog lover” or “free spirit”. [Note to self: Change job title from “data scientist” to “data diva”.]

It’s unclear the exact reason for this disparity. Perhaps there are few male assistants and female engineers that actually want to be on the show. Perhaps people in these professions aren’t dramatic enough to be on the show. Or perhaps they’ve all found the love of their life already. 😉 Whatever the case, please consider them for future seasons, ABC!

So my next question was, how old are these contestants, and which ones get picked?

age_females

I found that the average age of the bachelor is 31. What was surprising was that the average age of all of the female contestants that he gets to choose from is actually 5 years younger at 26. Given that age gap, the bachelor still tends to choose a women who is a year younger than the average age of the group. Note that no woman over 32 has ever made the final four, and while a few 30+ women have made it to that stage, the oldest winner was 28 years old and the youngest was 22.

While the average bachelor chooses a winner who is 6 years younger than himself, the average bachelorette tends to choose someone who is 1 year older than herself.

age_males

The average age of the bachelorette is 27 and the male contestants that she gets to choose from are 2 years older than her on average at 29. It’s worth noting that the two oldest contestants who were 38 and 41 both got eliminated during the first rose ceremony. What’s interesting is that again, the winner is a year younger than the average age of the group. It seems that when given the choice, both the bachelor and bachelorette tend to choose someone younger than the average of the contestants presented to them. [Pro Tip: Try to stand next to people older than you.]

Finally, I looked into the physical attributes of the women and men on the show and the most interesting tidbit I discovered was regarding height. I found that the Bachelorette is 5’5″ on average, which is just above the national female average of 5’4″, while the Bachelor is 6’1″ on average, which is a whole 3 inches above the national male average of 5’10”. [Pro Tip: To be successful in life, be tall or wear heels.]

I will end this post by sharing with you some of the youngest (and cutest) Bachelor contestants on Jimmy Kimmel’s The Baby Bachelor. Hopefully it will keep you busy as we all wait for next week’s Bachelor episode!

The data for this analysis was taken from Wikipedia (which had information on two-thirds of the seasons), Hollywood Life and Spotted Ratings.

December 21, 2014
by Alice Zhao
3 Comments

How Christmas Songs Have Evolved Over Time

‘Tis the season of giving and eating and of course… Christmas songs! When I stumbled upon the ASCAP’s list of the Top 30 Holidays Songs of the Century, I knew I had found my data set for this month’s blog post. It’s pretty awesome when blasting Christmas music = doing research for a blog. At one point my husband and I started speaking to each other in song lyrics, so I had to tone it down a bit on the Christmas music even though it’s the most wonderful [thing to listen to] of the year.

I decided to start by looking at when the top Christmas songs were written. It turns out that nearly two-thirds of them first came out in the 1940’s and 1950’s. When artists today release Christmas albums, it’s pretty much expected that they’ll include covers of songs from this time period.

It made me wonder, what’s the reason that classic Christmas songs are so much more memorable than newer Christmas songs? I decided to group the songs by theme and see if any patterns emerged.

I found that certain themes such as describing “Christmas in the Air” and wishing people “Merry Christmas” via song are timeless, while others seemed to evolve. For example, songs about “Being Together in the Cold” and being “Home” for the holidays were written up until the mid-1950’s when they switched over to songs about having fun and “Partying” at Christmastime.

One of my favorite trends was the transition from songs about Santa (“Santa Claus is Coming to Town” & “Here Comes Santa Claus”) in the 1930’s and 1940’s to songs about Santa being in love (“I Saw Mommy Kissing Santa Claus” & “Santa Baby”) in the 1950’s to songs straight up about love in the 1970’s and beyond (“Last Christmas” & “All I Want For Christmas Is You”).

When you look at the use of the word “Christmas” in songs over the past few decades, you see that every one of the top songs since 1963 has the word “Christmas” in it. It’s almost as if the word “Christmas” is put in a song just for the sake of telling you that it’s a Christmas song.

Take “Last Christmas” for example:

Last Christmas, I gave you my heart
But the very next day, you gave it away
This year, to save me from tears
I’ll give it to someone special

If you remove the word “Christmas” from the lyrics, the song actually has nothing to do with Christmas at all. It is simply a love song.

This made me wonder if Christmas songs have had less substance over time as well. After doing some digging, I found that there is no significant correlation between the year a song is written and the complexity of the song. It’s still fun to look at the vocabulary size of songs though, regardless of year. Here I’ve listed the top and bottom 5 songs in terms of the number of unique words.

“All I Want For Christmas Is You” has the most words and uses the most Christmas references of them all, which I guess is why the song feels extra Christmas-y (along with the choir and bells in the background). I also found it pretty humorous that “Feliz Navidad” has the smallest vocabulary of any Christmas song. It uses only 24 words over the course of 3:02 minutes.

Overall, we see that Christmas songs have evolved over time, from songs about fictional characters and being home for the holidays to ones about celebrating with friends and hoping to get your love interest for Christmas. While some might be concerned with the changes, I actually think it’s pretty impressive that the classic songs have prevailed. It shows that they are beautifully written and capture the spirit of the season better than any songs can do today. When else can you get kids from 1 to 92 all singing along to the same tunes? That’s the magic of Christmas songs.

Happy holidays, everyone!

November 11, 2014
by Alice Zhao
2 Comments

Today’s Top Female Pop Artists (In Charts)

A couple weeks ago, I read an article on Billboard about how female artists have been ruling the charts. I took a moment to think about this phenomenon (as I paused Meghan Trainor’s “All About That Bass” on my phone) and realized that I really had been listening to a lot of female artists lately. It seemed like ages ago when I was telling the DJ at my wedding that she absolutely HAD to play “All of Me” by John Legend and “Happy” by Pharrell.

So I decided if female artists are ruling the Billboard charts, I might as well create some of my own charts about them too. I used Billboard as my inspiration and looked at female artists with #1 singles on the U.S. Billboard Hot 100 list within the past six years.

The first thing I wanted to do was figure out a way to map these 15 women onto one chart. I decided to look at the number of Hot 100 songs and #1 singles that they’ve had during their careers.

B Female Artists

I found a couple clusters of artists on the chart. On the bottom left you have the ladies that are new to the scene, such as Iggy Azalea, Lorde, Carly Rae Jepsen and Meghan Trainor. In the middle you have the artists that are well-established, having three or four #1 singles and around twenty songs reach the Hot 100 list. These are women such as Kelly Clarkson, Pink, Ke$ha and Lady Gaga.

B Female Artists Labels

Out of all these female singers, there are two that really stand out from the crowd – Taylor Swift and Rihanna. Since Taylor Swift’s debut single “Tim McGraw” made it onto the Hot 100 list back in 2006, she has had a whopping 67 Hot 100 songs. During Rihanna’s nine year career, she has had 13 #1 singles, tying her with Michael Jackson.

Let’s take a closer look at Taylor Swift. How is it possible that she’s had so many Hot 100 songs?

B Taylor Swift

It turns out that two-thirds of the songs on her five albums have made the Hot 100 list, with 100% of the songs on her 2010 “Speak Now” album making it onto the charts.

How about Rihanna? Just how significant are her 13 #1 singles?

B Rihanna

While it doesn’t look like much spread over nine years, if you total up all of her weeks at number one, she has spent almost a full year as the top artist in the U.S.

After doing all of this digging, I thought to myself, I can’t believe Taylor and Rihanna are so successful – they seem so young! So, I took a look at the age of all 15 women when their first song reached #1 on the Billboard charts, and I confirmed my theory – they are so young.

B Age

To my surprise, Carly Rae Jepsen was the oldest at 26! Most of today’s female pop stars make it to the top of the charts around 20 – 23 years old. After this analysis, I’ve learned that my chances of making it to the top of the Billboard charts are pretty slim, given my age (…and okay, my singing abilities too). It’s okay though, I’m just gonna shake it off (I shake it off, I, I, I shake it off). And then make some charts of my own.


Photo Credits (images were cropped and used in graphics)
CC BY 2.0: Taylor Swift (photo by Jana Zills on Flickr), Rihanna (photo by liammendez on Flickr), Britney Spears (photo by rhysadams on Flickr), Miley Cyrus (photo by MelissaRose14 on Flickr), Pink (photo by blumonkey14 on Flickr), Katy Perry (photo by ellasportfolio on Flickr), Lady Gaga (photo by aphrodite-in-nyc on Flickr), Ke$ha (photo by minglemediatv on Flickr), Adele (photo by Christopher Macsurak on Flickr), Iggy Azalea (photo by rarvesen on Flickr), Lorde (photo by Annette Geneva on Flickr)
Public Domain: Beyoncé, Kelly Clarkson
Fair Use: Carly Rae Jepsen (“Kiss” album cover), Meghan Trainor (“Title” album cover), Taylor Swift (“Taylor Swift” album cover, “Fearless” album cover, “Speak Now” album cover, “Red” album cover, “1989” album cover)

October 14, 2014
by Alice Zhao
102 Comments

How Text Messages Change from Dating to Marriage

Way back in October 2008, my now husband and I went on our first date. On our one year anniversary, his gift to me was a Word doc of all of our text messages since our first date (what he likes to refer to as #thegiftofdata). This was especially high tech back in the day (given that we both had feature phones) and what I considered to be the most thoughtful gift ever (given that we are both nerds).

To celebrate our six year anniversary, I decided to take his present to the next level. I took a look at all of our text messages from our first year of dating and compared them with our text messages from the past year as an engaged couple and then newlyweds. I started by looking at the words we used in our text messages six years ago versus present day.

Text_Words2

First of all, we can clearly see that my husband has an obsession with the word “home”. As for me, my early twenties self frequently started conversations with the term “hey”, and more recently I seem to have decided to no longer greet my husband, but instead agree with most of the things he texts me.

I then looked at the frequency of specific terms we used in our text messages when we started dating compared to the past year.

content_final2

Our conversations changed from “hey, what’s up?” to “ok, sounds good”. We stopped saying each other’s names in our text messages. We don’t say in “love” as much anymore. Several words stayed relatively consistent over the years though, such as “home” and “dinner”. I took a look at the actual text messages that contained these words, and found that although the terms stayed the same, the context they were used in actually changed over time.

actualtexts_final

The main difference is that while we were dating, we didn’t see each other every day, so a lot of our communication had to happen via text. We’d often message the other person to see what they were doing or tell them that we were thinking of them. As a married couple, since we’re together all the time, we set up date nights and say sweet things to each other in person, so texting is mostly used to confirm logistics or share random thoughts.

Since the content of our text messages changed so much, I decided to look and see if the time of day we’d send text messages to each other changed as well. I focused on the messages we sent each other during the month after our first date, our engagement and our wedding.

timetext1

While we were dating, we started thinking about each other around 3pm in the afternoon and we’d send each other text messages until 3am in the morning. As a married couple, our texting schedule has pretty much flipped. We text all through the workday and never at night.

timetext2

We see the same story here. As a new couple, since we were apart the majority of the time, we had to check in with the other person every now and then, especially during the evening and late at night when we had no idea who they were with! It was also to tell the other person that we were out late doing something cool without them… and wishing they were there, of course.

As a committed couple, the only time of the day that we aren’t together is during the workday, so that’s when we text. We know exactly where the other person is each evening and if we’re doing something cool, it’s likely that we’re in it together and telling each other about it face to face.

Overall, our text messages started out very flirty and personal. Since we were new in our relationship, we made sure the things we said were interesting and thoughtful. As our relationship progressed, we spent more time together and got more comfortable with each other. Our text messages became more predictable, but only because all of the unpredictable things were said in person. We no longer have to text “I love you” from a distance in the middle of the night. I can now roll over, snuggle with my husband and whisper it into his ear.

To my husband: happy anniversary and I hope you enjoy your 2014 version of #thegiftofdata.