Summarize it!

End-term exams got over on 9th. I've been looking for an internship ever since (if you're reading this and looking for a machine learning intern, please contact me). Also, I've been going to hackathons! Went to AngelHack Delhi this weekend. I had formed the team a week back. Yask: my junior, Pranu: friend from first year now in MAIT and Ketan: Pranu's friend, I met him at a hackathon a week back.

We decided to do something new and started brainstorming ideas on our WhatsApp group. Since Yask was away most of the time these discussions happened, he had to deal with reading 200+ messages, which he ended up didn't reading. We thought that this is a problem many of us face. If one isn't online for a day or two, and some important discussion takes place on a group, the person misses out on it. This leads to social isolation. Ketan mentioned about Slack, which is a team collaboration tool people at the startup he's interning at, are using. Read this article on Medium, about Slack.

So we came up with the idea of summarizing chat logs on Slack as it has an open API, just one day before the hackathon. I found out that important sentences from a text can be extracted using TextRank. It tokenizes the text into sentences and builds a graph with the sentences as nodes, assigning them same scores. Sentences are then ranked by recursively; a sentence 1 points to another sentence 2 when they have similar features (we used tf-idf). A score is assigned to 2 taking in account the scores of sentences that pointed to 1. Lastly, sentences are ranked in decreasing order. It is similar to Google's PageRank.

Since I've never studied NLP, I didn't know what else to use in addition to this to make the summary somewhat perfect, but as this was just for a hackathon, I thought that TextRank would be nice for an initial prototype.

The hackathon wasn't as expected. According to the site and e-mail, they promised Breakfast of champions on arrival but we were given two samosas and tea xD Games, games, and more games to keep you PUMPED! (Nerf war anyone?), just foosball which was nice though. Awesome food, and LOTS of it, false. Coding through the night (because you never know when inspiration might strike, right?), but the ACs weren't working in the night. Post-event parties!, nada. Swag on swag on swag!, nope.

Anyways, we were able to build a prototype which was working quite well. We also used one of HP's IDOL OnDemand API to extract concepts out of the chat logs. When we tested our prototype on Carl Sagan's famous quote,

it extracted That's here. That's home. That's us. which amazed us xD

We were given just 120 seconds to pitch our idea to the judges which we screwed up. The summary extracted from Slack messages wasn't readable enough as the screen was quite far away from the judge's table. Sponsor prizes were given to the teams which used most APIs from respective sponsors. But, I still feel good as this was the first time I was able to complete my proposed project in a hackathon. So, I've got that going for me, which is nice. :)

Here's a summary of this post given by our prototype:

We thought that this is a problem many of us face. So we came up with the idea of summarizing chat logs on Slack as it has an open API, just one day before the hackathon. A score is assigned to 2 taking in account the scores of sentences that pointed to 1. Since I've never studied NLP, I didn't know what else to use in addition to this to make the summary somewhat perfect, but as this was just for a hackathon, I thought that TextRank would be nice for an initial prototype. which amazed us xD We were given just 120 seconds to pitch our idea to the judges which we screwed up. But, I still feel good as this was the first time I was able to complete my proposed project in a hackathon. So, I've got that going for me, which is nice. :) Here's a summary of this post given by our prototype: A picture of us with Neil Mathew, an intern at IBM, who we met at the hackathon.

A picture of us with Neil Mathew, an intern at IBM, who we met at the hackathon.

[From left to right: Neil, Ketan, Pranu, Me, Yask]