Visual QA - Fall 2018 Report

Retrospective Analysis of VQA Project (December 2018 )

We have received best poster for a project this semester. This is a retrospective analysis on what we think went well. We have tried to keep this non technical so that the hypotheses and conclusions become generic.

There are 2 reasons for writing this.

1. Firstly, we believe we understand the difference between lessons taught by failure vs success. Accompanied by a thorough audit, failure typically teaches us what not to do. Success on the other hand, has the potential to teach us appropriate principles and path way that we can use for future endeavours. We are interested in growth mindset and hence believe it is important to audit the reasons we believe we succeed in any effort.
2. Secondly, we believe that our efforts are going to result in medium to long term dividends. In other words, even though our semester is done, we believe that we are yet to fully reap what we have sown. Therefore, it makes sense to take a holistic view and consolidate the thought process before stepping up to the next level.

With these in mind, here are the top 5 concepts that we believe helped us. Most of these are obvious. But we aim to highlight some non obvious components of the obvious concepts.

1. Giving your best :

   This is an easily misunderstood concept. Hence we believe it is important to understand what it means to give the best. Our favourite version among many explanations is something close to how Katrin Davidsdottir puts it . In our context, this translates to giving everything to the project not in the final week, not even every single month/week/day, but giving 120% to every single script we write; every single technical approach we speculate about; every design choice we make. We are not interested in putting out sub optimal performance in any metric. If working as a team, we are not interested in picking our battles. We want to fight them all. To realize this, we have divided our project into phases and made sure that each phase received our best possible collective effort. For instance,

   • During the literature review, we have curated and annotated a tiringly extensive list of papers.
   • During our engineering phase, we architected our scripts to run at ridiculous speeds.
   • During our R&D phase, we went overboard with a topic that transcends our current project.
   • During our experimentation phase, we built probably the most number of systems compared to other groups in the class.
   • We maxed out the page limit on every report we submitted

2. Switching Priorities :

   Setting priorities is the obvious part. But as we progress in the project, we need to have the flexibility to switch priorities. For instance, let's consider in person meetings. Early stages typically need a lot of brainstorming and bouncing off ideas. This will be the time to have each person physically present. Meetings typically tend to be longer. Since people are usually less occupied early in the semester, this is feasible too. But it seems that there is a point when this has to switch. It is important to realize that as project end date approaches, the meetings need to be taking up as less time as possible. In other words, it typically is not a good sign if we are still spending a lot of time meeting and brainstorming towards the end of project. The concept itself is applicable at every phase of the project: When to stop focusing on R&D and up the ante on engineering, when to target rapid experimentation vs first principles thinking, etc.

3. Team Size and Innovation vs Execution:

   For this project, we followed the strategy of working in 2 person teams all through. Working in small teams has some obvious benefits like promoting a sense of empowerment and accelerated decision making process. But we believe that the biggest ROI is the accomplishment of harmony between innovation and execution. As Marissa Mayer points out , innovation and execution are natural opposites. Hence separating the team specifically such that both are fed leads to miraculous results. For instance, in our project this separation meant we had extremely lightweight scripts ( execution and engineering) which we used to test and deploy super cool ideas ( innovation) within the limited GPU budget we had.

4. 1 + 1 = 11:

   We didn't realize it fully in the beginning but when we formed our team, we essentially created what is usually referred to as Mastermind Alliance(MMA) . MMA is different from a peer group. Napoleon Hill, posits in his book Think and Grow Rich that when two people work together, a third mind is created and names this the MasterMind. MMA is when there are more than two people involved. MMAs are usually commonplace in the entrepreneur circles but they can be forged in academic setting as well. However, there are some caveats. In a typical industry setting, collaborations are top down; Manager forces them. However, when we talk about MMA, they are usually bottom up: We forage them. Therefore, there is no middle ground in MMAs: They are either phenomenally awesome or lead to an awful experience. Forcing an MMA with wrong peer group might affect the personal relations as well. Essentially, the success of an MMA depends on the incentivization strategy applied and the leadership skills deployed. It has to be noted that leadership in MMA is not static but dynamic: Alice leads the project in one phase while Bob leads in another. One more thing to keep in mind is how we approach when things go wrong in such MMAs. Given that MMAs are usually formed among alpha type personalities, the default reaction when things go wrong is to try and remedy the situation aggressively. Since each participant in MMA has a set role and no one wants to slack off, this approach might sometimes lead to intimidation and other negative effects on the individual. However, it helps a lot to deploy patience and learn how to control the situation. When done appropriately, it not only builds character in the person who deploys patience, but also renders the recipient more accountable. This has the potential to give asymmetric returns in the long run.

5. Chasing good vs Chasing Excellence:

   Ivanka aptly highlights the importance of attention to detail . We were obsessively inclined towards excellence and the last 5%. This meant we spent extra time on the fit, detail and the knitty-gritty: How we name our approaches, How we design our poster, etc. However, the important thing to realize here is not the fact that we cared about the last 5%. We understand that no one gets up and decides to put in a mediocre effort. The punchline is to establish excellence as first class object in the architecture of the project, specifically allocate time and energy towards it and accelerate parts of the project such that we don't compromise on the last 5%. It is hard. But we love doing hard things, don't we? :)