Three Components of Research Success

Many PhD students suffer from imposter syndrome, and wonder whether they are “good enough” to belong. I never had this problem. Coming out of my PhD, I felt confident that I was good at research, and filled with ideas for new projects.

Years later, most of those projects have never been started. My job market paper languished for years before I submitted it to a journal, and it took more than another year for me to complete the revision. This has been a source of anxiety, and caused me to question my earlier self-assessment: maybe I am not so good at research after all!

What I have come to realize is that the research process requires several very different skills.

• Discovery. This is the process of writing down models, identifying interesting questions, coming up with conjectured answers (through examples or simulations), and then proving (or disproving) these conjectures.
• Dissemination. Work is shared through papers and talks. You need to convince experts and non-experts that your questions (and answers) are interesting. This requires clear writing, simple examples and diagrams that convey your main ideas, and giving many talks.
• Management. You are your own boss. You need to decide which projects to work on, and in what order. You need to estimate how long tasks will take, set deadlines, and determine when your work is ready to submit.

Self-Diagnosis

When I self-identified as “good at research,” I was primarily thinking about the discovery process. I love thinking about new models, and reasoning through their implications. I generally have fairly good intuition, and am creative enough to generate both conjectures and proof strategies.

I’m also fairly good at dissemination. I enjoy giving talks, and coming up with simple ways to explain ideas. Although I can be a slow writer, I think that the final output is usually well-organized and clear.

I’ve come to realize that I really struggle as a self-manager. My tendency is to work on whatever project is most exciting, which leads to many started projects and few completed ones. Furthermore, I’ve never developed the habit of breaking large tasks into bite-sized ones, estimating completion times, or setting daily goals. I can be a perfectionist, and tend to revise repeatedly, with little awareness of the associated opportunity costs. At Stanford, I was aware of some of these challenges, and would often tell prospective students that the hardest part of moving from undergrad to grad school is no longer being told what to do and when to do it.

Despite this, I felt successful in graduate school. A lot of the credit goes to my two phenomenal advisors. Meetings with them forced me to set short-term goals, and they helped me decide when work was polished enough to move on. Another contributor to my success was my passion for learning and sharing ideas with others. In early stages of a project, I often get caught up working through the night, hardly noticing the passage of time.

A Challenging Period

When I joined Columbia, much of my work consisted of putting finishing touches on (a.k.a. publishing) ideas that I already understood fairly well. I felt like I needed to do this work, but wasn’t learning much from it. Furthermore, my job market paper was solo-authored, so I couldn’t rely on conversations with others for inspiration. I frequently felt like an extrovert in an introvert’s job.

No surprise, I gravitated towards parts of my job that involved learning and interacting with people. Through the amazing Deming Center, I got to visit businesses across the globe, and do a little consulting. I poured myself into thinking about the class I taught, and I started a project with Peng Shi that remains my favorite collaboration to date.

Good things came out of these efforts, but they also allowed me to put the task of finishing my solo-authored papers on the back burner. This fell into the “important but not urgent” category. I didn’t have a mentor or advisor to help me set and stick to deadlines, so when I did work on my papers, I tended to polish small details rather than focusing on the most important tasks. In addition, a lack of passion and deadlines meant that I often found excuses not to work during time that I had nominally set aside for writing.

My failure to submit my job market paper for review started to affect my happiness. Even when I wasn’t working, I felt a level of anxiety and inadequacy. Furthermore, the delays only heightened my own expectations: given that I’d taken so long to finish the paper, I had to make sure that it was perfect! Completing my paper seemed like an ill-defined and ever-more-daunting task, causing me to procrastinate further. And I started to ask what was wrong with me, and why I couldn’t just hit “send.”

This blog in part grew out of that frustration. While research papers seemed never-ending, a blog post (like a lesson plan) can be prepared on a shorter time horizon, and offers a sense of accomplishment that was otherwise missing from my life. Furthermore, I hoped that blog posts could reach a larger audience than research papers, and inject more intellectual conversations into my life.¹

I don’t want to make it sound like the last five years have been all bad. In that time, I have learned a lot about business and operations management, developed a much clearer research identity, begun mentoring students, and even published a number of papers. But overall, they were probably the least happy years of my life, and I know I can take steps that will make me both happier and more productive.

Some Good News, and Lessons for the Future

2021 has actually come with a lot of good news on the research front! Four of my papers have been accepted to journals. Three of these are in Management Science, and two of them are solo-authored. I think that these successes have made me more comfortable with discussing periods where I felt less successful.

I am more humble than I was five years ago (which, to be fair, is not saying that much). While I am very good at parts of the research process, I recognize that I am not so good at others. Moving forward, I hope to play to my strengths, while also trying to address my weaknesses and maintain perspective.

Take Advantage of my Strengths. I see my enthusiasm for ideas as my biggest strength. I want to create an environment that exploits this, which means more research conversations. I want to give myself permission to have these conversations even if they won’t lead directly to papers. I am most successful when loving my work, and sharing ideas with others is the part of work that I love the most.

Another way to take advantage of my strengths is to find collaborators! Although I have proven that I can write papers on my own, it’s a lot less fun (especially during revisions), and I think ideas are improved through discussion. Furthermore, I can find co-authors with different strengths: my collaboration with Peng Shi was so successful in part because he is so effective at structuring and managing time.

Develop Habits to Improve Self-Management. Even if I work with others, I will need to get better at managing my own time. A big part of this is breaking up tasks into pieces that can be accomplished in an hour or an afternoon, and setting daily, weekly, and monthly goals. Doing so offers several advantages:

• It helps me “keep my eye on the prize”, ensuring that I spend my time on true priorities.
• It makes it possible to feel a sense of accomplishment on a regular basis. When my only goal is “work on paper X”, it is hard to see progress. This can be demotivating!
• It provides a sense of urgency, without which important tasks can slip.
• It forces me to clearly define the scope of what I hope to accomplish, helping to combat my instinct to always prove one more result before submitting.
• It makes opportunity costs of honing a particular sentence or figure more apparent, enabling me to combat my perfectionistic tendencies.

I’m also trying to limit multi-tasking. In 2020, I worked on seven different papers – often three or four in the same week! This is way too many. I found that it is more effective to dive deeply into one paper, and work on it until it is complete. This isn’t always possible, but I think it’s the right goal to strive for.

I thank my wife Amanda for helping me see these things more clearly, and for her patience listening to me reflect on work (and digging into the content of my papers!). Some of my recent success completing projects is clearly attributable to conversations with her.

Maintain Perspective. Most academics I know have very low-stress jobs on a day-to-day basis. Some (myself included) have a lot of unscheduled time, at least when they are not teaching. Others lead much more scheduled lives, but generally set these schedules themselves, and nothing disastrous happens if they have to cancel a meeting.

Despite this, many academics suffer from more long-term existential stress. We live in a “publish or perish” mindset, worry about whether we will get tenure, and have difficulty taking our mind off of work. We question whether what we’re doing is valuable, and whether we are good at it. When we do have success, rather than celebrating it, we immediately start thinking about the next task on our list.

For the most part, I think I experience less stress than many of my colleagues. However, in recent years I have noticed a significant uptick, which is affecting my enjoyment for work, as well as my ability to be present when not working. This increased stress is primarily internal, and does not stem from meaningful changes to my responsibilities.

While some people find that stress is motivating, I find that it causes me to set aside more time for work (foregoing experiences that I would enjoy) while simultaneously working less efficiently – either because I switch between tasks without a clear plan or priority, or because I try to avoid whatever task is causing the anxiety. While there will always be times when I feel like I have too much to do, I can choose how to react to that situation. It’s important for me to keep a forward-looking and solutions-oriented attitude: if I don’t accomplish everything that I hoped to, the right reaction is to create a new plan, rather than to spend time berating myself.

The reality is, I am amazingly lucky! I have a job that provides a lot of freedom to decide what I want to work on, and when. It also pays enough that money is not a concern in my life. Over the coming months and years, my resolution to myself is to enjoy and appreciate what I have, while making sure to fill my week with activities that I find enjoyable and fulfilling. I hope that you have the opportunity to do the same.