The post-mortem: how small businesses and startups can learn from their mistakes Mike | January 18th, 2010
As you may be aware, last month we launched our fully re-factored site after many months of effort on the part of the team. Not only did this re-launch mean a 100% new code-base, it also meant 100% new hardware at our hosting provider. Our goal was a faster, more stable site, and a code base that allows us to add new features and enhance existing ones quickly and easily. Our community is already benefiting from this: along with the improvements introduced last month, the site’s performance and site stability are greatly improved; in the 30 days since launch our site uptime has been 99.9%!
As good as it felt to launch, the process was not without pain. Wait a sec – what am I saying? What I meant was, as good as it felt to launch the process was intensely painful for the entire team. Many long days, some heated disagreements, more than a few all-nighters, way too many bugs at launch, many customers impacted (and frustrated), and a huge spike in customer service requests served to drain the team, reduce our capacity, and destroy our productivity for weeks. We have emerged from this process older, wiser, more tired than ever, but having learned some truly valuable lessons.
We took some time last week to meet as a team and spent the better part of a day doing our own “post-mortem” on the process. Our goal was to come away with some lessons learned and to use these to inform our internal process and to improve our performance going forward. In 2010 we will be introducing some major new features and we hope to better execute these “mini-launches” and lessen their impact on the team and the community. One of the thoughts that occurred to me as we reflected on our own mistakes and developed our own learnings, was that other businesses and organizations might benefit if we shared our own process.
Within a few days after launch, we put a meeting on the calendar for the entire team: Lessons Learned. We decided that this meeting would not take place for a few weeks, because we knew that problems were still arising in those first weeks and we wanted to be able to discuss ALL of the issues and all of the consequences which were revealed. This was important, because it wasn’t immediately evident where some of the issues lay and we knew that, given enough time, they would be uncovered.
We structured this meeting in 3 parts: 1) identify pain points; 2) identify “themes”, and; 3) develop “lessons learned.” It was important to us to give the meeting a structure and this 3-part approach allowed us to focus on the process, but also to give everyone a voice and some ownership of the meeting itself. We set out some ground rules at the start: everything was fair game (meaning that nothing and no one would be exempt from critique); everyone was expected to contribute (we wanted to understand the different pain each individual experienced); and finally that accountability would be a focus. We determined that the best outcome would be to identify a small number (5 or 6?) important lessons and use these to inform and improve our process going forward.
We gathered in our conference room, which has two large whiteboards: 12 running feet of room for thinking. notation, and charting. First we agreed that we would break the process up into a chronological sequence of phases: planning, pre-launch, launch-day, and post-launch. We would look at each of our functional areas: management, software development, customer service, and marketing/PR. This framework allowed us to work our way thoroughly through the entire process and carefully evaluate actions performed during each of these phases. It hadn’t been planned a such, but this became an all-day process; several hours were spent working through the pain-points from each of the phases; another hour was spent identifying thematic similarities. And finally, boiling these down to a handful of lessons learned took another couple of hours.
Pain-points: some interesting developments occurred during the “pain” part of the meeting. It became clear that the parts of the process that contained the greatest collective pain were the planning and pre-launch phases. As we listed out the individual comments and points, we noticed themes beginning to emerge. We agreed that we would jot down notes on these, but circle back in the second part of the meeting. The board quickly started to fill as we went around the room talking about each individual’s and each department’s experience and perspective. It took a bit, but as we talked folks became more and more comfortable articulating mistakes we had each made and identifying our lapses and the consequences of those. A clear catharsis took hold as we voiced for one another how difficult the process had been.
Themes: these started to emerge quickly, with several becoming evident within the first 30 minutes. Communication problems (both internal and external) jumped out at us quickly. Transparency, quality issues, planning problems, and accountability were several no-brainers, But there were other less obvious themes, and we took the time to comb through the pain-points to reveal them.
Lessons learned: This was harder than we thought it would be. We needed to keep this to a manageable number, so as not to overwhelm ourselves going forward. These lessons needed to be a distillation of the themes we identified, much as the themes were a distillation of the pain-points. We ended up with nine lessons to take forward and apply to our process:
- Keep eyes on the prize; don’t lose track of the big picture.
- Improve communications: restructure regular meetings, remove firewalls, and improve the tools we use.
- Do a better job defining project scope. Always try to reduce scope and simplify whatever we are building.
- Implement faster and shorter iterative cycles.
- Define production schedules early and be accountable for this.
- Hold ourselves to high standards of quality and improve our own QA and testing regimens.
- Be customer-centric: don’t lose track of who we are building products for and the impact our process will have on them.
- Improve the quality of our leadership. Management needs to do a better job enabling. motivating, and communicating.
- Be transparent internally and externally. Make sure that people stay in the loop and know what the other departments are doing.
Well, that’s it on our own post-mortem. The very act of this undertaking has helped to make us a better and stronger team and we hope this will extend to a better and stronger product and a more successful business. The time we spent examining our own process caused pain of it’s own: some deadlines extended, some customer service responses delayed, and some business opportunities put on hold for a day. But, the net result was incredibly useful to us, and if we allow these lessons to inform our process, they should go a long way to reducing the pain next time around.
I would love to hear from any of you who have been through similar efforts or have found good processes of your own for this exercise. In the meantime – happy dissecting of your own efforts!