Ep. 99 Exploring Growth Engineering at MongoDB

Michael Lynn: It's the number one goal of every startup, every business. Increase the number of customers, the number of satisfied users. As software developers and engineers, we spend a good bit of time, even before the very first line of code is written, trying to make sure that the end product will satisfy the intended users. We want to make sure it's functional and it'll help them accomplish the goal they had in mind when they came to use it. But, of course, the work doesn't stop at version 1.0, there will be feature requests and enhancements, new functionality, all against the backdrop of an ever- changing landscape of requirements. Wouldn't it be great if, as engineers, we had enough time to continue to monitor our products? How they're being used, how satisfied the users are, how they're interacting with them and how the product might be improved. This area of specialization dedicated to ease of use and product feature utilization has grown into an entirely separate engineering practice, known as growth engineering. Today on the show, we'll spend some time with the growth engineering team at MongoDB. This is the MongoDB podcast, and my name is Michael Lynn.

Jesse Krasnostein: My name is Jesse Krasnostein. I'm the lead product manager on the growth team at MongoDB I've been with the company for just over four years now. So the growth team, Mike, is focused on really the full customer life cycle. I would say from when customers first hear about MongoDB, bringing them in into the world of MongoDB and helping educate them on our products and things that we can do to make their life as a developer easier and more efficient, which I would say is largely the goal of everything that we do at MongoDB. But I think where the growth team differs from every other product team and marketing team is that we tend to be able to spread ourselves pretty broadly across the customer life cycle, listen to where problems are coming from from our users. Whether they're in the acquisition stage of the customer journey, they're learning to use our products for the first time. They're having scaling issues and sort of help direct resources accordingly to ensure that we can help more customers overcome any issues in a specific part of the funnel of the user journey.

Michael Lynn: Now, from what I know about growth engineering, the skillset required are part engineer, part user interface designer, part graphic designer. But I think the most important skill to me seems to be a curious mind. I asked Jesse to talk about his background and how he got interested in this area of specialization.

Jesse Krasnostein: How did I get involved in this area? So I mean, my journey to MongoDB is actually quite interesting. When I was at university some time ago at this point I was in a side job. I was working at a restaurant in Australia and we faced this problem and together with someone else that I was studying at school, we built this really small application. And we used that in the restaurant and we built it in, I think in six weeks. We were fairly inexperienced at any form of software engineering or database development. We had been studying maybe what you would call a competitive technology at the time. But someone had suggested this new thing out of the US called MongoDB. And we built this app on MongoDB. And I can't tell you how fun it was and how impressed it was that it was just so quick to get started with, within 10 minutes, we had data being stored in the database. We were figuring out the data model as we went along, we kind of knew the end goal of the product. That was 2011, maybe. From that moment on, I think I always held MongoDB sort of in its own category of products, just in how wonderful it made my job as building something to solve a problem. And from then on I finished university, ended up moving to the US, did another startup, also did that with MongoDB. And then when that didn't actually work out, I was at a tech conference put on by a venture capitalist in New York City and was fortunate enough to meet one of the early hires at MongoDB. And I talked a little bit with that person. And pretty shortly thereafter, I got an introduction to someone that was hiring and looking for someone with a mix of computer science and a business background. I kind of fit the bill. And it was in an area of the business that I was fairly green in. It was sort of product marketing and go- to- market. It was area I wanted to grow in and it was to focus on sort of the self- serve developer experience. That's a massive channel for us in terms of engaging with the market. We have really many, many, many thousands of developers that come to Atlas without any form of assistance from support, from a salesperson. And we really, the business wasn't serving that user base in sort of any particular way at the time. And so I started there and pretty quickly saw an opportunity to sort of actually evolve the product experience itself, not only just the marketing experience. And that was sort of the beginning, at least from where I stand of how I got involved with growth at MongoDB.

Michael Lynn: Now, the funny thing about growth engineering is you're not necessarily going to know when you're interacting with the product, whether you're interacting with the growth team or whether you're interacting with part of the product that's been in place for some time. Now you may notice slight changes and you may actually be participating in what they call an AB test. I asked Jesse to explain how users might see some of the work that the growth engineering team does.

Jesse Krasnostein: A developer would encounter the work of the growth team. After four years of working on growth at MongoDB, I would say at almost any point in the customer experience. We've been lucky enough to grow from a really small group of individuals partnering with our documentation and education teams. Our acquisition teams work on marketing across all varieties of channels. When you see an ad for MongoDB on a blog, or maybe even when you see some content that's been... In some way shape or form growth has probably been involved at some point in optimization of the experience in finding some sort of insight that maybe led us to create a certain type of content or a certain type of ad, or reach out to customers in a certain way through to when you come and sign up for our fantastic products. When you go into Atlas, you're creating a cluster the first time, that was one of the biggest problems we were hearing when the growth team first started was how complex it was to configure a cluster. And we had enterprise users of MongoDB who obviously understood all of the terminology, the jargon, and were able to make decisions having a lot of experience behind them. But then we had users coming from high school into our cloud products were faced with the same questions that someone with 20 years of being a DBA might face. And we realized, not all of those things are relevant to everyone. So we started to figure out how we could optimize the experience, make it simpler for those that need it to be simpler. So they can also learn to develop with MongoDB and keep all of the Opt- In complexity for those that want to be out of control, all the dials themselves as well. That's a good example.

Michael Lynn: Okay. So Jesse's been here from the beginning, help to grow the growth engineering team. I thought it might be interesting to add a different perspective, someone newer to the team, newer to MongoDB

I'm Akshay Sadarangani, and I'm a Software Engineer 3 in the Atlas growth team here at MongoDB,

As we learned from Jesse it takes a special skill set, it's a really good mix of things that you need to know in order to be effective in growth engineering. I talked to Akshay and asked him about what it is in growth engineering that he finds compelling, compelling enough to take a role in this specialization.

Akshay Sadarangani: So the good thing about being in growth is that we have multiple things happening in a typical day. So maybe I could be working with our product and analytics team to work on some experiments which are user- focused. We're trying to get the user the best experience possible that they can have, or we could be working with cross- team collaboration projects, such as upgrading our JavaScript projects or just working with various products, cross- team collaboration, cross- product collaboration, maybe honing our skills, doing some R and D work, trying to get what the industry is heading towards and upgrading our knowledge sets so that we can provide our skills in the work that we actually do in our day- to- day.

Michael Lynn: What is the goal? What's the mission of the growth team?

Akshay Sadarangani: I would say that if I had to give a one- line tagline for growth, I would say that we work hard so that our users don't have to. And the long explanation for that would be, we strive to get our users the best experience that they can get using our products. So all the way from onboarding to say, connecting to your first cluster at Atlas. So what we do to attain that is we conduct a bunch of experiments where we try to compare our current widgets with alternatives. So let's say we do AB testing where the users see what they see in a live production environment. And some users may be allocated randomly to an experiment where they see a slightly different version of the same UI components. And then we try to gauge, okay, which one is better for our users. So we have some statistical analysis going there. We have the rate of success, or sometimes failure as well, so that we can make informed decisions about providing our users with the best experience.

Michael Lynn: What are some of the tools that you use to capture user interactions?

Akshay Sadarangani: Sure. So actually we use a tool to capture the analytics events that we have. We also have our at- home tool dashboard called Folta, where we have all the metrics regarding an experiment where analytics can go in and check- in which experiment is performing better, which widget is better and make decisions based on that. We do have our data science team as well, where we have our data being stored in MongoDB. So they do use the NoSequel warehouse to get the data that we require to make these decisions as well.

Michael Lynn: Yeah. And what types of resources were made available to you to kind of bring you up to speed on the products and Atlas and MongoDB in general?

Akshay Sadarangani: I have been using MongoDB for a while. I started in grad school and I was actually one of the point people who got the MongoDB sales team in touch with the acquisition team. So I was actually helping facilitating those conversations. So I was already familiar with the product, but we do have a course called NHTT, which is New Hire Technical Training, which everybody in the firm has to go through. And that's actually pretty good in getting people up to speed as to what the product does, how to use it. You actually need to give these quizzes, which you need to get, I think, above an 85, 90% to pass. So they make sure that you get up to speed with the product.

Michael Lynn: Yeah. How did you get interested in tech and software in general?

Akshay Sadarangani: Oh, well I think since I was a kid I was doing this, so it's been so long. I've been in the tech space for a very long time. I actually think maybe the initial scope was when I was, I think in my third grade, when we got Computer as a subject, as a new, the next big thing that's going to happen. And it was only available in school, multiple students would share one computer system. It was so limited and scarce, but it was so fun when we would get those 10, 20 minutes to play with it. It was so much fun that I actually came back home and I asked my dad to get one at home. And my dad through the difficult, he did actually end up getting one. And that's where my interest just spark. I literally stopped going outside to play as much. And I started spending more time indoors, trying to explore what's actually going behind the scenes on a computer. So I think that's what piqued my interest. That's where I've got into the tech space and I've never been happier. I've never wanted to leave the space ever since.

Michael Lynn: So you get interested in computers, you start studying it in school and you study software in university?

Akshay Sadarangani: Yes. I was a Computer Science Major, actually. I also got a master's of Science in Computer Science from Northeastern University Boston. So that was my transition from moving from India to the US was through, grad school, actually.

Michael Lynn: And then prior to Goldman Sachs, were you working as a software developer before

Akshay Sadarangani: I was, but at a lower capacity. So I was a what we call a Web and SharePoint developer. So, which was a very niche line of business. I was in it for two years and that was something that I actually picked up on the job. It was definitely not something that was being taught in school, but it was something I picked up on job and yeah, it was good fun while it lasted, but I...

Michael Lynn: What do you think the key components that make software engineers successful? What are, what are those key things that make people successful in the software engineering space?

Jesse Krasnostein: I think problem- solving in general, having the curiosity to not just solve but also keeping on learning. It's okay to not know the answer to a particular question, but you should have the thirst to get there, to figure out how to get there by any means possible. I think as a developer, I speak for the entire community that we spend a lot of time on Google on Stack workflow. And it's not like if my manager sees me spending time here, learning these resources or looking up how to code or something, he's not going to be mad at me. He knows this is the way, this is the right way. You look at the documents, the resources where to find them, and then you figure out how to do it. So I think it's not about knowing the answer immediately. It's about that want to get there and to be able to get there eventually.

Michael Lynn: What drew you to growth as an area of focus?

Akshay Sadarangani: Sure. So I was actually contacted by the recruiter and when I was pitched the idea of growth and what we actually do in the Atlas growth team, it piqued my interest. What I really like about this space is that you can see what the users want, and it's pure math statistical, so you don't have to rely on intuition. You don't have to wait for, I don't know, some miracle to happen. You can actually see the maths. You can see, okay, this is what users like, this is how we can help them get the best experience. And at the end of the day, when you have a successful experiment and you can make the changes in production and you know that you're contributing in a significant way to make your users get the best experience that they can possibly get.

Michael Lynn: How large is the growth team? How many people are you working with directly?

Jesse Krasnostein: So the growth team has actually divided into two small what we call squads. Squads are nothing but multiple teams come together to work. So for example, we have Atlas Growth 1 and Atlas Growth 2. These are the two engineering divisions and we have about four engineers each. But we also work with product and analytics and they come together in the picture and we kind of at any time have maybe seven to eight people that we're working. So, it almost doubles up in size.

So as developers, the real challenge is in understanding what the user is going to prefer in terms of an experience. The challenge then exists in the space between the developer and the user experience. How then does the developer close that gap? I asked Akshay to talk about how the Atlas growth team goes about the process of closing the gap between the developer and the user experience.

Akshay Sadarangani: So at Atlas growth, we conduct experiments where we compare different versions of the UI. And we have say AB testing where some users will see a controlled environment, whereas the others would see a variant, which is basically just a slightly different version of what the existing UI is. And then we conduct experiments in a way that we have statistical information out of it. So we gather Intel such as, how much impact does it make to the user? Does the user click a button more in a variant A versus variant B and things like that. So that helps us make informed decisions as to which experiment is better and which user experience is more suitable for our users. So for example, an experiment that we conducted recently involved showing the users a" Connect" button when they have zero connections. So it's just a slight nudge to the user that; hey, maybe you want to connect to your cluster, or maybe you want to add data to your cluster. So we would show them an" Add Data" button if there is no data currently. So it's all already available to our users, but we're just trying to give them a variant where it's much more accessible, much more intuitive, and much more easier to access, basically. So the way we gauge whether an experiment is successful or not is based on the variant that we provide. So we have two terminologies over here, a control and a variant. So a control is what most users will see in the production environment, which is what the current version is, what the base version is. Where as a variant is a slightly different variant. So it's a slightly different version. The users might see slightly different experience basically. And we're trying to gauge whether this different experiment is better. If this variant is better than what we see currently or not. So the way we gauge this in the way we measure the success for this is through analytics events. So for example, let's say we have an additional button available in this variant. We will capture when the users click on it, whether the user clicked on the button or not. So based on that we can know, okay, more users are engaging in the variant versus the current control version. And that way we can know whether the experiment is successful or not. There are times where there has been an experiment may not be successful, whether the current will is actually doing better. So we won't actually push that to production. We will maintain the current version, but if an experiment is performing better than the current version, we will actually push that and give our users the new code so that they can have the better experience now. Recently, we had something called a three- door experiment where when the users are creating a new cluster in the Atlas UI, they're given three different options, which is a Dedicated, a Shared or a Serverless cluster. And the users can actually, just in the same page, decide which one they want to create. So earlier they would have to go through a different mechanism. They would have a toolbar where you would select this, and just having all three buttons together, just made it a much more easier experience and much more intuitive. And we actually saw a large engagement score there. So that was actually successful. And it's now available in production.

Michael Lynn: Feedback is so powerful and in a way, you're gathering feedback through your experiments, but is there any other way that maybe users can provide feedback on other elements of the user interface or the experience?

Akshay Sadarangani: Absolutely users can actually leverage feedback @ MongoDB. com and just go to Atlas and submit their feedback over there. We're also open to accepting new ideas for any experiments or feature requests or bug requests that are available for us to consider. So please feel free to log into feedback @ MongoDB. com and submit a request there.

Thanks so much to Akshay and to Jesse for sharing their experience around the growth engineering team here at MongoDB. You heard something you liked? If you were inspired to work at a great company with great people, check out mongodb. com/ careers. You can reach out to me on social media. You can find me on Twitter @ MLynn. You can always reach us at, @ MongoDB. Look forward to hearing from you. Have a great day.