Ep. 111 Kunal Lanjewar of That Game Company at GDC

Kunal Lanjewar: Hey, I'm Kunal Lanjewar here at the Game Developers Conference 2022. At thatgamecompany, we are leveraging MondoDB's Atlas manage service, so we can focus on Sky and let Atlas take care of the operational stuff for us.

Michael Lynn: Welcome to the MongoDB Podcast. My name is Michael Lynn, your host. Today, we're talking about gaming once again. Had the distinct pleasure of being at Game Developer Conference 2022 a couple of weeks ago. It's where I met Kunal from thatgamecompany. You remember thatgamecompany, that groundbreaking game called Journey just caught my attention. It's an anonymous adventure, where you travel along a life's passage with the opportunity to connect with companions along the way. Really fantastic game. Stay tuned to hear more about thatgamecompany, and how they're leveraging MongoDB. MongoDB World, June 7th through the 9th, coming to New York City. It's an amazing conference for developers by developers. You can register today, visit mongodb.com/ world. Use the code podcast, P- O- D- C- A- S- T, to get 25% off of the ticket price, as well as some interesting swag. Hope to see you there.

Kunal Lanjewar: My name is Kunal Lanjewar. I'm a senior software engineer focused on the back end of Children of the Light game. And I currently work for thatgamecompany.

Michael Lynn: So thatgamecompany, it sounds familiar. What games have they produced?

Kunal Lanjewar: Couple of games that come to my mind is Flow, Flower, and the most critically acclaimed was Journey.

Michael Lynn: Journey, that's it. So Journey had this amazing interface, it was kind of unique when it came out. Do you want to talk a little bit about Journey?

Kunal Lanjewar: Sure, sure. So, Journey was this amazing experience where at least for personally me, it was extremely awesome looking, it was really amazing. It was a very unique design and art style. It was extremely calming. And when you get in the game, you don't know what you want to do. There's literally nothing telling you to do anything, you have to kind of figure out yourself. That's something that was really unique about Journey. But then there was this amazing aspect about Journey, that a random person would join your game session. And the person almost felt like an AI, but it turned out it was actually a real player. And the only way to communicate with that player was through a call sign. You cannot talk to them, you cannot send them a message. It was only through a call sign, literally honking in the game. And it tested, it basically tested the bond between the players, on how we play with each other together to get to the end of the game. So, that was an extremely amazing aspect of Journey.

Michael Lynn: Yeah, love that. And so we're here at the Game Developers Conference, and how are you enjoying the show so far?

Kunal Lanjewar: It's been fantastic. I'm just glad to meet people. It's been two years, I haven't seen a lot of people in one place like this.

Michael Lynn: It's so good to be back.

Kunal Lanjewar: So this is definitely, definitely good to be back. Yeah. This is also my first time at GDC, and I'm just having a blast here.

Michael Lynn: Yeah, yeah. Fantastic. Have you seen any sessions, listened to any talks?

Kunal Lanjewar: Yeah. I've seen quite a bit of talks here. And I was mostly interested in the Online Technology Summit because those were the backend related talks. So, I found the EA talk was pretty awesome, I found there was a Pragma talk that was pretty interesting, about how they solve interesting problems. And yeah, I'm looking forward to the MongoDB talk tomorrow.

Michael Lynn: Me too. Yeah, so Sig and Jim will be delivering that tomorrow at 12: 45.

Kunal Lanjewar: Perfect, perfect. Yeah.

Michael Lynn: So tell me a little bit about, you're focused on the back end at thatgamecompany. What are you working on today?

Kunal Lanjewar: So, I've been leading a lot of our microservices architecture initiatives, and also creating a cloud native platform where we can host all of those microservices. And create a platform such that is scalable, cloud agnostic and vendor neutral. So, that has been what my focus at thatgamecompany is. Apart from that, I also do some gameplay engineering work, because I think for me to be a good backend engineer for a game, I also need to understand how the game works. So, I get my hands dirty in the game code as well.

Michael Lynn: Yeah, so you keep your skills fresh.

Kunal Lanjewar: Yeah, exactly.

Michael Lynn: So, how are you using MongoDB at thatgamecompany?

Kunal Lanjewar: Sure. So Sky primarily used postgres at the back end. And Sky's backend application-

Michael Lynn: Sky is the game?

Kunal Lanjewar: Sky's the game. And its backend primarily had an analytic application, which will then connect to a single shard postgres. And the problem we ran into is when we started to hit a large scale at Sky, we needed to basically horizontally shard the postgres. And we were actually a very small team, and sharding postgres is not natively supported. And we basically needed some solution that could basically either shard postgres for us, or the other option is try to find the highest QPS, non- relational read write request that are going to the database, or postgres in this case, and move them out to a non- relational database. So, we could then feature shard some of the services. And that is why we ended up basically picking MongoDB, because we had a lot of the data that was non- relational in nature, and had highest amount of rights. And then MongoDB came out to be a good database.

Michael Lynn: Fantastic. And what is the actual data that's being stored? Is it player engagement data?

Kunal Lanjewar: Sure. It is literally gameplay data. So player data, meaning one of the examples I can give you is our checkpoint service. And our checkpoint microservice actually stores a lot of the game save data, if you will, into MongoDB. And it is extremely non- relational. We don't need any kind of SQL query to do joins or anything like that. It was literally just write and forget, and then read exactly-

Michael Lynn: Key values?

Kunal Lanjewar: Pretty much, yeah. Like an object with a key on it, basically.

Michael Lynn: And then are you doing any kind of telemetry data or analytics? Or what other types of data are you...

Kunal Lanjewar: For MongoDB specifically?

Michael Lynn: Yeah.

Kunal Lanjewar: We're actually not doing any telemetry data in MongoDB right now. We are literally using MongoDB as a player data database, where we don't need to do crazy SQL queries.

Michael Lynn: What's it look like from a scale perspective? How much data are you storing?

Kunal Lanjewar: Yeah, we have terabytes of data.

Michael Lynn: Terabytes, okay. Wow.

Kunal Lanjewar: We do. And another pretty cool use case that we recently found that we could do, is now that we have sharded our data into MongoDB and postgres, because our relational data is in postgres, our non- relational data is in MongoDB, we had to do some distributed transactions between the two. And what made it pretty interesting with MongoDB, is that we could hook up something called a change data capture for MongoDB. It would then basically tell the op log, we would then take those events, put them into Kafka. And from Kafka, we could use Saga choreography pattern to do a distributed transaction that would eventually succeed. So, write to MongoDB, get that event into a change data capture like Debezium, or we are using the Atlas connector in this case. And then take the Atlas-

Michael Lynn: The Atlas connector for Kafka?

Kunal Lanjewar: For Kafka, yes. It's a source connector. And then that would take that event, write that, or commit that into Kafka. We then have a consumer group inaudible that Kafka topic, it would then commit into postgres. So as long as it eventually succeeds, we are okay with that. And it does.

Michael Lynn: Fantastic. Well, that's a great architecture.

Kunal Lanjewar: Yeah. That basically helped us do inaudible on this really transaction, then we were to the races.

Michael Lynn: How did you get into game development? How did you get into the industry, and software development in general?

Kunal Lanjewar: In software development in general, I was, since my childhood I was pretty involved in computer science or programming in general. I guess the first language I learned was DOS prompt.

Michael Lynn: Oh yeah.

Kunal Lanjewar: Yeah, so I go back then. I'm not that old, but still. So I was always interested, and then my vocational training was in electrical or electronics engineering. And then I did a lot of mechatronics as well, so mechanics and robotics. And I was always interested in programming because of that, since a child. I think I was 11 years old when I did my first HTML page or something, back in the day. So, that's how the journey started. And I got a sweet gig at NASA and I worked at NASA, or NASA Projects, lunar reconnaissance, orbital camera. I was also at the JPL, I did the software there.

Michael Lynn: Jet Propulsion Lab?

Kunal Lanjewar: Jet Propulsion Lab, yeah. And from there, I got to work in a cloud based company called Circle with Disney. And that is where I learned the real good stuff about backend, microservices, cloud native architecture, AWS, Terraform, vault, all that cool stuff. And yeah, that's how my journey began. And the way I landed a gig at TGC was, or thatgamecompany, was they were looking to modernize a lot of the backend architecture. And they needed someone who can come in and do this kind of thing. That has been my primary role here.

Michael Lynn: What do you love about thatgamecompany?

Kunal Lanjewar: I think the people, I absolutely love the people. I also absolutely love the games. I was a big fan of Journey. And I still remember literally crying when I finished the game at the end. There was an emotional attachment with the company.

Michael Lynn: Yeah, it's a unique interface. How does that come to be? Who's responsible for that level of design and art?

Kunal Lanjewar: We have some of the industry's best designers at play here. And I think a lot of credit also goes to our CEO, creative director Jenova Chan. It's basically his vision that he wants to bring people together emotionally, connect them into a game. So, a lot of direction comes from him. And he then brings a lot of these amazing artists and designers together. So, it's all about bringing awesome people together to get something awesome out.

Michael Lynn: Yeah. That's a lot like MongoDB, no wonder that we work well together.

Kunal Lanjewar: Yeah, yeah, yeah. I've had nothing but amazing experience chatting with a lot of the SAs from MongoDB. Every time we have some kind of fire in the backend, you guys are always there to just quickly jump in and help us solve it.

Michael Lynn: Fantastic. I love to hear that. It's unavoidable. There's going to be times when the user activity level spikes beyond what's expected. Speaking of that, are you leveraging any type of automated scalability? Well, I guess specifically with MongoDB. MongoDB and Atlas gives you the ability to automatically scale, are you leveraging that?

Kunal Lanjewar: We are leveraging that. So we scale up based on the IOPS, we scale up based on the CPU and memory. And MongoDB will basically send us an alert that," Hey, this specific cluster is hitting the peak, 90% of the CPU limit that you set, and we're going to be scaling it up," basically. And then we get those notifications in Slack, so we're also alerted inaudible. But MongoDB, the good thing about the Atlas which is managed part of MongoDB, is that it really takes away a lot of the operational aspect from our hands, and we can focus on the backend features. So, a lot of the operations is taken by the Atlas platform. Scaling is another one, the biggest one is sharding and rebalancing clusters, is the amazing one that we don't have to ourself, things like that.

Michael Lynn: That's great. Is there anything else you'd like to let the audience know about maybe thatgamecompany, yourself, or MongoDB to be in general?

Kunal Lanjewar: About thatgamecompany, sure. We have an amazing team here. We're trying to do something really, really interesting that no one else is trying or have tried. And if you would like to come work for us, just send your resumes, we'd be happy to take a look at them.

Michael Lynn: How can they get in touch with you?

Kunal Lanjewar: They can get in touch with us, there's a career page and they can apply there as well.

Michael Lynn: And what's the domain? Thatgamecompany-

Kunal Lanjewar: Thatgamecompany. com/ careers, people apply there. They can always reach out to me, or anyone on LinkedIn, so that would be a way to do that. Other than that-

Michael Lynn: What kind of experience are you looking for?

Kunal Lanjewar: We are hiring in all kinds of spectrum here. All the way from gameplay engineers, designers, artists, customer support folks, backend folks. Specifically in my team, I'm hiring for backend engineer, and devops or SRE engineers as well. So, if any of them are interested, send your resume to me. Hit my LinkedIn and send me a message.

Michael Lynn: Absolutely. We'll try and drum up some interest there. Where would they need to be located?

Kunal Lanjewar: We are pretty flexible. So, thatgamecompany now is a remote first company, if you will. And we can chat. If you're out of country, we can chat about that. We have history of looking for people within the US and Canada region, just because the time- wise, it works out better for everybody. But if you are super passionate and if you're out of the country, I don't mind chatting with you as well.

Michael Lynn: Okay, fantastic. Kunal, thanks so much for taking the time to talk with me. It was a great chat.

Kunal Lanjewar: Thank you. It was awesome. It was my pleasure.

Michael Lynn: Right on.

Kunal Lanjewar: Have a good one. Thank you.

Michael Lynn: Thanks to Kunal for joining me today. Thanks to you, the listeners. Make sure you head on over to mongodb.com/ world. Check out all of the things happening June 7th through the 9th at MongoDB World 2022. If you want to register, you can use the code podcast, P- O- D- C- A- S- T, to get 25% off your tickets and some cool swag while you're at it. I hope to see you there.