AI and its place in sex ed with the National Director of Innovation at Planned Parenthood Federation of America
AMBREEN MOLITOR
Danielle Bezalel:
Hello, Ambreen, how are you today?
Ambreen Molitor:
I'm well, how are you?
Danielle Bezalel:
Fantastic. I did some exercising in the morning, which I don't usually do. I usually kind of am like procrastinating doing it until like the end of the day, but I started my day with it. So I'm feeling energized.
Ambreen Molitor:
Great, I think I mentioned to you earlier, I brushed my hair this morning, so.
Danielle Bezalel:
You did and that's equally as important i'd say confidence wise brushing your hair, exercising.
Ambreen Molitor:
You went a little bit more, you're way more active, but yeah, appreciate that.
Danielle Bezalel:
So why don't you go ahead and introduce yourself to folks listening and tell us about your work as the National Director of Innovation at Planned Parenthood, okay, I'm gonna do that again. Why don't you go ahead and introduce yourself and tell us about your work as the National Director of Innovation at PPFA, which is the Planned Parenthood Federation of America.
Ambreen Molitor:
Yeah, so in the five and a half years that I've been at Planned Parenthood, I've worked in innovation in different realms. So I've had the pleasure of working directly with incredible people on products like our website that garner 200 million views annually.
Danielle Bezalel:
What?
Ambreen Molitor:
We...
Danielle Bezalel:
That's a very high number. We need to really highlight that.
Ambreen Molitor:
Yeah. And then we have our chat service, which is the sort of hybrid service that offers AI and trained educators to answer sexual health education questions. It's about anywhere from 400 to 700,000 people annually. And this product is offered anonymously, free, and very much, you know, accessible to all. And then finally, I also work with a team that oversees SpudOn, which is our period and birth control tracking app. That's had about two and a half million downloads, and it's available both in English and Spanish. And so together, all of these things, these products bring close to two million people to book an appointment on our site to a Planned Parenthood Health Center.
Danielle Bezalel:
Wow, wow, okay. And that is that kind of, how do you come up with the goals of like how many people kind of view these products and services and apps versus like they convert, right? It's called conversion in the biz. How
Ambreen Molitor:
Yeah.
Danielle Bezalel:
do you convert people from looking at something to actually working with a Planned Parenthood professional?
Ambreen Molitor:
Yeah, I mean, most of that direction is actually we lean into whatever is most needed for the user. So sometimes folks come in and they just simply want to learn. And when it feels appropriate, we come in and say, you know, if there's something more that you need to learn about, you can do that with a clinician or a provider or health care professional. So we're able to, within these, you know, experiences, whether you're like reading content on a website or tracking your period or menstrual cycle, or talking to an educator and have particular questions, we don't like, you know, pop out a banner and say, right now is the time to book
Danielle Bezalel:
Right.
Ambreen Molitor:
an appointment. It's very much more embedded very organically, very seamlessly into either a conversation or experience. So we're very mindful of balancing, you know, the way of meeting people at the right moment and getting them to book an appointment and then not feeling so inauthentic when we do that too. to find balance.
Danielle Bezalel:
Absolutely, I have to say the way that you're describing your job very much is how I would have described my dream job Like five years
Ambreen Molitor:
Good
Danielle Bezalel:
ago
Ambreen Molitor:
luck.
Danielle Bezalel:
like you're living like my and many other people's actual dream Like I got into sex ed and I was like I'm gonna make like sex ed and we're and do you know media stuff I'm gonna create videos and content and Using AI and like figuring out how to reach people that way is so Like, do you have a lot of clout in, like, your family and your friend circles that you think you have the coolest job ever, or is that just me?
Ambreen Molitor:
Um, I think my parents are still struggling to figure out what I'm doing.
Danielle Bezalel:
Okay, they're like, we're trying to understand.
Ambreen Molitor:
Um, they were like, there's tech and there's like care and there's Ambreen. She's some, she's doing something. Some combination of that is really just succeeding. I think in a very, you know, I think most of my family and friends are just very proud that I'm, I found a path from like being a tech nerd and found a home where a tech nerd can. also be part of a movement. So I think that's, I think that in the fuzziest way, what I feel most people are excited and proud about. I don't know if they know the details as I'm telling you this, but yeah. And the highest level, that's what, they're excited that a tech nerd has a job here, so.
Danielle Bezalel:
Correct. Yeah, tell me, tell me like how you got here. Like what, what got you into this work? Did you have kind of an aha moment where you were like, this is when I want to be in the field of sexual health, like coming from tech or tell me, tell me all about that.
Ambreen Molitor:
So my sort of story is very, it came from a very personal narrative that got me here. I had a myomectomy in my late 20s and for folks that are listening and that don't know what it is, it's basically a surgical procedure that removes these benign tumors that are called fibroids and they oftentimes increase chances of infertility. So, you know... In my late 20s, getting pregnant wasn't like something that was top of mind for me then. But once I knew what was happening to my body, I was kind of like, oh, hell no. You know, I'm going to be the most informed human about my reproductive
Danielle Bezalel:
Hmm.
Ambreen Molitor:
health. And I wanted to make sure I was gonna get pregnant on my own terms, which I have, by the way. I'm a mom to a very sweet three-year-old boy. Thank you.
Danielle Bezalel:
Oh, congratulations, that's so sweet.
Ambreen Molitor:
And my body was just, it was so healthy. And I was so happy throughout my entire pregnancy. And I think a lot of that empowerment came from, and I'm just not even exaggerating when I say this, it was reading Planned Parenthood's website. It was downloading SpotOn. So I was like our super user and I just love what was being built and I wanted to be a part of it. And so I wanted to help people in the same way that the organization was helping me, when I was in this moment in... in my early 20s, really understanding what empowerment means. And that is through like just knowing your body, just educating yourself. The other thing is I also had an abortion and to have that right was life-changing for me.
Danielle Bezalel:
Hmm.
Ambreen Molitor:
So casually all to say, I'm in this fight with Planned Parenthood for the long haul. I mean, a lot of what we bring, whether that be the services or the tools that we built, we're what... you know, made me plan my family and, you know, empower me throughout my journey. And so that's, that's my personal, you know, connection to Planned Parenthood and how I got really interested in figuring out a path towards how I could find a career here.
Danielle Bezalel:
That's very powerful. I don't think many people have as personal of a story. First of all, thank you for sharing. And I think the field of sexual and reproductive health is really unique in that sense. Like when we're talking about our bodies and our experiences and our relationships with ourselves and with other people, it can get really, really personal. And so it makes a lot of sense why it would feel so. empowering, I imagine, for you to be a super user all the way to the person who's kind of running, you know, in charge of these programs. Is that how you would describe it?
Ambreen Molitor:
Yeah, yeah, you said that much more eloquently, but yes.
Danielle Bezalel:
No, I think I think you you basically said that I just am kind of repeating but it just I just really relate with that I think it a lot of people who come to this field come from personal experiences whether that be from motherhood or pregnancy or birth control or abortion or STI is whatever it is and I think a lot of it comes from at least from what I chat with guests about I Didn't have this as much as I wanted And in your case, you know, Planned Parenthood is amazing. I'm also a super user of Planned Parenthood site. As a sex educator, I use the content constantly and reference it. And so I think it's really, really important. Yeah, just in general, that we kind of recognize that folks who are in this field, like a lot of us do have a personal connection to it.
Ambreen Molitor:
Yeah, and I think there's a connection to the curiosity, right? We want to dig deeper. And so I think that's the sort of similarity I find with my colleagues, too, is we're just kind of going deeper into the why, and surface level isn't enough. So yeah.
Danielle Bezalel:
Totally, yeah, absolutely. I wanna talk a little bit about the way in which you design these programs and products. So, in doing some research on you, it seems like you and your team use human-centered design to build products. And I'm wondering if you can talk about, a little bit more about your products and kind of a one-liner on each of them and what people use them for, and what human-centered design actually looks like and how it's realized.
Ambreen Molitor:
Yeah. OK, so maybe I could start by talking about what human-centered design is. And then maybe within that, I'll give examples of how we do that in our products. And then I'll dig deeper. So human-centered design has been a common approach in a lot of tech or design-driven organization. And it's the approach to designing usable and useful products. And you do that by focusing on what the user wants, what the user needs. And then you layer it on top with human factors, which we call usability in the solution. I think I used a lot of words that are more tech speak, so I'll translate them soon. But bear with me here. So another way to describe it, it's often the antithesis of a business center design. So business center design is where the bottom line is revenue and the question is, what's in it for the business? human-centered design says, what's actually best for our user? What does a user need? And inherently, the answer to that question brings revenue to an organization. So it's sort of flipping the question of, what's the bottom line? How are you going to make money? And you don't even ask that, because inherently, when you ask, what does a user really need, and what are we going to do about it, that answer is the same answer to the bottom line. But you're just actually. putting the user's needs ahead of the business when asking that. So OK. So there's conversations around user needs and the usability piece. So I can give an example about what we mean when we talk about user needs. So Spot On, which is our period and birth control tracking app. And I'm going to give a little bit of context so people can wrap their heads around the product. It's, as I mentioned before, is our period and birth control tracking app. And the way that it was built, we focused on user needs. So when we built it in 2016, there were already, I'm gonna say at that point, maybe like dozens, if not hundreds of period tracking apps already in the app stores. But a lot of people were sort of targeting it, even still, if you look at the way that period tracking apps are advertised, they're talking about fertility, about getting pregnant. But what you miss out on is this opportunity to talk about why people wanted to track their periods and acknowledge that people were sexually active or wanted to track their periods and did not want to get pregnant. And so when we did that, you almost instantly, the first year we had millions of downloads and a lot of people saying, this is great. I've seen a lot of period tracking apps. I've downloaded them myself, but you could tell they're catered towards someone that really wants to get pregnant.
Danielle Bezalel:
Mm.
Ambreen Molitor:
I don't, I actually want to prevent pregnancy, but I still want to track my menstrual cycle, want to intake birth control. I also want to just understand my body more. And so that was us kind of looking at the landscape and saying, okay, well, everyone's doing period tracking apps around fertility, but what we're hearing from the users is, this would be cool if actually you had things that helped me not get pregnant, right? And so that's how we flipped the model. And to this day, we... continue to get awards. I mean, I think it was like just this week, we got like the five, six of the world's best period tracking apps.
Danielle Bezalel:
Wow.
Ambreen Molitor:
You know, we mentioned as one of the, we continue to get those awards. And it's always because we are thinking about a particular group of users where there are not a lot of period tracking apps that are focused on pregnancy prevention. And the other thing I said about human-centered design was the usability piece, right? So we say. How can we build something that has users front and center, and then we make sure that it's usable, essentially? And so the other example I can give you, and this is for our website, for our chat bot, even for Spot On, for us, usability means accessibility. That means for every digital product we build, we also think about the kinds of people, that kinds of information that people deserve and they're seeking about their sexual and reproductive health. So that means making sure our products are free. We try to make it as accessible as possible and making sure that for as many products as we build, we can be able to sort of website and spot on, for example, or in Spanish. And then we make sure we are doing a good job of talking in a way that feels very accessible and it doesn't feel like you're talking to a doctor, but like if you engage with Rue, for example, it feels like a friend. If you engage with Spot On, it feels like an informed. you know, ally of yours versus you're talking to a doctor as well. So we think about accessibility in many of those concepts, and that's where we, you know, sort of focus on usability, which is like, can someone get this? What are the barriers for someone to get this? And for us, it's cost. For us, sometimes it could be languages. And, you know, there's a lot of conversation around stigma and shame, and we, the way we combat that is thinking about tone and language as well.
Danielle Bezalel:
makes a lot of sense. Yeah, I am thinking a little bit about spot on as well. And you know, after the Dobbs decision, right, and like after Roe v. Wade was overturned, I think I heard a lot of rubbings and saw a lot of rumblings on social media about period tracker apps and about people being worried about how their data was being leaked or used potentially against them. And I wonder if you could talk a little bit about data privacy with SpotOn, because I think that folks listening, if you're in these spaces of sexual and reproductive health, that was very present for a lot of people. And I think that deterred a lot of people from wanting to share that information. And so I wonder if you could talk a little bit about those policies and data privacy.
Ambreen Molitor:
Yeah, well, I won't get into the policies as much, but what I could tell you at a high level for, first of all, this is a really good question we get asked this all the time. And so I'm really happy that you asked that. I will say for SpotOn, we built it, as I mentioned many years ago, and we thought about data and privacy back then. And I remember people being like, oh, you're just being overly cautious. You don't need to do this. SpotOn can be used. And people have been using our product for like over five years and they have never registered their account. We don't link it to social media because it's free. You're not linking it to your bank account because, you know, so there's a lot
Danielle Bezalel:
Hmm.
Ambreen Molitor:
of opportunities for spot on, um, that allows you to remain anonymous and we will never know, and we do not intend to like poke at it or do not want to know if that's not something you feel comfortable with. The other thing is, um, in terms of data and privacy, there are products, you know, social media is free. But the way that there's an opportunity where people sell your data, we do not sell your data and don't ever intend to. So I think in terms of where Spot On stood in this moment when people were very afraid of what was happening in a post ops world, we were okay. We had a lot of downloads, a lot of appreciation because we already had built in all of these features and safeguards. We... allow people to remain anonymous. We don't have many integrations that are needed. And so you are able to safely use the app as intended. The other thing I worry about just long-term about building this sphere is we're just, it is so important, as I just said in my personal narrative, to be empowered to know what is happening to your body. And... to not be able to track it or not be able to have a system in which you're trying to understand and learn about your body is a loss of allowing someone to feel empowered. I think not being able to do that is also just a loss in women's health in general. And
Danielle Bezalel:
Mm.
Ambreen Molitor:
it's really important for everyone to be able to track their data in a meaningful way and
Danielle Bezalel:
Totally.
Ambreen Molitor:
feel safe and in part to do so.
Danielle Bezalel:
Yeah, the anonymity thing is huge. And like the the free thing, right? I think like most period apps aren't free, right? Like, how could they be if their business model is to be able to survive where Plant Barrenhood, you know, is this dynamite, really well known 100 year old organization. And that's amazing that you're able to keep it free.
Ambreen Molitor:
Yeah, we're a nonprofit.
Danielle Bezalel:
Of course, makes sense. Makes sense. Okay, so let's talk a little bit about the most common questions that you get in you know, whether that be the spot on have like a question availability to or is that just Roo and the chat text.
Ambreen Molitor:
Yeah, it's with Roo and Chat Text. Those are our other products.
Danielle Bezalel:
cool. Yeah, so Roo is a sex ed chat bot for teens, but I've used it, everyone can use it, you don't have to be a teen.
Ambreen Molitor:
Thank you.
Danielle Bezalel:
And then there's Chat Text, where trained health educators are available to answer your sexual health questions and help you come up with a plan. If you're listening, please remember Roo, R-O-O, and chat slash text. Refer everyone you know to these free resources, because I feel like people... would use them if they knew about them. And I think it's just important to try to get it out there to as many people as possible. So there's, I'll get off my soap box now because these are very important programs. But I'm wondering, what are the most common questions that you get? And then as a follow-up, I'd love for you to talk about how Gen Z, how the younger generation, right? So teens, people in their early 20s, are gaining access to sex ed info differently. than maybe someone like me who's a millennial or Gen X or older generations.
Ambreen Molitor:
Yeah. So the common questions we get from Roo versus Chat Text, and just to kind of dig deeper about what Roo does versus what Chat Text does is, so Roo, as you mentioned, is our AI-powered chat bot, and it's designed to help teens, young adults visit PlannedParenthood.org and get personalized sexual health questions instantly. And it's available 24 hours a day, seven days a week. And you're able to ask questions in a safe and anonymous way. So there's no sort of digital trace of the questions you're asking or the conversation you're having. And Rue will answer all of those questions in a welcoming, non-judgmental tone. And the answers that Rue gives are actually oftentimes no longer than the size of a tweet. So 280 characters are roundabout. And that's specifically designed so we can remain succinct. and so that it's easy to consume, going back to usability and accessibility. So what's really interesting about Roo is as more users interact with Roo, it's going to evolve and it'll be able to answer even more questions on more topics and learn more about what people are looking for and how they ask questions online. And chat text is the component of Roo. So there's also a... contain within the conversation when you're having with Roo or just separately, there's an opportunity for you to talk to an educated, trained professional about the same sort of sexual health questions you have online. That's our chat texts offering that we have. And you could do that, again, through the website, or you can text PP now to get those answers. And so from both sides, to answer the question about what are the most common questions,
Danielle Bezalel:
Mm-hmm.
Ambreen Molitor:
this is gonna be really fascinating. When we looked at all of the questions that we were being asked, we looked at it and we sliced and diced it in so many different ways. We looked at it from location, suburbs versus urban places, race, ethnicity, sexual orientation, gender identity, all remarkably the same. There was no distinction between each. But when you sliced and diced it by age, there was a distinct differentiation amongst the common questions that were asked.
Danielle Bezalel:
Oh.
Ambreen Molitor:
So, yeah. So if you were younger than the age of 15, you were more concerned about your body and things that you were wanting to do or about to do. So questions around what different terms mean, a lot of questions around puberty, and then a lot of interest and consent, which I think is a very healthy trend,
Danielle Bezalel:
Right?
Ambreen Molitor:
right? How to ask someone out, how to talk about. how you like someone with your community, whether that be friends or family. So you see a lot of that in the younger age. Folks above 15 were actually gathering information about what you can do based on something you've already done. So when it, there are questions that are indicating, so something like, when should I take a pregnancy test?
Danielle Bezalel:
Mm-hmm.
Ambreen Molitor:
Can you get pregnant if your partner pulls out? I missed my period, things like that. You know, like those are sort of the questions we start seeing. I don't want to say that these questions are not even asked by younger folks. They are, but I think it's sort of framed in the sort of situational versus, you know, the what's the common word? I'm trying to like hypothetical versus situational,
Danielle Bezalel:
Exactly,
Ambreen Molitor:
right?
Danielle Bezalel:
yes.
Ambreen Molitor:
So that you can kind of draw those conclusions, but age happens to be the differentiating, like there's almost like a change in the tone and the way people are asking the types of questions you're asking. Yeah, those are generally the, if I could like, I mean, we've got millions of questions. I'm just trying to like synthesize.
Danielle Bezalel:
That makes sense I think like you know if you're thinking about it young teens like I think when I was in public health school I learned that the average age of first like PV sex in the US was seventeen right so if you're thinking
Ambreen Molitor:
Mm-hmm.
Danielle Bezalel:
about that average then it kind of makes sense that fifteen you know fourteen fifteen sixteen year olds are saying I haven't had sex yet and I'm wondering what to do when I want to whereas people who are a little older in their young twenties. I already did that. I had sex, what do I do? And so that timeline totally tracks. But I think specifically around, there just still to me is so much misinformation about pregnancy and about missing your period and about when to take a test. We don't get as many DMs anymore because I think I try to make it really clear, hey, I can’t answer every single DM. And I do refer people to Planned Parenthood. I also refer them to a really awesome app called OK So that's geared towards teens. And
Ambreen Molitor:
Mm-hmm.
Danielle Bezalel:
you know, it's the same premise where it's like, OK, you can ask this trained group of professionals and they will answer you. And chat text is another another resource that I share with folks. But it's kind of like, no, like the second after you have unprotected sex, you are not going to be able to tell if you are pregnant. Like you need to. Either, you know, if you're wanting to take emergency contraception, here are these options, and here's what that looks like, and here are the percentages of how effective each of these things are. But I just find that there is so much panic, of course, right, around that question. Do you find that with the data that you have, is that something that you try to make as much, like a lot of content around? Or what does that look like for you?
Ambreen Molitor:
Yeah, there are very, there are multitude of ways of asking that question, right? Like, if you skin it, if you like trim it down, the heart of that question is, am I pregnant?
Danielle Bezalel:
Am I pregnant? Right.
Ambreen Molitor:
Right? And there are literally, I think at this point, probably hundreds of different ways that people are coming in and asking us, right? And so that is what we spend a lot of time focusing on, but as we do with consent and all of the other questions. But the ways in which people ask particularly that particular question of, am I pregnant? It stems from so many different ways. And we just talked through those examples of like they did send the pullout method. They're like missed their period or, there's just so many different ways that people are intending to start or their mindset is coming from a different place or where they are. So yeah, we spend a lot of time trying to, both with the AI and then also our trained educators, trying to unpack, you know, whether that is like in a curious state or in like a very high, where it's timely, right? Where the question to answer needs to be timely, needs to be very non-judgmental. We train both a bot and a human to think about all of those circumstances and contextualize that question as much as you can, right?
Danielle Bezalel:
Yeah.
Ambreen Molitor:
And in a way that it feels safe and they don't feel like... any sort of shame in starting to open up and get deeper into that, to the core of that question as well.
Danielle Bezalel:
Right and I guess my like my follow up what I which I didn't do a very good job explaining but like does it make you feel that like this content needs to get to them earlier like that there should be more kind of emphasis for the 1415 16 year olds be like hey you you might wonder about this in the future
Ambreen Molitor:
Thank you.
Danielle Bezalel:
and like here's you know extra content on that that we wanna kind of push towards that population I don't know just I'm wondering if there's kind of like. a reaction not only to the individual with the right answer, but I'm wondering does it make us want to push a little further and get there before that even happens?
Ambreen Molitor:
Yeah, I mean, I think there's so many different ways to, we talk about intervention, right? Like so many different ways to introduce that. And one of the common ways is sex education in a classroom, but you're not really in the moment at that time. So there are opportunities where you can do that proactively. And we've tried many different things, everything from like influencer marketing to... really thinking about, you know, even when they're engaging with Rue, we always have the opportunity of like, we asked, do you want to learn more? And there's, you know, we provide proactive ways for them to keep learning and poking and going deeper.
Danielle Bezalel:
Mm-hmm.
Ambreen Molitor:
But that's the secret sauce, which is like, where, what's the fine line and when and how do we do it in a way that feels like this
Danielle Bezalel:
authentic.
Ambreen Molitor:
is timely, appropriate? Yeah, exactly.
Danielle Bezalel:
Absolutely.
Ambreen Molitor:
And it doesn't feel like, again, you're just kind of pushing an agenda or pushing anything, you know, towards, towards that.
Danielle Bezalel:
Yeah, I imagine like many things, it's like you don't really think about it until it happens to you. And so it's really hard to get people to care about how to recognize, like, appropriately what happens when the condom breaks when the condom hasn't broken yet for them. It's tough.
Ambreen Molitor:
Yeah. Yeah, but when you see an inkling of interest, like maybe there's a way to, to, yeah, there's like so many different experiments that we do constantly. But yeah.
Danielle Bezalel:
I'm sure. I would love to transition to AI for a second, artificial intelligence. For those of our listeners who don't really know much about AI, I think chat GPT has entered the chat, so to speak, people
Ambreen Molitor:
Thank you.
Danielle Bezalel:
are really interested and aware because it's in the news and people are playing around with chat GPT. And there's a lot of articles about how AI are going to take people's jobs. And you know, there's a lot of fear, I think, along with excitement for AI right now in our current day. And I'm wondering if you can just give us kind of like an AI 101 lesson and like how it works for Planned Parenthood.
Ambreen Molitor:
Yeah, OK. So I'll start with the definition. So AI is a piece of technology that learns to mimic what humans quote, unquote, think and speak. And how it does that is it processes these large data sets. And these data sets could be code, words, numbers, images, for example. And they process it faster than the level of what a human can even do. or even large groups of humans could possibly do. And then they start recognizing patterns and make predictions based on what it's learning. And they can do that in fractions of a minute or even seconds. And the easiest way to compare that is like, think about when you had to learn a language or even like a baby learns to talk. That takes years, right, for a human to like be able to... really be able to communicate and start, you know, mimicking patterns to start feeling like they're having a conversation with someone else. AI can do that in seconds, minutes sometimes. And what that can do, so once, you know, AI has the ability to do that, it can do like so many different things from, you know, you could, it could help you think of a recipe from existing food you have in your fridge. It could write your college essay. It could help businesses scale by mitigating some sort of repetitive staff workload or repetitive tasks that staff are doing. And you know, for us that the latter piece, which is like, how can we scale by reducing repetitive tasks, you know, we're not interested in being part of the burnout culture, we're actually not interested in that. And so that's why we started exploring AI. So at Planned Parenthood, we're using AI, and I would like to say very well, I would add, because we've acknowledged in the category of sexual health, like over, you know, chat text has been around for almost a decade, if not over, and we've been able to monitor like millions of conversations, and we can at this point cover and anticipate the majority of questions being asked, and we can train our chat bot, Rue, to answer, you know, we have a can bank of about I want to say like a range of around 750 plus-ish questions. And we recognize that there are multitudes of different ways to ask those questions. And so therefore, you start multiplying the possibilities. Like the question we just talked about, am I pregnant? Like people can type that, or simply there's just like 100 different permutations of that question. And so we start, you know, we built on this foundation of 10 years of conversation and kind of categorized in a way and summarized. there's really only a few hundred different flavors of questions being asked. Multitude of permutations, but we have found like the base of it. And that's been like, that's been the secret to our success for how we've used AI. We've been able to launch within a year and a half, we worked on it, we launched it. And a year after we launched, we had users saying, We have a 90% accuracy rating. So 90% of users that are asking a question have said, this is, you know, Ruz answered my question to my expectations. It answered it correctly. I feel like, I feel great about the experience too. And then, you know, the other thing I will also say is we don't just let the AI bot kind of go with the flow. We're constantly, like humans are actually training the bot, retraining the bot, constantly rewriting rules. And they're actually telling the bot what to say and how to say it. So our bot is only successful because of the human component of it. And we, depending on the spike in the number of conversations we have, we have people looking at it multiple times a day to multiple times a week. And we're looking at it over and over again, just to ensure medical accuracy, to make sure that the tone isn't getting lost as it's like retrained. because it retrains itself and then we have to retrain it too. So we're just doing a layer of double checking. And that's essentially how, going back from what AI does to what we've been able to do. And the way that we've been working, our solution for success was just not letting go of it and letting it do its own thing. But we've been sort of, it's like our partner. It's just like another colleague that we're working with and we continue working alongside it. So. That's how we've been using it.
Danielle Bezalel:
Are you ever surprised by an answer for some, I mean it's created by humans so maybe not, but is there anything surprising going on on the back end?
Ambreen Molitor:
I think the only thing that has been surprising to us, which shouldn't be, as I say that loud, given how much time we spent with it, is that I think a lot of people feel way more comfortable talking to a bot than a human. In certain, in specific, on specific questions, right? And I think it's generally tied to questions that they feel might be stigmatized. So it allows people to get a little comfortable and just get into the spot where they want to go to versus... That's the only sort of differentiation we've seen or something surprising. But again, I don't, you know, it's also, we're talking to a generation that is very comfortable and grew up on digital. So I think that's probably their norm and how they feel most, like they're just more well-versed in that than in IRL sometimes or with humans. So it feels like a little bit natural to just get into it with. technology than a human sometimes.
Danielle Bezalel:
Yeah,
Ambreen Molitor:
So yeah,
Danielle Bezalel:
yeah.
Ambreen Molitor:
it's been really surprising.
Danielle Bezalel:
Yeah, I'm just thinking about my own use. I feel like I am not of the norm, but I always wanna talk to a person. Like if I have a customer service issue, right? And like Planned Parenthood's no different, right? If I need a question answered, I'm thinking
Ambreen Molitor:
Mm-hmm.
Danielle Bezalel:
about it the same way, right? As if I'm whatever, with the dentist, whatever it is. And I always want to go on the phone and I always wanna speak with someone. And I feel like my... always like, why are you always like calling. Like, don't like that's my nightmare. Like, why would you know? And I think it's interesting. Like, are you someone who likes using bots as someone who is the, you know, master of them, creator of them?
Ambreen Molitor:
That's such a great question. I think I am more comfortable messing up. And so when I feel like I'm having a conversation where I'm not an expert, I am so okay to be digital. But when I have to show up as someone that is either like, yeah, you know, in specific customer service situations where I feel like I'm the expert, I definitely want a human. But like when I'm like, I don't, I don't know, like, the text friendly, I just don't want to be there.
Danielle Bezalel:
Totally.
Ambreen Molitor:
Yeah, and I think a lot of it also, too, for me, what makes me feel more comfortable on digital is like the timing. You know, I don't have a half hour during my day to kind of just get on
Danielle Bezalel:
Wait.
Ambreen Molitor:
a call sometimes. So yeah, I think maybe that's when I would turn to it. But it's a good question. Yeah. For me, I think it's like where my layer of expertise lives. That's when I'm like, I'm going to get on this call. But if not,
Danielle Bezalel:
Yeah,
Ambreen Molitor:
I'm like...
Danielle Bezalel:
I...
Ambreen Molitor:
Hahaha
Danielle Bezalel:
right, right. And it's really important again, to point out like the access that people can have with a bot, right? I mean, it just kind of makes me think of have you seen the movie Her with Scarlett Johansson?
Ambreen Molitor:
Yes.
Danielle Bezalel:
and Joaquin Phoenix? Like that's happening with us, right? I mean, like that, you know, that movie is over a decade old, I think. But you know, he... spoiler alert, if you haven't seen the movie and want to, at the end of the movie, he kind of asks his robot girlfriend, like how many other... relationships are you in? And she says like, this astronomical number, right? Of like, 1,576,000, it's just kind of like, that to me relates to this, where it's like, one bot can be talking with quote unquote, millions of people at once when like, there aren't enough customer service reps or Planned Parenthood educators who would be able to reach all of those people. And so
Ambreen Molitor:
Yeah.
Danielle Bezalel:
I'm wondering, like, other than that. How might the use of AI improve sex education for people of all ages, but especially young people? And is there
Ambreen Molitor:
Mm-hmm.
Danielle Bezalel:
anything that we as users and consumers of content, while we're chatting with AI and bots, that we should be skeptical of?
Ambreen Molitor:
Yeah, okay. So I think, so I will say what I have seen, as I mentioned to you, right? Like chatbot technology is lowering users' fears in some way and their willingness to discuss information is just far greater than if it were to be with a human. So I think that is interesting and something, you know, if in the area of sex education, there's a there there, that's something to pay attention to, right? I think the other value add or something to think about as you go into, you know, what I like about the bot too is you can go to it whenever you need to, which is like that sort of, you know, where we're just talking about what's the right time and the right place. This is where technology is awesome because you just kind of sign on and it's there versus, you know, in a sex education course you have that foundation and you can kind of refer back to it through memory. But it's... the just-in-time conversation that you absolutely needed. That's the win with tech. I mean, not just chat about technology, but just technology in general, that's the win. I think the younger generation is just, as we just talked about, is much more comfortable online. We did a research with, it was in the height of the pandemic, but it'll be interesting, and we continue to have the same conversations. I remember distinctly a lot of folks we were talking to saying, I feel so comfortable online because there's an edit and a delete button, and there's not in real life. And I think about that a lot because to me, it seems that they feel like technology is very forgiving, right? You can type in a Google search and it doesn't have to be spelled correctly and you can mix up the words and you're fine or something like that, right? But when you want to ask that question in real life, maybe you need to know the pronunciation or you need to, you know, so I think there's something there that is deeper than, that gets to the core of what they're thinking about, which is, you know, they're very comfortable in that space. they're all, yes, yes, precisely. I think the other thing that they're all very, very aware of, and I will also talk about something that when we were building, Ru, and I never will forget this interview, there was this really amazing junior high kid, and we were talking about, you know, why don't you just search for these questions on Google versus, you know, you're talking to Ru. And he was like, well, what if I'm a president? Like, what if I want to run for president, and Google knew exactly what I searched for? Like, I don't to get into that. I was like, you are 12. You are amazing. I do hope you become the president! Yes. But that's what they're thinking about, right? Which is they know the concept of a digital trail. And I think they're very aware of their data and privacy. And with Roo, as I just mentioned, you can just go to our website and there's not a digital, on the browser, it doesn't show all the questions you've asked or the things that you're thinking about. And so there's that anonymity. We don't collect or ask any. personal information and so I think there's the safeguard that they feel really good about when they're talking with Root, which I think is different from other chat bots or chat technologies where they are upfront being like, give me your email and with your email, there's a lot of attachment to what you've been doing. So I think that's the thing that I would say, people are already paying attention to and thinking about. So,
Danielle Bezalel:
And how about that skeptical question? Is there anything, I mean, again, you're the creator of it, hard to shit on it for lack, you know, criticize it, but I do think that it is valuable to consider that as well because I do think there is a lot of fear from folks just around like robots taking over, right? And like when we kind of really lower that and. talk about it in the context of sex ed, like is there anything we should be fearful of?
Ambreen Molitor:
Yeah, I mean, I do, I'm skeptical of like the data and privacy pieces, right? Like Roo happens to be one of the bots that isn't collecting information, but there are a lot of, there are a lot of bots that can make it so seamless. If you just provide this one piece of information loaded with a lot of information about you, like you instantly have the opportunity of giving too much weight and unwillingly or unknowingly. And so that's the thing that I would, I, I would be careful about. Um, But I think, as I mentioned, I think a lot of people already know that, the younger generation especially. The other, I'm trying to think if there's any others, I don't think, I'm skeptical of AI taking many people's jobs, but I do think, I think people should look at it as an opportunity as we have, which is like, I'm not think of it as a threat, but like, how can this be an opportune moment to like, buy me more time ultimately to be with my family and not do these. this redundant bullshit work that I'm doing. So I think that's the sort of mindset I would be in when it comes to AI in general. But I think there's the sort of careful, cautious kind of steps to take, which is like, there's an opportunity where, as we're doing with Rue too, we're training the bot. There's still the opportunity for humans to kind of control and think about it, which I think is like the... the exciting part, but I think there's also the other part, which is like you also, there are opportunities where you may, there are holes where you can lose control and you just need to be aware of that. And so that's the skepticism. And that's not different than what we're seeing in the many headlines today too, which is like how and who sets those boundaries and can we all collectively live in a world where it's like we focus on the betterment of what AI brings versus the sort of harm that it could bring.
Danielle Bezalel:
Interesting. Yeah, I mean that's a good point that like humans run it and humans have faults and can be discriminatory and all of those things and so of course if they're training the bot then that's what the bot is gonna be it listens to the human I'm wondering what exciting projects you're working on right now if you want to share any
Ambreen Molitor:
Yeah, I mean, so innovation's been expanding a lot of our strengths to lead transformation work throughout the org. So, you know, the things that we've been mentioning are really focused on how we scale what we do. And, you know, a lot of the things we're working with innovation now, especially with like academics and some startups that we're partnering with, is really pushing to build social transformation and social impact much more broadly. That takes a little bit more time, right? It's easy to be like... 200 million people came to our site, Annali, but to change social impact takes a little bit more time. So we're spending a lot of time on the innovation team, really thinking about that impact story and like, what does it mean to really change for the better for social good, essentially.
Danielle Bezalel:
Yeah, that's really, really important. I'm super curious and excited to see how that comes to fruition and what that looks like. Ambreen, this has been so fun and wonderful. And like I said, you have my dream job, so it is very
Ambreen Molitor:
Thanks for watching!
Danielle Bezalel:
cool that I'm able to chat with you. And I'm.
Ambreen Molitor:
And otherwise, I love what you're doing too. This is kind of fun. You get to like hang out with amazing people all day.
Danielle Bezalel:
I sure do. Yeah, it's really, really special. And yeah, I'm wondering if you can share where folks can find you, you know, where to find these programs that you're talking about and how they can access all these amazing resources.
Ambreen Molitor:
Yeah, I mean, if you go to planparenthood.org slash learn, you'll be able to find all of the great resources that we've been talking about. You know, we have hundreds of webpages with all this content and a lot of them link you out to the products that we have. To stay in touch with me specifically, I will be posting on LinkedIn. So find me on LinkedIn and stay in touch. I'll be posting about our projects, developments that we're doing in the innovation side and you know, really thinking about, as I mentioned right now, putting a lot of effort into thinking about impact and measuring impact. So, yeah.
Danielle Bezalel:
Absolutely. Well, thank you again for being on. It's been an absolute pleasure chatting with you. So thank you.
Ambreen Molitor:
Yeah, thank you.