Background to my current dilemma. I am working on recommendation systems. One use case we work on is sending targeted campaigns.
We have been working on a POC that would predict which customers were more likely to purchase product X. The goal would then be to have the product X campaign target only those users who have a high likelihood to convert and thus, produce a better conversion rate.
Additionally, those who have no interest with product X could receive something else more targeted to them. We can only send a limited number of campaigns per month, so you’d think this idea would go over well.
However, after our initial clustering of the data, we met with our stakeholders to present and get feedback. We learned they are not interested in targeted campaigns anymore because they don’t want to reduce the audience size and limit exposure. They point blank said their strategy this year is “send to as many people possible in hopes of converting as many people as possible.”
So, my question is: Do we continue to work on the POC and try to persuade them with the potential benefits, or should we focus efforts elsewhere? I don’t want to waste my teams time and we could explore other things, but I know there’s that saying about customers don’t really know what they want - you know like the one about Henry Ford and how people would’ve asked for faster horses or something.
To summarize, I have a POC for a new feature that ideally benefits stakeholder and helps achieve their goals. Stakeholder doesn’t seem interested because it goes against their strategy, and I would need them in agreement to be able to validate the feature. So, there’s no hope to persuade with data. Do I continue with hopes of them agreeing to at least test the idea or just move on?
I have worked product in a movie studio, and they were never interested in anything that was not Shiney. They would constantly reduce budget for infrastructure and platform while dumping money into VR experiences which our users did NOT ask for… some companies won’t listen… and some will. Such is life.
Build it and prove the value…because the company failing due to bad product, but you get to say “I told you so” isn’t worth it
“SHOW THE MONEY” nobody will say no to money.
Why is conversion rate your key metric? Optimizing conversion doesn’t mean you’re optimizing revenue.
If I were you and getting pushback, I’d ask myself are there bigger opportunities for impacting the key metrics that drive business? It’s the same pushback I get internally when I talk about recommendations and personalization and it’s not wrong.
But like another person said, if you truly believe they’re wrong you need money to convince them. Either run your own campaign for a large enough size to show the difference (can be hard with internal politics or such). Or get them to carve out a small batch of users that you believe would benefit from the treatment.
@NathanEndicott, I have no control over the campaigns being ran because my product is just logic / the back-end services that deliver them. I would need the stakeholders to agree to changing the targeting rules of their campaigns. Also, conversion rate is not my metric. It’s the business.
Personally, I’m not a big fan of trying to hyper personalize content in this role and definitely think there’s more value elsewhere. But my leadership insists personalized content is what we need
@AnaRodriguez, Then you’re in a tough spot. I’ve been there too. I think that this point if you think the POC proves the value you need to focus on individual stakeholders. I’d stop or slow the development investment on the POC and meet with stakeholders one on one. Get their concerns and then get them involved in planning solutions to those concerns. As you get into the weeds with these people then focus on driving the message to leadership of their concerns and what it’s going to take to assuage them.
At some point it’ll go one of two ways either you’ll break down barriers and get a test run or you’ll have enough evidence to push up and make the leadership work it out.
When you have to have group presentations of the project make sure you’ve been over and over things with people. Even send pre-reads to leaders. Do your best not to have a question catch you off guard.
Can you expand on this a little further? Wont optimizing conversion rate directly impact Revenue? I’m sure there are other ways too, but this approach isn’t wrong
It will but there’s a couple things that factor in here.
The new program increases the cost of running multiple campaigns, tracking reporting and design aside from just the standard work of maintaining the model and infra.
Pushing multiple products means you have multiple net revenue per acquisition depending on which product you’re converting on. So, a lower conversion rate on a higher revenue generating product will often create a higher net revenue than just increasing conversion on a few simultaneously.
Segments with enough data to meaningfully personalize may be so small that they provide almost zero gross impact.
Not my current job but at my last I ran our recommendation engine and found that the number of targeted users with a generally desirable offer was better than trying to segment and use a mix of more offers.
Plus, one offer is simpler for retargeting which is also effective.
If they’re really just a stakeholder like a marketing person who is going to be running the ad campaigns later, you need to convince someone above their level or tie it to a different strategic goal that is out of their area to justify it. E.g., If you’ve done a RACI chart and they are just ‘informed’ but being loud, you can tell them this is what’s best and you’re going ahead.
But… Are they just stakeholders or are they the people paying your salaries and bearing the financial consequences of the decisions they make? You don’t get to spend other people’s money on stuff they didn’t ask for, even as an employee thinking you are doing your job. Even if it means them losing money, it’s ultimately their choice to make. They also might not be telling you the whole strategy if it’s commercially sensitive how exactly the company maximizes profits. That’s not to say you can’t build it, but rather: If you can convince them, you should build it. If you can’t, definitely don’t.
Getting alignment on ideas is a critical part of the PM role and you should see this as challenge, to convince people, and only go ahead if you succeed. You definitely should not skunkworx it as a PM - while engineers can sometimes say they still did their main work while skunkworxing something only while blocked and justify it, as a PM being part of that would mean you’ve failed the key part of your role which is ensuring the engineering teams are delivering value in areas the business has approved business cases for.
@RichardsonEva, It is the business development team who own the sales of the different products they are trying to advertise through various campaigns. Marketing team just writes the copy, but the bds own how they’re distributed.
Honestly, I’ve realized this was kind of a dead end and knew we could add more value elsewhere outside of content recommendations. The only reason I started making attempts is because my leadership kept pushing for “content personalization” so I was trying to make do with the options available. I feel like now that I’m getting confirmation from other stakeholders that they’re not interested, I might finally have some leverage to continue exploring other ideas. I just wanted to get feedback on if you should actually kill a feature just because someone isn’t interested lol
@AnaRodriguez, Yeah, totally depends on who the someone is! Sounds like you might be able to justify it by aligning to a higher-level strategy though. Good luck!
Do you need them to go ahead?
@NaomiNwosu, Yes, because they own the targeting rules. For example, I would need them to agree that we aren’t going to send this campaign to 2 million people, but actually only 1 million.
We still don’t know if the model actually works. It needs to be validated and the only way we could do that is through a live experimentation.
@AnaRodriguez, Does it even matter if the conversion rate is high? If this is digital and the cost of engagement is zero why even do segmentation, unless you will have a different more targeted message.
I guess the segmentation would be important because you can only send x number of messages a month. Ideally you would want to optimize that spot for the maximum return. If you aren’t strategic about who sees what message it could be wasted “real estate”
Since you have the stats and ml folk on your team; how small of an experiment could you send to show that better targeting leads to hire sales?
Also, curious if your company gets paid by the email rather than by the value of the conversion for an internal or external customer? If you get paid by the email sent than you need to find a way to show that better targeting leads to better sales leads to sending more emails. Maybe you can charge more for sending more targeted and therefore more successful emails?
The best way to go about this is to do it as an ab test. If the audience for your solution is small, I can see why the stakeholders might not be very interested.
My problem is I can’t do AB testing unless they were on board with it. I have no control of the campaigns getting created. I only work on the logic / back-end service that delivers them.
We could build logic that would display the campaign to the “optimal” audience, but they need to sign off on having a reduced audience reach.
I’m not super convinced this is a great feature. Personalization and segmentation is smart but optimizing for conversion rate over revenue isn’t smart. Plus, if you build this, you need the stakeholder to implement the strategy/use the feature, right? If you have no say in making sure the feature gets used, I wouldn’t build it. But if marketing is on board to use the feature if you build it, then I would go ahead.
@FelipeRibeiro, Yep! That’s my exact dilemma. We can build it, but unless they agree to let me test and validate the model is working with a live campaign, we’ll never know.
I personally want to scrap it and move on because I’m not a fan of content recommendations lol. I just didn’t know if it was wrong of me to move on!