Background to my current dilemma. I am working on recommendation systems. One use case we work on is sending targeted campaigns.
We have been working on a POC that would predict which customers were more likely to purchase product X. The goal would then be to have the product X campaign target only those users who have a high likelihood to convert and thus, produce a better conversion rate.
Additionally, those who have no interest with product X could receive something else more targeted to them. We can only send a limited number of campaigns per month, so youâd think this idea would go over well.
However, after our initial clustering of the data, we met with our stakeholders to present and get feedback. We learned they are not interested in targeted campaigns anymore because they donât want to reduce the audience size and limit exposure. They point blank said their strategy this year is âsend to as many people possible in hopes of converting as many people as possible.â
So, my question is: Do we continue to work on the POC and try to persuade them with the potential benefits, or should we focus efforts elsewhere? I donât want to waste my teams time and we could explore other things, but I know thereâs that saying about customers donât really know what they want - you know like the one about Henry Ford and how people wouldâve asked for faster horses or something.
To summarize, I have a POC for a new feature that ideally benefits stakeholder and helps achieve their goals. Stakeholder doesnât seem interested because it goes against their strategy, and I would need them in agreement to be able to validate the feature. So, thereâs no hope to persuade with data. Do I continue with hopes of them agreeing to at least test the idea or just move on?
9 Likes
I have worked product in a movie studio, and they were never interested in anything that was not Shiney. They would constantly reduce budget for infrastructure and platform while dumping money into VR experiences which our users did NOT ask for⌠some companies wonât listen⌠and some will. Such is life.
Build it and prove the valueâŚbecause the company failing due to bad product, but you get to say âI told you soâ isnât worth it
9 Likes
âSHOW THE MONEYâ nobody will say no to money. 
8 Likes
Why is conversion rate your key metric? Optimizing conversion doesnât mean youâre optimizing revenue.
If I were you and getting pushback, Iâd ask myself are there bigger opportunities for impacting the key metrics that drive business? Itâs the same pushback I get internally when I talk about recommendations and personalization and itâs not wrong.
But like another person said, if you truly believe theyâre wrong you need money to convince them. Either run your own campaign for a large enough size to show the difference (can be hard with internal politics or such). Or get them to carve out a small batch of users that you believe would benefit from the treatment.
7 Likes
@NathanEndicott, I have no control over the campaigns being ran because my product is just logic / the back-end services that deliver them. I would need the stakeholders to agree to changing the targeting rules of their campaigns. Also, conversion rate is not my metric. Itâs the business.
Personally, Iâm not a big fan of trying to hyper personalize content in this role and definitely think thereâs more value elsewhere. But my leadership insists personalized content is what we need
7 Likes
@AnaRodriguez, Then youâre in a tough spot. Iâve been there too. I think that this point if you think the POC proves the value you need to focus on individual stakeholders. Iâd stop or slow the development investment on the POC and meet with stakeholders one on one. Get their concerns and then get them involved in planning solutions to those concerns. As you get into the weeds with these people then focus on driving the message to leadership of their concerns and what itâs going to take to assuage them.
At some point itâll go one of two ways either youâll break down barriers and get a test run or youâll have enough evidence to push up and make the leadership work it out.
When you have to have group presentations of the project make sure youâve been over and over things with people. Even send pre-reads to leaders. Do your best not to have a question catch you off guard.
7 Likes
Can you expand on this a little further? Wont optimizing conversion rate directly impact Revenue? Iâm sure there are other ways too, but this approach isnât wrong
7 Likes
It will but thereâs a couple things that factor in here.
-
The new program increases the cost of running multiple campaigns, tracking reporting and design aside from just the standard work of maintaining the model and infra.
-
Pushing multiple products means you have multiple net revenue per acquisition depending on which product youâre converting on. So, a lower conversion rate on a higher revenue generating product will often create a higher net revenue than just increasing conversion on a few simultaneously.
-
Segments with enough data to meaningfully personalize may be so small that they provide almost zero gross impact.
Not my current job but at my last I ran our recommendation engine and found that the number of targeted users with a generally desirable offer was better than trying to segment and use a mix of more offers.
Plus, one offer is simpler for retargeting which is also effective.
6 Likes
If theyâre really just a stakeholder like a marketing person who is going to be running the ad campaigns later, you need to convince someone above their level or tie it to a different strategic goal that is out of their area to justify it. E.g., If youâve done a RACI chart and they are just âinformedâ but being loud, you can tell them this is whatâs best and youâre going ahead.
But⌠Are they just stakeholders or are they the people paying your salaries and bearing the financial consequences of the decisions they make? You donât get to spend other peopleâs money on stuff they didnât ask for, even as an employee thinking you are doing your job. Even if it means them losing money, itâs ultimately their choice to make. They also might not be telling you the whole strategy if itâs commercially sensitive how exactly the company maximizes profits. Thatâs not to say you canât build it, but rather: If you can convince them, you should build it. If you canât, definitely donât.
Getting alignment on ideas is a critical part of the PM role and you should see this as challenge, to convince people, and only go ahead if you succeed. You definitely should not skunkworx it as a PM - while engineers can sometimes say they still did their main work while skunkworxing something only while blocked and justify it, as a PM being part of that would mean youâve failed the key part of your role which is ensuring the engineering teams are delivering value in areas the business has approved business cases for.
5 Likes
@RichardsonEva, It is the business development team who own the sales of the different products they are trying to advertise through various campaigns. Marketing team just writes the copy, but the bds own how theyâre distributed.
Honestly, Iâve realized this was kind of a dead end and knew we could add more value elsewhere outside of content recommendations. The only reason I started making attempts is because my leadership kept pushing for âcontent personalizationâ so I was trying to make do with the options available. I feel like now that Iâm getting confirmation from other stakeholders that theyâre not interested, I might finally have some leverage to continue exploring other ideas. I just wanted to get feedback on if you should actually kill a feature just because someone isnât interested lol
5 Likes
@AnaRodriguez, Yeah, totally depends on who the someone is! Sounds like you might be able to justify it by aligning to a higher-level strategy though. Good luck!
5 Likes
Do you need them to go ahead?
4 Likes
@NaomiNwosu, Yes, because they own the targeting rules. For example, I would need them to agree that we arenât going to send this campaign to 2 million people, but actually only 1 million.
We still donât know if the model actually works. It needs to be validated and the only way we could do that is through a live experimentation.
4 Likes
@AnaRodriguez, Does it even matter if the conversion rate is high? If this is digital and the cost of engagement is zero why even do segmentation, unless you will have a different more targeted message.
4 Likes
I guess the segmentation would be important because you can only send x number of messages a month. Ideally you would want to optimize that spot for the maximum return. If you arenât strategic about who sees what message it could be wasted âreal estateâ
3 Likes
Since you have the stats and ml folk on your team; how small of an experiment could you send to show that better targeting leads to hire sales?
Also, curious if your company gets paid by the email rather than by the value of the conversion for an internal or external customer? If you get paid by the email sent than you need to find a way to show that better targeting leads to better sales leads to sending more emails. Maybe you can charge more for sending more targeted and therefore more successful emails?
3 Likes
The best way to go about this is to do it as an ab test. If the audience for your solution is small, I can see why the stakeholders might not be very interested.
2 Likes
My problem is I canât do AB testing unless they were on board with it. I have no control of the campaigns getting created. I only work on the logic / back-end service that delivers them.
We could build logic that would display the campaign to the âoptimalâ audience, but they need to sign off on having a reduced audience reach.
1 Like
Iâm not super convinced this is a great feature. Personalization and segmentation is smart but optimizing for conversion rate over revenue isnât smart. Plus, if you build this, you need the stakeholder to implement the strategy/use the feature, right? If you have no say in making sure the feature gets used, I wouldnât build it. But if marketing is on board to use the feature if you build it, then I would go ahead.
1 Like
@FelipeRibeiro, Yep! Thatâs my exact dilemma. We can build it, but unless they agree to let me test and validate the model is working with a live campaign, weâll never know.
I personally want to scrap it and move on because Iâm not a fan of content recommendations lol. I just didnât know if it was wrong of me to move on!